The threat-hunting process
Threat-hunting is a multi-stage, cyclic process. Ideally, threat hunts are proactive, so the hunter doesn’t know what they’re looking for in the absence of a known threat. As a result, the first stage of the hunt is defining the purpose of the hunt. After a goal is defined, it’s possible to collect and analyze data and cycle through the phases of a hunt until a threat is detected or disproven. If a threat is detected, remediation and response are necessary to purge the threat from the system.
Defining the hunt
When performing a threat hunt, the first thing to do is to figure out what you are hunting. The wide variety of potential threats and the sea of potential data to collect means that an undirected hunt is likely to miss things. A series of short, well-directed hunts is much more likely to be successful that a single large, undirected one. When performing a proactive threat hunt, you don’t have a specific target that you’re hunting, which may make defining the hunt difficult. Two options for hunt definitions are data-driven and target-driven.
Data-driven threat hunting
A data-driven hunt begins by collecting a data set and then analyzing it for indications of a particular threat worth hunting. For example, a threat hunter may collect the network traffic logs for the enterprise Web server and look for items of interest. If an anomaly is detected, this is a good starting point for a more in-depth hunt. When trying to decide on a data set to start a hunt from, it’s often useful to consider the life cycle of a cyberattack. Lockheed Martin’s Cyber Kill Chain and MITRE’s ATT&CK framework are useful resources for this, since they describe the stages of an attack and the methods of achieving each stage. A data set that would allow detection of one or more threats may be a good starting point.
Target-driven
A target-driven threat hunt is geared towards determining if a particular threat is present in the network. Example targets include:
Indicators of Compromise (IOCs) for a known attack A particular attack vector from the MITRE ATT&CK framework The tools, techniques and procedures (TTPs) of an advanced persistent threat (APT)
Choosing a target provides a starting point for the search and a focus for initial data collection. As the hunt continues, new information (like evidence of a different threat) may arise that causes a change in focus.
Data collection
Good data is essential to good threat hunting. If a hunter is basing analysis on corrupted or incomplete data, the hunt may be beyond useless since it provides a false sense of security. The data collection stage is a vital one and will be revisited multiple times through the life cycle of the hunt. When performing data collection, it may seem like more data is always better. However, there are several reasons why this may not be the case:
Volume: The more data that is collected, the more data there is to process. This can require significant resources and, depending on the circumstances of the hunt, may cause unacceptable delays in the hunt Processing: Some processing techniques used in threat hunting (like grouping and stack counting) are more effective when working with smaller datasets Visibility: Advanced adversaries present on the network are more likely to detect and attempt to evade bulk data collection efforts
When performing a threat hunt, it’s better to focus on collecting the data that is necessary to answer the core question of the hunt. Hunting should be a continuous, cyclic process with previous hunts providing the groundwork and motivation for future ones. Another major consideration when performing data collection is the quality of the data being collected. An adversary with access to a certain machine may have the ability to modify log files and other local data sources to hide their presence. When performing data collection for the hunt, it’s important to balance ease of collection with the quality of the data produced and choose data sources and collection methods accordingly.
Data analysis
In many cases, the effectiveness of the threat hunt comes down to the quality of the data analysis. Most threat hunters have access to the same datasets, but the truly mature threat-hunting programs are the ones that can and do analyze the data in ways that are likely to detect the threats present on their system. The simplest and most common form of data analysis for a threat hunt is IOC analysis. Many organizations subscribe to a threat feed of IOCs for known threats and then search for these known IOCs on their network. While this can be effective for finding the low-hanging fruit, the existence of an IOC means that someone has already been breached and advanced adversaries are adept at creating attacks that don’t generate traditional IOCs. The next step up the ladder is analytics-based hunting. Hunters will feed collected data into anomaly detection and machine learning algorithms, which search for anything out of the ordinary. This type of analysis can be effective at detecting zero-day attacks; however, it’s a cat-and-mouse game when dealing with advanced adversaries who know the types of anomalies and correlations that can be detected and develop attacks that are designed not to produce these indicators. The most advanced threat hunters use similar analysis techniques but with a different focus. Rather than collecting data at a large scale, they focus on the common behaviors of their adversaries. The ability to detect a single, crucial stage in the attack progress can render an attacker’s complete strategy worthless. As shown in the Pyramid of Pain, TTPs are the hardest thing for an attacker to change. Completing a round of data analysis frequently doesn’t produce a yes/no answer about the existence of a threat. In many cases, it raises new questions to answer or demonstrates that more data and in-depth analysis are necessary to answer the question at hand. Threat hunting is a cyclic process, with each cycle increasing your level of knowledge and visibility into your organization’s threat surface.
Remediation and response
Reaching the remediation and response stage of the threat-hunting process is a bit of a mixed blessing. On the plus side, you’ve found a threat on your network. On the minus side, there was a previously undiscovered threat to find. Remediation and response can be a complicated process. Based on the level of sophistication of the adversary, the specifics of the attack and more, different techniques may be necessary to remediate the threat. Before entering the remediation stage, it’s important to learn everything that you can about the threat. This involves even more data collection and analysis but typically is more focused, e.g., looking at the registry, process or network communications of the malware. Once you have a complete understanding of the threat on your system(s), it’s time to remove it.
Conducting the hunt
The details of a threat hunt vary from environment to environment and hunt to hunt; however, some things are always the same. In order to effectively hunt in your environment, you need to know what you’re looking for, how to look for it and how to tell if it’s there. This requires access to the right tools for the job and an understanding of your environment, adversaries and the tools and techniques you will use. There are many available resources on threat hunting including courses, books, whitepapers and forums that are invaluable to a threat hunter seeking to improve their skills.
Sources
The Cyber Kill Chain, Lockheed Martin MITRE ATT&CK, Mitre The Pyramid of Pain, Enterprise Detection and Response
