This post highlights triage data acquisition in the context of attack/defend CTFs such as Pros Versus Joes. These concepts carry over to real world DFIR practices, but CTFs provide a bit of leeway when it comes to legal and regulatory matters. At work, you should be following pre-defined procedures which have been approved by leadership.

What is Forensic Triage

Similar to triage in the medical profession, forensic triage is the process of collecting, analyzing, and prioritizing data from an investigation.

When responding to an incident, it is imperative that you preserve as much data about the system’s current state as possible, as quickly as possible. From the context of a security analyst, you would probably want to have data which unearths malware or illicit activity; evidence of process executions, network traffic, browser and shell histories, file system activity, etc. Ideally, this kind of telemetry is collected prior to an incident, but this often isn’t the case.

Organizations victimized by cyber crime often focus on evicting the perpetrators from their systems and getting their business back up and running as quickly as possible. This set of priorities leads to IT staff destroying evidence unwittingly by powering off machines, killing or deleting malware, rebuilding systems, reverting virtual machines, and so on.

Making a byte for byte copy of a drive can be labor intensive and expensive. An incident involving 10 systems, each with 1TB drives, would require at least 10TB of storage to clone these drives, plus however many gigabytes of RAM these hosts have for full memory dumps. Furthermore, most of the information on a disk is not even relevant to an investigation, which ultimately makes storing everything a wasteful approach.

Luckily, most of the artifacts an analyst would need to review do not take up much space. Often, this data is only a few megabytes versus several hundred gigabytes. The process of triage data acquisition can take seconds rather than hours required for to full disk copies. Collecting only the forensically relevant items on a system enables analysts to get to work quicker, prioritize attention given to a particular host, and protects against data destruction as a result of “friendly fire”.

Real World Example

At every attack/defend CTF I’ve participated in, players have made the mistake of finding malware and immediately killing or deleting it rather than properly archiving and reviewing it. While killing the running processes and deleting files may remove the attacker’s code, this ultimately isn’t the best move. In many cases, basic analysis of the malware can lead to the discovery of additional infected systems or provide an entry point into another team’s systems.

An example scenario is an analyst who discovers a web shell and simply deletes it because they know it is malware. Since it has been deleted, we now only know the filename and host which it was being served from. Checking opposing teams’ hosts, we were able to observe that the malicious file was still being served, but we had no idea how to leverage the shell. Reviewing our web server’s access logs, we were not able to observe it being utilized by the attackers. In short, we knew that other teams were vulnerable, but we could not exploit them. If we had a copy of the malicious code, we probably could have figured out how to use it offensively.

Collecting triage data as a first step when the game starts is a powerful strategy to avoid these sorts of mishaps. Web server directories should be collected during the triage data collection process.

It can also be much easier to work with artifacts locally versus remotely over a flaky RDP or SSH session to a host which likely has a rootkit installed on it. Having the data locally means that you don’t have to install forensic analysis tools and their dependencies on every single host, only your workstation.

What to Collect

Hashing out the details of what to collect and how to collect it can be a great team building exercise when preparing for an event. You should collect anything that may reveal exploitation of a system, the activities of the attackers, and persistence mechanisms. Here are some examples of what to collect:

• Volatile Data
  • Memory dumps
  • Netstat
  • Running processes
  • Open files/handles/mutants
  • Environment variables
  • …
• System Information
  • Hostname, IP address, …
  • System configuration settings
  • OS and software versions
  • Amount of RAM and Disk
  • …
• Filesystem Artifacts
  • $MFT, $Logfile, $USN, $Journal, and other filesystem-specific artifacts
  • Registry hives
  • Startup folder contents
  • Configuration files
  • Scheduled tasks, jobs, etc.
  • Prefetch files
  • Hashes of executables
  • Recent documents
  • Logs
  • Users home directories
  • …

How to Collect Triage Data

It depends. RFC 3227 Guidelines for Evidence Collection and Archiving provides some basic guidelines that should be followed regardless of the situation at hand.

CyLR is a nice tool for collecting triage data at CTFs. Alternatively, you can write your own software to gather and archive data. Be mindful that certain software will not work on all hosts. A host may have an older version of PowerShell, the wrong .NET framework installed, or be running on a different architecture than your tool supports.

Triage data should be collected as one of the first priorities in a game. This data must be transferred off of the victim machine as soon as possible and stored somewhere that the rest of your team can easily access.

Finally, be mindful that an attacker may have employed anti-forensics techniques against you.