Security professionals are constantly ingesting threat reports, write-ups, and blogs on various defensive and offensive techniques. These include detailed write-ups on actual intrusions.

Here, we’d like to continue this trend, but with a slight twist.

Rather than present a neat timeline of events (although we’ll do this too), we’ll approach this particular write-up from the point of view of the actual analyst performing the investigation for this case.

Every intrusion is different, and so too are the investigative paths that unravel them.

Join us as we walk you through one of these paths!

‍

It Begins

Although we may think of intrusions as linear, with the attack progressing neatly from step one to step two and so forth, that’s rarely how the intrusion comes into focus from an analyst’s point of view.

Often, an analyst will receive a signal or an alert for an intrusion that’s already taking place.

For example, an analyst may receive an alert about lateral movement. In this case, we can assume that some kind of initial access and perhaps credential dumping techniques were also in play, as these are often necessary prerequisites for lateral movement.

This dynamic can be illustrated by the image below:

Intrusion timeline, with a defined start and end — *Figure 1: Illustration of intrusion timeline, with a defined start and end, and analyst awareness in the middle.*

In this particular case, the analyst received a number of signals, all clustered around the same time frame. These signals contained evidence of:

Addition of a local user to the affected host
Adding the newly created user to the local administrators and Remote Desktop users’ groups
Modification of the affected hosts’ registry to facilitate further lateral movement via the Remote Desktop Protocol (RDP)
Modification of the affected hosts’ registry in order to bypass User Account Control (UAC)

*Figure 2: Analyst view of initial signals*

‍

At this point, the analyst has some key pieces of information:

The threat actor is somehow within the network—the initial access method is not yet known.
The threat actor has administrative credentials to the affected host, as those are necessary to add user accounts and modify the registry.
The threat actor seems to want to move laterally throughout the network.
The threat actor has established a means of persistence via the newly created user account.

During investigations, an often paradoxical dynamic plays out.

More information sometimes leads not to further clarity, but only to further questions.

We may know that the threat actor is within the network, and we may also know the methods of lateral movement, but key pieces of the intrusion are still missing.

To unravel this case further, we need to continue our investigative process.

‍

Pulling on a Thread

One thread that we can pull on is parent-child relationships within the process telemetry that’s available to the analyst.

Although there are a myriad of ways to create a user on a Windows host and add that user to a particular group, if these actions were performed in a normal fashion by a systems administrator, the process lineage would look something like:

Explorer.exe spawns a command prompt or a PowerShell prompt
Command prompt or PowerShell spawns net.exe or net1.exe
net.exe or net1.exe has a command line similar to something like: net localgroup Administrators <user> /add

In addition, if a systems administrator was legitimately modifying RDP configurations, we’d expect these events to come from some kind of system management process.

If the systems administrator performed these actions via command line, we’d expect to see a process lineage similar to our [.highlight]net localgroup[.highlight] example above.

In this case, an examination of parent-child relationships yielded interesting results:

Parent-child relationship indicating lateral movement via WinRM — *Figure 3: Illustration of parent-child relationship indicating lateral movement via WinRM*

‍

The registry manipulation commands outlined above were spawned from the [.highlight]wsmprovhost[.highlight] process. This indicates the threat actor was moving laterally within the environment using the WinRM protocol.

This piece of information is interesting for several reasons. Firstly, it provides us with a pivot point. We know that this particular threat actor is moving laterally through the network using WinRM—among other methods—so it would make sense for us to review any other processes spawning from [.highlight]wsmprovhost.exe[.highlight] within the environment.

Secondly, this information lets us know that the threat actor most likely moved laterally to the affected host from somewhere else, so we need to keep pulling on this thread to find the original source of the lateral movement.

Finally, this information tells us that the threat actor has credentials that grant them access to various network hosts.

This is an unfortunate dynamic that we often encounter. Once a threat actor is inside the network, there are typically not many controls in place to prevent lateral movement. In most cases, administrative accounts, particularly local administrator accounts, have rights to most—if not all—of the assets within the network, including sensitive assets like domain controllers, file, and application servers.

With the above information noted, we continued to go down the path of examination of parent-child process relationships. This examination yielded results. As we kept pulling on this investigative thread, we discovered that the parent processes for all the user manipulation activity outlined above spawned from the suspicious [.highlight]Veeam.Backup.MountService.exe[.highlight] process:

Suspicious parent process of Veeam.Backup.MountService.exe — *Figure 4: Image of suspicious parent process of* ***Veeam.Backup.MountService.exe***

‍

Very interesting and somewhat concerning indeed! Why are these commands spawning from the Veeam Backup service? How did the threat actor gain access to this system or application?

Armed with this new information, the affected victim host was enumerated further to see if a vulnerable version of Veeam Backup & Replication existed. Sure enough, we found that the installed service was out of date and missing patches.

The Huntress team has blogged about this vulnerability in detail before, and we all know that patching is difficult, so it’s not at all surprising to see older software out there.

After some further investigation of the host, a PowerShell script artifact was found via the PowerShell Operational log channel, event ID 4104:

*Figure 5: Image of PowerShell script exploiting CVE-2023-27532*

‍

This script was used to pull credentials from the vulnerable Veeam service, and these credentials were ostensibly used in order to gain code execution via the vulnerable service.

With this knowledge in hand, we now have a deeper understanding of the intrusion for sure, but are still in the dark about how the threat actor:

Gained a foothold into the environment in the first place (Initial access)
How was the vulnerable Veeam instance located? (Discovery)

To answer these questions, we need to switch gears slightly, from looking at process telemetry to looking at Windows Event Logs!

‍

To the Event Logs We Go

Due to the amount of malicious activity Huntress observes, we have a unique vantage point and can extrapolate indicators of compromise from the various intrusions we come across.

One such indicator is workstation names. Certain workstation names are observed across intrusions, and in this particular case one such known malicious workstation name popped up on our radar.

This malicious workstation targeted a handful of user accounts with a brute force attack that lasted about two minutes prior to, unfortunately, gaining access to one of the accounts on the network. This activity took place for an internal IP address, suggesting that this activity was nearly adjacent to the initial access vector, but not the initial access vector itself.

Event logs illustrating a successful brute force — *Figure 6: Image of event logs illustrating a successful brute force*

‍

The intrusion is now coming into slightly more focus.

We now know that:

A threat actor landed in the environment and performed a brute force attack.
This brute force attack was successful and led to credentials being compromised.
These credentials were used to gain access to a system in the environment.
On this system, a Veeam service was exploited for further credential theft.
Follow-on activity on this system entailed the creation of a user account in addition to registry modification to facilitate further lateral movement.

A few questions remain, however. We still don’t know how the threat actor got into the environment in the first place, and we also don't know how the Veeam service was discovered. Was this service specifically targeted or was it spotted in an opportunistic fashion?

When pivoting from the malicious workstation name, we found some interesting authentication events that contain a target user of “[.highlight]ANONYMOUS LOGON[.highlight]”—the brilliant security researcher Lina Lau has a fantastic blog on what these authentication events mean. While they’re logged as a successful authentication, that isn’t always necessarily the case.

A better way to think of these events is more in line with an attempt at successful authentication. However, just because the authentication fails doesn’t mean it doesn’t yield interesting investigative information.

When we examined the authentication events containing these “[.highlight]ANONYMOUS LOGON[.highlight]” type events, we discovered another interesting machine name that contained the name of the popular scanning tool [.highlight]nmap[.highlight].

Authentication attempts with a machine name of nmap — *Figure 7: Image illustrating authentication attempts with a machine name of* ***nmap***

‍

We now have another piece of the puzzle, and some potential answers for how the Veeam service was discovered in the first place. The word “potential” here is extremely important, as often the data available is incomplete, fragmented, or often simply missing.

In this case, evidence is circumstantial at best, as all we have to go on is the machine name, which can be easily changed. In addition, even if these events did occur from an [.highlight]nmap[.highlight] execution, we don’t know that the scan itself targeted Veeam specifically.

Being comfortable with gaps and unknowns is a huge part of dealing with these kinds of intrusions. Although we all wish every system was instrumented with all the various telemetry-related bells and whistles like Sysmon and full packet capture, unfortunately this isn’t the case in the real world.

At this point, only one piece of the puzzle remains—the initial access vector.

‍

Uncovering the Initial Access

In our previous initial access blog, we highlighted an externally facing VPN appliance as an extremely popular initial access vector. This case is no different, and VPN compromise is strongly suspected as the initial access vector.

Pivoting off the IP address associated with the lateral movement activity, the Huntress Tactical Response team reviewed the provided VPN telemetry and found a suspicious authentication event. However, since the VPN device doesn’t hold a lot of telemetry, we were only able to identify a log-out and not a log-in event:

*Figure 8: Image illustrating VPN telemetry analysis*

‍

Similar to our above [.highlight]nmap[.highlight] execution, evidence of this is also circumstantial and non-complete. This dynamic illustrates a very common, yet not so much talked about element of intrusion analysis and incident response: dealing with unclear or missing telemetry. In this case, we can surmise that the VPN appliance was the initial access vector, but unfortunately, we’re lacking definitive evidence of this.

Now that the full picture of the intrusion is much clearer, mapped to the MITRE ATT&CK framework, and laid out in operational order, our intrusion will look something like the below:

*Figure 9: Timeline of events within the intrusion*

‍

Wrapping Up

We hope that you enjoyed this write-up of one of the investigations that the Huntress Security Operations Center (SOC) & Tactical Response team handled.

On the investigation side, we aimed to highlight methodology where possible. It’s one thing to generically state “monitor for brute force attempts” or “look out for anomalous parent-child process relationships,” but it’s quite another to actualize and implement such investigative techniques. We hope this inside look at our approach helped solidify and clarify these concepts.

‍

MITRE ATT&CK Mapping

Tactic	Technique ID	Technique Name	Description
Initial Access	T1133	VPN Compromise	Compromise of VPN appliance
Reconnaissance	T1595	Active Scanning	Use of nmap
Credential Access	T1110	Brute Force	Brute force attack against existing accounts
Lateral Movement	T1021.006	Remote Services: Windows Remote Management	Lateral Movement via WinRM
Execution	T1203	Exploitation for Client Execution	Exploitation of Veeam Service
Persistence	T1098	Account Manipulation	Adding User to Remote Remote Desktop Users Group
Defense Evasion	T1112	Modify Registry	Enable Further RDP Lateral Movement