Advanced Persistent Threat Targeting Vietnamese Human Rights Defenders

Executive Summary

It doesn’t matter if you’re a small organization, a non-profit, or a Fortune 500 company, there’s always someone who will want access to your information. In many instances, this access is primarily for financial gain; however, for many non-profits and small organizations the harsh reality is that the nature of their work, or their clients, also makes them an ideal target for intelligence gathering and espionage-motivated threat actors. 

Threat hunters at Huntress recently discovered an intrusion on a Vietnamese human rights defender’s machine which is suspected to have been ongoing for at least four years. This intrusion has a number of overlaps with known techniques used by the threat actor APT32/OceanLotus, and a known target demographic which aligns with APT32/OceanLotus targets. This post highlights just how far advanced threats will go for information gathering purposes when it aligns with their strategic interests.

Background

Huntress regularly performs threat hunting operations to find intrusions that may have slipped past normal security defenses. In a recent case, Huntress analysts identified an intrusion against a non-profit supporting Vietnamese human rights which has likely spanned the course of at least four years. While detections in the Huntress platform found some anomalous activity which was reported to the Huntress partner, the threat hunting team was able to find well-hidden persistence, and actions taken by the threat actor. This information was then used to piece the intrusion together and trace it back long before the Huntress agent was deployed. 

Hunting Methodology

Huntress is uniquely positioned to look for threat actors across millions of systems. This comes through the combination of process behavior insights and persistent footholds gathered from the Huntress EDR. Leveraging process behavior insights, threat hunters use intelligence, or a hypothesis, and their knowledge of what is normal on a system to create threat hunting rules. These rules differ from product detections as they are generally higher in frequency, and lower in efficacy given they target techniques used by threat actors who are trying to blend into an environment. Using created hunting rules, threat hunters often take three different approaches to threat hunting including looking for: rare hunting signals, multiple signal clusters, and statistical anomalies.

The Huntress Managed EDR consistently identifies persistent footholds on a system. This allows threat hunters to locate anomalies where a persistent foothold may be found on a small subset of the systems protected by Huntress. These anomalies could be a difference in persistence mechanism, name, binary, or another attribute to what is normally seen across other Huntress partner environments. Whilst investigating a new hunting signal, it was found that a system would infrequently and inconsistently run a small number of administrative commands from an unusual process. 

The admin commands run were deliberate and rarely exceeded three commands in a ten minute period, with a max of twelve being run on a system during any given day. Despite this, the unusual activity was enough to raise the attention of Huntress threat hunters who proceeded to look over persistent footholds in the partner environment and piece together the larger scale of this intrusion.

Investigation and Analysis

Host 1

Persistence Mechanisms

While onboarding to Huntress, host 1 presented with a scheduled task titled Adobe Flash Updater:

Scheduled Task 1

The referenced scheduler.ps1:log.txt, is an alternate data stream named log.txt within a file named scheduler.ps1. This file was already removed prior to the Huntress agent being deployed; however, the naming convention and use of an alternate data stream has some overlap with public reporting by Cybereason detailing a VBS and PowerShell-based loader used to load Metasploit and Cobalt Strike payloads.

In the following weeks, new scheduled tasks were created on the host and identified by the Huntress agent roughly 10 days apart:

Scheduled Task 2

Scheduled Task 3

This scheduled task contained a different user SID than the one found in the AdobeUpdateTaskUser scheduled task. The MSSharePoint.vbs script was designed to use a private key already placed on disk, authenticate to a remote SFTP server, and download / run a script called cloud.bat.

The cloud.bat file used the same private key to authenticate to the same remote SFTP server, and pulled down a file called cloudlog.txt.

At the time of investigation there was no cloudlog.txt file on disk. Modification timestamps on the private key, SFTP, and SSH binaries all indicate that they were possibly present since November 2023.

Less than a day later, schtasks.exe was used to create persistence that would run cloud.bat once every 5 hours.

Scheduled Task 4

Creation of the Handler scheduled task was later found to have originated from a DllHost surrogate process which was executing a DLL from a COM object stored in the registry with the identifier {1F7CFAF8-B558-4EBD-9526-203135A79B1D}.

It was found that this process was being launched from another scheduled task that was previously setup prior to Huntress deployment.

Scheduled Task 5

This task attempted to masquerade as the legitimate UpdateLibrary task on the system and had an identical description to the legitimate UpdateLibrary scheduled task also on the system. The task creation and modification timestamps indicate it was first set up in June of 2020. The StartBoundary within the XML file used for this Scheduled Task also had a timestamp value of 2020-01-01T00:00:00 indicating that the task was expected to be run from the start of 2020 onwards.

Although the scheduled task didn’t have an executable set to run, it did have a COM Handler that was to be invoked. Analysis of the host found a COM object setup using registry keys.

COM Object

This COM object DLL set to run was a signed, legitimate iisutil.dll used by IIS Express, which happened to match a rule created by Florian Roth from Nextron systems 5 years ago called APT_OceanLotus_ISSUTIL_Sep18. Although this match was a false positive, a malicious sample was found on VirusTotal matching this rule, which was submitted with the names iisutil.dll and iisutil2.dll.

This sample has been flagged by some AV engines as being tied to APT32/OceanLotus and has significant overlap with another DLL found on disk called iisutil2.dll. Further analysis of the DLL and 2 other files, which together act as a backdoor, are presented in the section: "Analysis of Malware."

A few weeks following the creation of these scheduled tasks, an enumeration command was observed on the host looking for current user’s privileges. 

whoami /priv

The next day, a forced restart was performed on a remote host. This same action was performed on another system roughly two weeks following execution on the first.

We don’t know the intent of this action, but speculate it may have been to ensure execution of malware on a remote system or to ensure any system configuration changes are applied.

Over the next few months, various discovery commands were performed to ensure access to remote workstations from host 1. Actions were taken to ensure network connectivity was still active on the host and remote hosts.

A run key was found on host 1 which referenced a McAfee OEM Module binary (mcoemcpy.exe) masquerading as WdiServiceHost. A DLL used for sideloading was not found at the time of investigation; however, public reporting by ESET is available which states that this executable is vulnerable to loading a malicious DLL named McUtil.dll.

Run Key 1

A second run key was found on host 1 referencing an Apple Software binary (SoftwareUpdate.exe) with a revoked code signature. This persistence mechanism was unique across Huntress customers and it’s believed this was used to sideload a malicious DLL. The DLL used for sideloading was not found at the time of investigation; however, public reporting by Recorded Future is available which states that this executable is vulnerable to loading a malicious DLL named SoftwareUpdateFilesLocalized.dll.

Run Key 2

Yet another run key was found on host 1 referencing a batch script called connection.bat. This had identical functionality to MSSharePoint.vbs except it launched PowerShell to run SFTP rather than a VBS script.

Run Key 3

Right before isolation occurred on this system, the threat actor was seen attempting to steal Google Chrome cookies for all user profiles on the system from the DllHost COM object backdoor.

cmd /c for /f "tokens=*" %G in ('dir /b "%localappdata%\Google\Chrome\User
 Data\Profile *"') do copy "%localappdata%\Google\Chrome\User
 Data\%G\Network\Cookies.bak" "%localappdata%\Google\Chrome\User 
Data\%G\Cookies" /y

Host 2

Persistence Mechanism

A separate host, host 2, had remote commands run via Windows Management Instrumentation to execute a batch script approximately 1.5 months after the first observed action on host 1. This batch script was used to query processes running on the host.

cmd.exe /c C:\Users\Public\Downloads\1.bat

The batch script content is below:

Domain Discovery commands were also observed on this system shortly after this.

net group "Domain Admins" /domain
nltest /dclist:<REDACTED>.local

The process which initiated this was a legitimate version of the calibre eBook management executable calibre.exe which had been setup to run as a task. Through Huntress telemetry, it was seen that a Scheduled Task was attempted to be created to run this calibre.exe executable from an unusual location.

schtasks /create /sc MINUTE /mo 300 /tn
 "Microsoft\Windows\WindowsColorSystem\Calibration_Update" /tr
 "C:\Users\<REDACTED>\AppData\Roaming\Microsoft\SPMigration\Bin\Calibre.exe
" /f

Scheduled Task 1

It should be noted that this is an attempt to blend in to the legitimate “Calibration Loader” task generally seen at C:\Windows\System32\Tasks\Microsoft\Windows\WindowsColorSystem\Calibration Loader. We speculate that the “Calibration Loader” task was chosen because of similar naming as the file calibre.exe.

Soon after this execution there was attempted privilege escalation via named pipes performed through the calibre process. This likely involved injection into the legitimate Windows gpupdate.exe process, which is a known process commonly injected into through the use of malleable Cobalt Strike profiles and is commonly seen when running the ‘getsystem’ command from Cobalt Strike.

Analysis of this host found the calibre executable running a malicious DLL called calibre-launcher.dll on disk; however, within a matter of minutes before the DLL and executable could be obtained the threat actor seemed to have killed the running process, removed the entire SPMigration directory including the implant. At the time of investigation, there was a suspicious entry still in the system DNS cache:

Although we weren’t able to confirm that this lookup was related to the intrusion in question, the domain was similar to one seen previously (msteamsapi[.]com) and the subdomain also had overlap with a subdomain seen on host 4.

Host 3

Persistence Mechanism

Shortly after performing named pipe impersonation on host 2, a command was run using the same Cobalt Strike beacon in an attempt to create a scheduled task on a third system. This scheduled task was set to run every 15 minutes as the SYSTEM user account (Note: the task name resembles a license key and as such has been redacted as a precaution).

schtasks.exe /u "<REDACTED>\<REDACTED>" /p "<REDACTED>" /S
 <REDACTED> /create /SC MINUTE /MO 15 /TN "<REDACTED>" /TR
 "C:\Users\<REDACTED>\AppData\Roaming\Microsoft\SPMigration\Bin\calibre.exe" 
/RU "NT AUTHORITY\SYSTEM" /K /f

Scheduled Task 1

Shortly after this, a command was run to invoke the calibre executable.

wmic /node:<REDACTED> /user:<REDACTED> /password:<REDACTED> 
process call create "cmd.exe /c start
c:\Users\<REDACTED>\AppData\Roaming\Microsoft\SPMigration\Bin\calibre.exe"

At the time of investigation, the executable and DLL weren’t found on disk.

Host 4

Persistence Mechanisms

Using available Huntress telemetry, a search was run to find any other instances where the calibre executable was set to run at startup. Three scheduled tasks were found on the system, two of which were masquerading as legitimate Adobe executables, with the other masquerading as a Microsoft update task.

Scheduled Task 1

Scheduled Task 2

Scheduled Task 3

Analysis of network connections on the system showed that one of the calibre executables posing as Adobe (AdobeARM.exe) previously had a network connection to a remote IP address.

Based on analysis of this infrastructure and malicious calibre-loader.dll files submitted to VirusTotal, this IP address and the calibre.exe implant were likely tied to a Cobalt Strike Team Server. 

Months after our initial detection on host 1, user privilege discovery was observed via a different calibre.exe process.

whoami /priv

Weeks following this command we observed a new service created to run a legitimate node executable. This executable was set to launch a malicious Node addon binary to evade detection on the system.

Service 1

The Node addon was created to specifically target the system and user account and included a hardcoded path to a file on disk at C:\Programdata\Adobe\ms-adobe.bin. This also included a hardcoded service name to be created called SrvAdobeUpd; however, at the time of investigation, this wasn’t found on the system. Analysis of network connections on the system showed that this node executable previously connected to a remote IP address.

Based on analysis of this infrastructure and the malicious node file, it’s believed that this was likely tied to a Cobalt Strike Team Server.

About a month following the Node addon being launched we observed a scheduled task creation spawning from the node.exe process.

This scheduled task was set to run a jar file which would run an embedded DLL into memory. 

Scheduled Task 4

Further analysis on msadobe.jar is mentioned in the following section.

Supporting Analysis

It’s most likely that this is only the tip of the iceberg and that the true extent of this intrusion stretches well beyond systems with the Huntress agent. Preliminary analysis was conducted into the malware found on these systems, and infrastructure used in the intrusion. This was done as a way of determining any known overlap with threat actor techniques which align with the target industry or demographic of the victim organization.

Analysis of Malware

This intrusion had several binaries and files which were involved. A summary of these files are included below.

During analysis of host 1, it was found that the legitimate cachuri.dll set to run as a COM object would explicitly import and run code from iisutil2.dll. Although iisutil2.dll had almost identical information as a signed, valid copy of iisutil.dll, this had been patched to run different code, and was modified to increase the file size above 50MB. It’s believed this was done to evade a number of YARA rules which often have file size constraints, and to prevent submitting the file to online sandboxing tools, many which have a file size limit of 50MB. This modification caused notable differences in the NT Header, Optional Header, and most significantly the .text section.

The entry point of this DLL had also been modified to offset 0x00025FB0 (155568) which differed from the original entry point of 0x00027FB0 (163760). A brief analysis of this binary showed it pushed the return address to the stack and then ran a function at 0x1002711e.

This is significant because these operations, the entry point, and the address of the function to be run are all identical to the previously mentioned malware submitted to VirusTotal which is tied to APT32/OceanLotus. A closer inspection showed that this file was actually identical to the sample on VirusTotal tied to OceanLotus mentioned earlier, with the only difference being data appended to it so that its file size grew above 50MB.

In contrast, the legitimate DLL would begin setting up necessary registers before having a branch condition depending on the arguments passed to the executable running the DLL.

The malicious DLL would then search the Process Environment Block (PEB) for a PEB_LDR_DATA structure so that it can identify the InLoadOrderModuleList. This structure contains a list of DLLs in the order that they were loaded.

The code includes multiple jump operations, such as the one shown in Figure 10, which would never be taken, or would only be used to run a small amount of instructions, before returning to the original flow of execution.

Interestingly, this malware contains a number of garbage op-codes and control flow obfuscation to throw off-static analysis and break disassembly. This overlaps with techniques known to be used by APT32/OceanLotus as previously reported by ESET.

This malware looks at the DLLs loaded and their exports so that it can dynamically resolve APIs used to facilitate decryption and injection of a payload into memory. This has significant overlap with malware reported by BlackBerry/Cylance called Steganography Loader #2.

Analysis revealed that this DLL would ultimately read in iisexpressshim.sdb, decrypt it using an XOR key of 0xFF, and then decompress the data using the LZNT1 compression algorithm. The decrypted iisexpressshim.sdb file showed more instances of junk op-codes being present which would never be evaluated.

The decrypted DLL in memory would then load logo.png, use a custom steganography routine, and then make a call to the Windows CryptDecrypt API to decrypt and load the final DLL into memory. The use of a custom steganography routine to hide malicious code in a seemingly benign PNG file, in addition to use of a XOR key and compression, has overlap with the previously mentioned Steganography Loader used by APT32/OceanLotus. It’s noted that there were a number of differences between this version of the Steganography Loader and the one previously reported which included use of LZNT1 instead of LZMA, and a hardcoded XOR key of 0xFF instead of it being retrieved from a file on disk.

The malware also had significant overlap with a sample analyzed by a security researcher back in March of 2019, and it’s highly likely both malware samples are from the same malware family. At the time of investigation, the host had active connections to 185.198.57[.]184 and 185.43.220[.]188 on port 8888 from the DllHost process running the COM object backdoor. 

Passive DNS information for the IP address 185.198.57[.]184 showed that domains mentioned in the security researcher’s blog from 2019 resolved to this IP address. This helps to validate that the malware described in their blog is the same malware found on this system 5 years later. It’s also worth mentioning that none of the domains appear to have lapsed or have been re-registered, and the domains were all originally registered in late 2017. This indicates that the below domains have likely been under control of the same threat actor for almost 7 years.

• cdn.arlialter[.]com - Domain originally registered: 2017-10-27
• fbcn.enantor[.]com - Domain originally registered: 2017-10-27
• ww1.erabend[.]com - Domain originally registered: 2017-10-27
• var.alieras[.]com - Domain originally registered: 2017-10-27

The domains also appear to masquerade as legitimate domains, which is notable given APT32/OceanLotus has previously used this technique throughout their intrusions.

The host was also found to have another four scheduled tasks which were masquerading as various services with identical descriptions. These tasks had a similar naming convention to previously seen scheduled tasks. In addition, a user run key also had a similar naming convention:

Scheduled Task 1

Scheduled Task 2

Scheduled Task 3

Scheduled Task 4

Run Key 1

Examining host 1’s scheduled tasks found another two instances of the malicious COM backdoor registered. These would no longer run the malicious code hidden within logo.png as the required malicious iisutil2.dll had been removed from the system. It’s suspected that multiple variants of the backdoor were established on the system over time to help ensure access remained even if AV products picked up on some of the existing backdoors.

Amongst the scheduled tasks was a DropboxUpdate task pointing to a legitimate executable. Although DropboxUpdate doesn’t directly import and use goopdate.dll, this is indirectly called and loaded by DropboxUpdate which is then used to load a malicious DropboxUpdate.bin file in the same directory as shown below in Figure 15.

Analysis of process memory found multiple domains and C2 configuration details for this malware:

These domains once again masqueraded as legitimate domains.

The malicious DLL goopdate.dll is more than 20MB in size and makes a check for a hardcoded GUID environment variable on the system. If it’s not present it will be set. This is done before setting memory permissions to RWX to allow injecting the .bin payload into memory.

Of note is that this DLL has a function at offset 0x0001010 which uses a hardcoded list of names in this injection routine. Specifically, it will take the last name in the array and concatenate it with all the other names which is then evaluated prior to injection.

No specific overlaps were seen with previously reported malicious goopdate.dll files used by APT32/OceanLotus. Despite this Facebook, Cybereason, and Volexity have all previously reported the use of APT32/OceanLotus using a malicious goopdate.dll which was loaded into a benign executable. It’s worth noting that this technique and DLL name is also used amongst other threat actors.

Examining the JAR files adobe.jar and msadobe.jar found these to be simple loaders that would run specific embedded DLLs into memory from a main class called UpdateData.

Looking at the DLL mi54giwp.dll found it would create a Mutex with the value okSSjZzAlnNOlQaGoDWx prior to targeting a .bin file located within a directory hardcoded into the DLL. This highlights the malware had been created specifically to target the system it was run on. 

Similar behavior was found on the the DLL zfhqq01v.dll which creates a Mutex with the value sbvjJpGLbbmnHNfWEetm prior to targeting a .bin file located within a different user account directory hardcoded into the DLL.

Whilst examining host 1 it was found that persistence had previously been set up to run a suspicious executable from a user run key. This executable was quarantined by Windows Defender.

Run Key 2

Of note is that the “TPM Console” directory had three files in it with varying modification timestamps which are of interest when it comes to timelining this incident.

Analysis of the quarantined TpmInit.exe found that this was a modified version of a legitimate TpmInit executable. This executable when initially run will create two files TpmInit.mdb and TpmInit.mdf on disk if they’re not present before terminating, at which point these files will no longer be modified.

Although it’s unknown whether this executable was related to the same intrusion, modification timestamps indicate this malware may have been present and running on the host since 2017. If both TPMInit.mdb and TPMInit.mdf are present when the executable is run, TpmInit.db (a DLL) is dropped from TpmInit.exe and run using rundll32.exe after first injecting into another rundll32 process. This file will have its modification timestamp change every time the executable is run, indicating a potential first and last time this malware was executed on the system.

To execute TpmInit.db, the malware leverages the legitimate rundll32 application to run an exported function called ‘TpmVCardCreate’. It’s worth noting that the exports in this DLL are named after a subset of exports found in a legitimate tpmvsc.dll usually found on Windows.

After execution, this would get a handle to kernel32.dll to get the address of modules to be used and check to see if Kaspersky AV was running on the system (avp.exe) and avg (avghookx.dll) as seen in Figure 28.

Analysis of Infrastructure

Examining the two suspected Cobalt Strike Team Server IP addresses found that both were signed with Let’s Encrypt certificates and were sitting behind a Cloudflare Load Balancer. Of interest is that the servers would present a 404 Not Found message with a Content-Length of 0 whenever a GET request with a URI containing a ‘/’ was sent. The servers would also present a 200 response with a Content-Length of 0, and the allowed methods OPTIONS, GET, HEAD, POST whenever an OPTIONS request was sent. This is significant because the same behavior is expected when you’re interacting with a Cobalt Strike Team Server as previously reported by Palo Alto Networks.

The combination of specific response headers and Cloudflare Load Balancer lead to a unique service banner which was seen across both of the suspected Cobalt Strike C2 IP addresses through a Censys search, seen in Figure 30.

A search for this banner found only seven hosts making this a fairly unique fingerprint. Looking for only hosts that were identified by both a name and an IP address found three unique IP addresses and domains, of which only one hadn’t been seen in this intrusion.

Interestingly, all of these IP addresses had domain names which looked to be masquerading as legitimate websites or software, and none of the ASNs or service providers overlapped.

Targeting and Attribution

It’s long been reported that journalists, bloggers, dissidents, and Vietnamese human rights advocates have been targeted by malware and tactics consistent with APT32/OceanLotus operations dating back to at least 2013. This has been reported by companies such as Google, the Electronic Frontier Foundation, Amnesty International, and a large number of other security vendors. During our investigation a number of overlaps were found between known techniques used by APT32/OceanLotus, the target verticals and interests of this threat actor, and what was found in this intrusion:

• The target was a non-profit supporting Vietnamese human rights
• The malware in question used a malicious DLL which was loaded by an IIS Express DLL named iisutil.dll. This has overlap with a YARA rule created by Nextron Systems that points towards the threat actor APT32/OceanLotus.
• The malicious DLL used in this intrusion used a modified version of iisutil with the entry point 0x00025FB0 (155568) and a function at 0x1002711e. All code in the malware is identical to malware uploaded to VirusTotal noted to be associated with APT32/OceanLotus besides extra padding appended to it.
• Port 8888 and 8531 were used within the malware C2 configuration. The COM object backdoor aligns with public reporting by a security researcher from 2019 where the final payload contained eight possible C2 server addresses with identical port numbers. 
• The use of hardcoded C2 addresses in a DLL resource has known overlap with malware used by APT32/OceanLotus as reported by BlackBerry/Cylance.
• The use of COM objects and Steganography using PNG files is a known technique reported to be used by APT32/OceanLotus as reported by BlackBerry/Cylance.
• Alternate Data Streams with the name log.txt were appended to a PowerShell script and loaded by wscript through a scheduled task. This has a naming convention similar to a publicly reported campaign attributed to APT32/OceanLotus ‘Operation Cobalt Kitty’ by Cybereason.
• Cobalt Strike is suspected to have been used by the threat actor by loading a malicious DLL into a legitimate executable, a known technique used by APT32/OceanLotus.
• Facebook, Cybereason, and Volexity have all reported the use of APT32/OceanLotus using a malicious goopdate.dll loading into a benign executable.
• APT32/OceanLotus has been known to use unique CLSIDs, Binary Padding, compression, and Scheduled Tasks in their intrusions as reported by ESET. The naming conventions used in their malware is also similar.
• APT32/OceanLotus has been known to use lots of unique domains and infrastructure with minimal overlap to help remain in environments for long periods of time which aligns with what we’ve seen here.
• APT32/OceanLotus has been known to incorporate Java-based malware into their operations.
• APT32/OceanLotus has previously used garbage op-codes in their malware to throw off analysis, and control flow obfuscation as reported by ESET.
• APT32/OceanLotus has previously used the McAfee OEM module to sideload malicious dll’s as reported by ESET.
• APT32/Oceanlotus has previously used Cobalt Strike servers behind Cloudflare as reported by Cybereason and Volexity
• APT32/OceanLotus has previously used the Apple Software Update binary to sideload malicious dll’s as reported by Recorded Future.
• APT32/OceanLotus has previously heavily used Let’s Encrypt TLS certificates in its infrastructure as reported by Volexity.

Indicators of Compromise

MITRE ATT&CK Mapping