I wanted to write such an article based on the interesting results of the EDR tests we have recently conducted at the request of our customers, because the outputs we obtained as a team caused us to seriously question this gigantic industry. EDR products, priced at thousands of dollars per client, are gaining a place in various institutions and organizations with the promise of comprehensive protection.

However, to give potential buyers an idea, many independent (!) security solution rating and testing organizations test EDR solutions using the techniques in frameworks such as MITRE ATT&CK and produce outputs that will be a reference in corporate marketing. But how true are the claims of these allegedly independent testing organizations and marketing teams? Do the products really deliver what is promised to provide enterprise-level protection, or are they inadequate? How effectively is the much-touted artificial intelligence technology really being used in the field of malware inspection?

Our 'Malware Research' team at Secunnix acquired some of the most well-known EDR solutions in the market and conducted various experiments with unique attacker methods. In addition to the experimental malware we developed within the scope of the tests, we also simulated how ransomware groups use techniques to circumvent/bypass EDR solutions when they take over a server.

Because it's a cat and mouse game, and if you are on the defending side instead of the attacking side, your job is to think about what the attacker can do and the methods they follow before the attacker and take the necessary precautions. So, throughout this blog post, we will briefly talk about what EDRs are, some of the methods used by attackers to circumvent them and what we do during the tests.

What is EDR?

Endpoint Detection and Response or EDR solutions, in shorter words, can be called computer software that can continuously monitor the endpoint devices within an organization and intervene in real time with the help of the technologies in it when it sees any threat. While their working logic is similar to antivirus software, they also provide more traceability, customized protection and coverage.

If you manage a large organization, you cannot know whether your users will open a malicious email attachment, click on a link, or upload a wrong file. In order to be able to quickly detect and take action in a negative situation, you need SIEM and EDR solutions suitable for the size of your organization.

If we briefly explain the working logic, unlike classical security solutions, EDRs have features called “sensors“. Sensors detect anomalies in the systems and ensure that the necessary action is taken to prevent them, while at the same time reporting to the control server. What we usually try to do when trying to bypass EDR is to develop malware that does not attract the attention of these sensors or to blind the sensor itself with various methods.

Research on Evaluation Mechanism in the EDR Circumvention Process

A. File Reputation and the Effect of File Structure on the Process

What we mainly worked on in the malware we developed was to find out why the target EDR solution marks a program and file (at least statically) as malicious. We prefer to call the whole of these processes file reputation. The analysis and evaluation of each solution. Since its mechanism is different, its success depends on your research.

As you can watch the tests in the post we previously shared on X (Twitter), the malware we developed in the example successfully managed to bypass the most well-known products in the industry such as SentinelOne, Trellix, ESET, and unfortunately, we achieved the same results from these products for months. This product sold for thousands of dollars. The only thing that allowed us to easily pass the products was our understanding of the working logic of EDR sensors.

#sentinelone EDR evasion test 🤷‍♂️
Remnants of red team operation pic.twitter.com/4Zqsfe9QbE

— Secunnix (@secunnix) May 13, 2023

Today, many EDR solutions cannot properly analyze binary files developed with new generation or, in other words, exotic languages. Since antivirus modules in security software are generally developed for static analysis of malware written in languages such as C, C++ and C#, these When they encounter a file prepared with a very remote structure, they cannot detect it naturally.

We developed our example only with Rust and did not use any extra evasion techniques. Even the socket connection we opened directly to receive a command shell session from our target was not detected as an abnormal activity by the EDR sensors. Our malware, which could not be detected in the scantime and runtime phase.

Now, the point to be questioned here is that you do not/should not expect a good security software in terms of engineering to rely only on the static analysis process and signature-based work. Behavioral detection models should generate an alarm when a socket connection is opened, at least if it is something that has not been encountered before.

B. Normalization and Naturalness

Another thing to pay attention to in EDR bypass processes is how normal the program looks. Normalization, which is partially related to file reputation, has a very important place among undetection techniques. For this, both APT groups and red teaming are used. We wrote a simple MsgBox outputting program using the AutoIt scripting language, which is frequently used in scripting scenarios, and conducted some experiments using VirusTotal.

In our investigations, we have seen that in many EDR solutions, detection rates vary depending on whether the file is 32-bit or 64-bit, and whether the file has an icon or not is also effective. Software that has an icon and is compiled as 64-bit is less effective. While it seemed suspicious, software that was 32-bit and compiled without icons was considered more risky.

32-bit without icon:

With 64-bit symbol:

From here we can understand how effective even the way the file is created is among the evaluation criteria of security solutions offered in the form of AI powered today. Our program was actually harmless, it did not pose any risk, but what marked it as harmful was what we call reputation.

From what we have discovered so far, the simplest things that can be done to reduce the detection rate are the following:

On the last issue, we can confirm that although it is a fairly common technique, it still works. Normally, when you open a socket connection directly to a remote target, this increases the risk level of your malware to the antivirus solution, while a connection to an ordinary messaging program or HTTP requests to a trusted web address It is relatively more acceptable to come out.

As Secunnix Malware Research Team, we have developed a simple experimental RAT (remote access trojan/remote administration tool) that uses Telegram as a command and control server. Simply put, when it is run on the target system, it transmits a small message to the Telegram bot API and waits for commands from the operator. It extracts the incoming commands appropriately and performs the operations.

With the help of this technique alone, we were able to successfully bypass several of the most popular and well-known security solutions that have topped the charts in tests, and the activities of the RAT we developed were never considered malicious by these solutions. This is because the incoming and outgoing data is just a Telegram address. Our malware worked like a messaging client.

C. Risks Brought by Trust in Digital Signatures

Unfortunately, today, many EDR and antivirus solutions still consider a program with a digital signature to be quite reliable, especially if it is not analyzed and added to the database maliciously.

Technically speaking, the purpose of a digital signature is to serve as a verification mechanism that a program has not been subjected to unwanted modifications and that it came directly from the developer. Since changes made by external sources (e.g. an attacker) compromise the integrity of the file, the signature becomes invalid and should not be trusted.

But rather than what is the real purpose of the digital signature, the issue has turned into building an artificial reliability mechanism, considering that it is unlikely that malware developers will obtain it because there is a fee to obtain this certificate. This is a sufficient precaution against novice developers, whom we call script kiddies. Even if it is, attackers working with very large budgets - ransomware groups, APTs? - can pay whatever they pay and get signatures with almost zero control.

For example, below you see the code signing prices of DigiCert, one of the largest certificate providers in the world. You can buy a 1-year EV/Extended Validation type certificate for 755 dollars.

Worse, attackers no longer even bother buying signatures, but directly use signed programs for their purposes. For example, below you can see how EDR solutions were disabled using TDSSKiller, a digitally signed rootkit remover.

Unfortunately, in our tests, 4 of the largest EDR families could not even protect themselves, let alone prevent the attack, and their services were easily removed by TDSSKiller without raising any alarms. This tool, which is also abused by many ransomware groups, can be installed on the system with local administrator privileges. When it is run and the target service is given to it with the -dcsvc parameter, it successfully performs what is requested.

The interesting thing is that TDSSKiller works similar to viruses in order to prevent various rootkits from detecting it. The processes it carries out, such as using TLS Callbacks, using a unique packer, and making use of WinAPIs that can be considered intensive and risky, are potentially harmful. Although it should be accepted, it was probably ignored by EDR solutions just because it has a digital signature. Where are the tamper protection modules?

2 – Design Weaknesses in the EDR Circumvention Process

EDR sensors typically work like rootkits. What they do is to monitor each running program with various agents and prevent risky operations or the execution of previously signed malicious code snippets. The EDR software and drivers installed on the system, each While the program is running, they apply the techniques we call API Hooking or Process Injection to ensure that the execution process is carried out on their own. Thus, they can follow the relevant program and terminate it if necessary.

A Windows PE (Portable Executable) program needs various libraries and APIs to function while it is running. The process of finding these APIs is usually carried out with a structure we call ntdll.dll. When the program first runs, ntdll It finds and presents Native APIs in the IAT (Import Address Table) structure via .dll.

EDR products intervene at this moment and inject themselves into the program, and the events continue through the hook DLL of the solution. If the program is innocent, it allows it to run, otherwise it exits by returning a certain code.

For example, in systems using Windows Defender, as many red teamers know, the process is controlled by adding Amsi.dll to potentially risky processes (powershell.exe, cmd.exe, etc.). To circumvent this, the AmsiOpenSession method is disrupted and Amsi .dll returns incorrect parameter code.

To examine the general structure of many antiviruses and to learn which processes they use, you can also take a look at the Antivirus Artifact III document written by etheral_vx about 2 years ago.

https://github.com/ethereal-vx/Antivirus-Artifacts

Since bypassing EDR sensors is not that simple, a more advanced technique is usually used on the user mode side. Direct System Calls (Direct Syscall) and Indirect System Calls (Indirect Syscall).

A. Direct Syscall

In its simplest form, the direct system calls technique can be said to implement the APIs to be used in the malware with assembly commands in the code instead of getting them through ntdll.dll, in order to prevent EDR sensors from hooking (I like to use Turkish terms).

In this way, the program does not refer to ntdll.dll and uses the APIs it needs directly. Since ntdll.dll is not called, the EDR sensors will be blind because they do not inject themselves into the process. There are many PoCs for this technique, For example, Hells Gate, Halo's Gate, Syswhispers2 and 3 are successful studies that prove the application of direct syscall.

Unfortunately, this method, which was quite effective at first, can be detected by some EDR solutions through kernel callbacks, since the system calls are executed outside of ntdll.dll and the RET command is located directly in the memory area of the program to which the direct syscall is applied. Because normal programs do not behave like this. As you can see, it comes back to what we explained under the heading of normalization and reputation.

B. Indirect Syscall

Indirect system calls are similar to the direct system calls technique in terms of their working logic. But beyond that, they have a few minor differences in the processing of system calls.

In the direct syscall technique, while the process belonging to ntdll.dll is completely outside the event and the RET commands are in the program itself instead of this DLL, in the case of indirect syscall, since the calls are executed through ntdll.dll, they are perceived as normal by EDR processes. Okay. So, if you ask how EDR cannot perform the hook operation, the logic is actually simple. During the execution of the malicious code, the JMP command goes to the block where the system call is located, instead of pointing to the entry point (starting point) of ntdll.dll. Thus, a device unaware of ntdll.dll It is a little more difficult to detect since there is no execution process.

Although both techniques still work, the indirect syscall method can help develop less detectable malware. Note that these are methods to avoid user mode hooking. If you are dealing with a security solution that does kernel mode hooking, this techniques can be identified.

In addition, the Event Tracing for Windows (ETW) structure offered by Microsoft for Windows systems can be used by some EDRs to control the stack structures where calls are made. If you want to see a very clean implementation of both techniques, read the article below. We can recommend it.

https://redops.at/en/blog/direct-syscalls-vs-indirect-syscalls

3 – Kernel Mode Methods in EDR Disabling Process

Based on our experiences and tests during the EDR bypass process, our conclusion is that studies carried out using techniques on the kernel mode side give more successful results. Because developing a special malware and ensuring that the file has the necessary reputation is a process that requires a lot of effort. Many attackers Based on the principle of time = gain, they have to use the malwares they have previously developed many times in different systems.

As we have seen, many ransomware groups focus on directly removing EDR or blinding the sensor of the malware that will perform the encryption process, instead of making it undetectable when it infiltrates the target system. Then, it easily executes the malware in the system. Here, it mainly takes advantage of the vulnerabilities of kernel mode applications. For example, vulnerable drivers or design problems caused by the kernel itself.

Using EDR SandBlast through Bring Your Own Vulnerable Driver (BYOVD) and PPLFault (GodFault) are the most well-known techniques that work successfully.

A. Bring Your Own Vulnerable Driver (BYOVD)

If you remember, we talked about the abuse of trust relationship at length at the beginning of the article. As it is known, in new generation Windows systems, due to the Driver Signature Enforcement (DSE) security feature, unsigned drivers cannot be installed without turning on test and debug mode. In other words, each driver must have a digital driver. It must have a signature. When it comes to drivers, of course, it is much more difficult to obtain a digital signature compared to normal programs.

Drivers with a digital signature naturally work in kernel mode, that is, as the most authorized and critical processes in the system. BYOVD technique is used to detect various security vulnerabilities that may exist on a driver that is currently considered innocent and safe, and to remove or disable EDR/antiviruses. It can be said that it is used for the purpose of opting out. In this way, both trust is ensured and processes can be intervened at the kernel level.

Various ransomware groups and state-sponsored cyber actors (APT) use this method a lot. Unfortunately, most EDRs counter this technique in a “signature-based“ way. That is, it has been previously identified as vulnerable by others.” If you have a driver that has not been used by others, you can often successfully disable target security solutions quickly.

B. EDR Sandblast via PPLFault

The PPLFault technique, presented at the conference at BlackHat Asia 2023, allows code execution through a TOCTOU vulnerability in the Windows Code Integrity module. Combining this with EDR SandBlast, security researchers have implemented EDR processes completely, without using any vulnerable drivers, only thanks to design errors in Windows components. and turned it into a tool that allows us to get a cmd session free of ETW.

We applied this method on EDRs as we found it very interesting during our research. Unfortunately, only 2 out of 5 solutions were able to block it. In the remaining ones, we were able to successfully obtain the command line that was not monitored by EDR sensors. This made the engineering of the relevant solutions quite difficult. It caused us to question because there is neither a digital signature nor BYOVD, and although the behaviors of the processes carried out were high-risk movements, they failed to detect them.

The other 2 EDR solutions that successfully blocked it were done in a signature-based way, not behaviorally. We managed to bypass the new generation and artificial intelligence-based solutions, and EDR Sandblast with GodFault (PPLFault) still works in most of them.

4 – Result

If we were to summarize, item by item, the inferences we have obtained about EDR and the security solutions industry as a result of the techniques we have examined and the tests we have carried out throughout the article:

•EDR solutions are very effective in terms of the security of the institution, but they should not be the basic building block of the drawn security architecture

•EDR solutions developed by antivirus companies that are known for their long history in the industry and appeal to normal users are more successful than software that is claimed to be artificial intelligence-supported but is not

•While EDR solutions successfully protect against conventional malicious threats, they are quite weak against unconventional attacks that abuse the trust relationship.

•Many of the results and protection percentages announced by various testing organizations do not reflect reality and are generally for marketing purposes

•When designing the security architecture of institutions and organizations, instead of leaving the job only to EDRs, tightening and SIEM solutions should be used as well as extra solutions

we can say that. As Secunnix, we take the security of our customers very seriously and try to implement correct and solution-oriented strategies instead of unrealistic statements for marketing purposes. At the same time, we are all in the same boat when it comes to cyber security, a lot of our data is in the digital environment and we are aware of this. For this reason, we will continue to actively inform you and write articles on our blog, whether it is our malware research team or our other teams working in the field of blue team and red team.

Stay healthy. - Note: This blog was originally written in Turkish. Therefore, there may be some ambiguities and spelling mistakes :-)