Goals and Limits

Anti-debug techniques, or anti-debug tricks, are methods that make debugging more difficult. The general process is to detect a debugger or debugging technique and respond accordingly. The response should be clever and not too close to the time of detection. Reliability is also an issue to consider: the application must obviously be functional when not running in a debugger.

It is almost always possible to remove anti-debugging techniques in a protected application. How well these techniques work depends on how stealthy, varied, and common they are in the protected program. Without trusted hardware features at its disposal, anti-debug techniques can only slow down a reverse engineer.

Reminders about Debugging

When a user places a breakpoint at a specific location in the code, the debugger does roughly the following:

• it saves the address corresponding to the breakpoint location and saves the address in memory;
• it replaces the original instruction with another one that throws an exception (software interrupt);
• it registers a handler for this exception, which is responsible for taking the control when executed.
• when the program reaches a breakpoint, an exception is thrown and the program stops;
• the debugger catches the exception and shows the program’s internal state;
• when the program resumes, the original instruction is executed and the program continues to the next breakpoint or to its end.

The x86 architecture provides the instruction int3, which is encoded either as CC (short version) or CD03 (long version). On ARM, the armv6 and later architectures allow the processor to enter into debugging mode using the instruction BKPT. Hardware breakpoints are handled directly by the CPU and only a few of them can be set. This requires a high level of privilege (kernel mode) on most architectures.

Linux

On Linux, the debugger starts by forking itself. Then, the child process calls ptrace() with the PTRACE_ME option. After that, the child process runs the program that needs to be debugged with by calling the exec() system call. Another option is to attach directly to a running process using the PTRACE_ATTACH option.

A classical anti-debug trick on Linux is the following: you call the ptrace() system call with the PTRACE_ME option: if the process is already debugged, the call will return a value of -1; otherwise, the call will return a value of 0. In other words: the ptrace() with the PTRACE_ME option call returns 0 only once.

Other ideas are the following: on Linux, the process ID and the session ID of the parent process are always the same. On a debugged process, they are different. This fact can be used to detect that a debugger is working. Or you can set many breakpoints in the process to protect against debugging using the int3 instruction and define an appropriate signal handler. If the process is debugged, those exceptions will have to be handled by the debugger.

Microsoft Windows

On Windows, the function CreateProcess() is used to launch a new process. The sixth argument, dwCreationFlags, must be set to DEBUG_ONLY_THIS_PROCESS so the kernl knows that this process will be debugged. All exceptions and other debug events will be redirected to the thread that is running.

If a program is already running, the DebugActiveProcess() function lets you attach to the process. Once attached to a process, the debugger performs a loop with WaitForDebugEvent() waiting for future events. When an event occurs, WaitForDebugEvent() returns and the debugger has the information about the event to be further processed.

The PEB (Process Environment Block) contains the properties of the current process and is accessed via the FS (32 bits) or GS (64bit) register. The BeingDebugged field from PEB shows whether a process is debugged or not. In normal circumstances, this flag will be 0. However, if a debugger is attached, the flag will be set to 1. Although this flag is accessible through the PEB structure, it is usually verified through the Windows API: IsDebuggerPresent() or CheckRemoteDebuggerPresent().

Many other anti-debug techniques for Microsoft Windows can be found on this website.

Timing-Based Anti-Debug Tricks

One feature of debugging programs is that they run much slower. Many anti-debug techniques follow this process: time a part of the code that has been executed; if the time is more than a set limit, then a debugger is detected.

The Intel architecture has a 64-bit counter called TSC that increases at each CPU clock tick. The RDTSC instruction can read the value in that counter, so it provides a very accurate clock. Other, less precise techniques can be used, such as the ones relying on functions declared in time.h.

If the application is always connected, the same methods using remote attestation can be used to detect if the program is running slowly. The application can send various timings to the server, which can decide based on certain statistical criteria that the program is in a seemingly unlawful state and take appropriate counter-measures.

Related Techniques

One of the easiest ways to find a debugger is an artefacts search. An artefacts search looks for: locations in file systems where a debugger is usually installed; debugger processes in the process list; certain registry keys associated with debuggers (on Windows); or specific strings in the process memory.

When you use the gdb debugger to debug a program, its environment variables are altered. Two new variables appear: COLUMNS and LINES. Additionally, the variable _ is modified and contains the debugger path instead of the program path.

When using the WinDbg debugger on a Windows system, the environment variable WINDBG_DIR is available while the program is being debugged.

The goal of an anti-disassembler technique is to fool disassemblers that use linear or recursive methods. One common technique is to insert junk bytes at the location of the next instruction and to use a jump instruction to go to the correct location.

An emulator, such as QEMU, can also be difficult to identify because it offers a near-perfect view of the system to the program. However, detection techniques hide in certain aspects that are not perfectly emulated. These techniques depend on the type and version of the emulator. They typically involve playing with the hidden aspects of the visible hardware.

Jailbreak and Root Detection

Jailbreaks

Apple devices running iOS have very strict security features, including tight sandboxes for apps. Jailbreaking an Apple device means using a privilege escalation exploit to bypass or remove the software-enforced restrictions imposed by Apple at the kernel level.

The main goals are:

• install apps that are banned on the Apple Store or pirated apps;
• customize the operating system or an app;
• unlock the phone’s network carrier;
• install malware;
• install pirated software;
• do security research;
• etc.

A small, but not insignificant, number (in the order of 1%) of Apple devices are jailbroken. Some applications won’t run on jailbroken phones for security reasons. For example, many online banking apps won’t run on jailbroken phones. Some applications detect jailbroken phones but tolerate them. Most applications don’t care.

For a mobile application, running in a jailbroken environment is often dangerous because the user has a lot more control over the program. Jailbreak detection includes finding special permissions that a sandboxed application usually doesn’t have:

• belonging to privileged groups;
• permission to write in certain locations that are normally forbidden;
• the ability to call system calls that are normally forbidden (likefork());
• etc.

Another possibility is to identify the remains of popular applications running on jailbroken phones.

Android Rooting and Custom ROMs

Rooting is the process of gaining privileged control (i.e., a root access) on Android. The goals of Android rooting are slightly different than those of iOS jailbreaking: jailbreaking is concerned with bypassing Apple prohibitions for the end user, typically through sideloading, while some Android phone companies let you unlock the boot-loader and replace the operating system entirely (this is called a custom ROM), providing you full OS privileges.

The Play Integrity API on Android allows you to check for rooted devices and custom ROMs. Otherwise, similar techniques than for detecting jailbreaking apply.

In the next episode, I’ll cover some advanced techniques of reverse engineering. Stay tuned!

Thanks for reading Crumbs of Cybersecurity! Subscribe for free to receive new posts and support my work.