Debugging DLL Files with IDA Disassembler

Author

Uriel Kosayev
Cybersecurity researcher and red teamer who lives both on the offensive and defensive fronts. The author of the “Antivirus Bypass Techniques” book, expert in malware research, reverse engineering, penetration testing, digital forensics, and incident response

In this video, I’ll show you how to debug a DLL file with an IDA Disassembler. Many people find debugging DLL files challenging with IDA Pro and often prefer tools like x64dbg, which automatically imports and executes the DLL. However, with IDA, the process is a bit different. Let’s explore how to do it.

Step 1: Determine DLL Bit Version

Before starting, identify whether the DLL file is 32-bit or 64-bit:

  • If the DLL is 64-bit, use the 64-bit version of IDA.
  • If the DLL is 32-bit, use the 32-bit version.

In this example, we’re using a 32-bit DLL.

Step 2: Load the DLL in IDA Pro

  1. Launch IDA Disassembler.
  2. Load the DLL file. In this example, your desired DLL (Emotet in this example).
  3. Place a breakpoint at the prologue of the first function.
  4. Confirm that we’re in the DLL’s entry point. Check the exports of the DLL – in this case, there’s only one export.

Step 3: Configure the Debugger in IDA

  1. Select the Debugger: Choose “Local Windows Debugger” as the debugging environment.
  2. Set Process Options:
    • Go to Debugger > Process Options.
    • Since DLL files don’t execute on their own and need a host executable to import and run their functions, we need a process that can load the DLL.
Windows Security Researcher Badge

$1408

$1128 or $113 X 10 payments

Windows Security Researcher

Provides the necessary knowledge, understanding, and tools to be a successful Windows OS researcher.

Step 4: Configure the Executable to Load the DLL

  1. Use rundll32.exe, a system executable capable of loading and executing functions from a DLL.
  2. Path Configuration:
    • For a 32-bit DLL, use the 32-bit version of rundll32.exe, located in the C:\Windows\SysWOW64\ directory (even though the folder name can be confusing, it contains 32-bit system binaries).
    • Enter the path: C:\Windows\SysWOW64\rundll32.exe.
    • For a 64-bit DLL, use the 64-bit version of rundll32.exe, located in the C:\Windows\Systems32\ directory.
    • Enter the path: C:\Windows\System32\rundll32.exe.
  3. Set Parameters:
    • Specify the DLL’s directory as the working directory.
    • For the parameters, include the DLL’s full path and add ,#1 to indicate the first (and only) export, the DLL entry point.

Step 5: Test and Debug

  1. Run the Debugger: Execute the setup by clicking “Run.”
  2. Verify the Breakpoint: You should hit the breakpoint set at the entry point, allowing you to debug the DLL freely.

Thank you for watching!

This content is part of the Windows Security Researcher learning path.

Gain Insider Knowledge

Subscribe to updates from the TrainSec trainers
blue depth

About the author

Uriel Kosayev
Cybersecurity researcher and red teamer who lives both on the offensive and defensive fronts. The author of the “Antivirus Bypass Techniques” book, expert in malware research, reverse engineering, penetration testing, digital forensics, and incident response

Black Friday & Cyber Monday Sale Started!

For a limited time, enjoy 25% off ALL available courses for the next month. Whether you’re looking to sharpen your skills or explore new areas of cybersecurity, now’s the perfect time to invest in your growth.

Use code BFRIDAY24 at checkout to claim your discount.

*Excluding bundles and monthly plans.