I finally decided to write my first security tool based on an idea I had for advanced hook detection, I couldn’t find any evidence of the method being used so I based a tool around it. It’s still a working progress but I’m posting so I can get some feedback early on (Currently only x86 systems are supported, but this shouldn’t be an issue as most researchers are running 32-bit VMs).
The scanner will iterate the export table for the following system modules: ntdll.dll, kernel32.dll, kernelbase.dll, user32.dll, ws2_32.dll, and wininet.dll. Like a conventional scanner it will go through each instruction of the exported function, comparing it against a clean version of the dll which has been loaded from the disk. Normally at this point a hook scanner would either report the modification or check to see if it’s a jump or call, but I’ve decided to take an extra step towards detecting obscure hooks.
If any modification is found at any point within the function body, the scanner will use my basic x86 emulator to begin emulating the function, while tracing push, pop, mov, lea, jmp, call, and ret instructions. The emulator will try to determine if control flow is altered by the modified instructions and if so, which instruction redirects execution and to where.
The purpose of the emulator is to detect more obscure hooks as well as accurately determine the destination of the hook, beyond resolving basic jump and call instructions. A good example of where this is applicable is on hooks placed by carberp within the native call stubs of ntdll.
This function does not use a relative call, instead it moves a pointer into the edx register and performs an absolute indirect call with it. Carberp replaces the address moved into the edx register, resulting in redirecting the call to an arbitrary location, and not showing up in hook scanners searching for jmp/call hooks. More advanced hook scanners will log any modifications; however, the user would then have to disassemble the modified function and determine the destination address of the hook, which my engine does automatically (if it fails to resolve the location or detect a control transfer, the modification will still be logged for further investigation).
Most rootkits hook by injecting their code (usually shellcode or a PE file) into the target process, hooks are then set to point to locations within the injected code. If the tool is successfully able to work out the destination of a hook, it will query the page it points to and then map out all pages allocated with the same base address. Using this information it is possible to dump the rootkit’s entire shellcode, injected PE or DLL, even if it spans multiple pages.
Again, this is a working progress and I can’t guarantee the current version won’t be a huge steaming pile of crap. please email any feedback/issues to email@example.com or leave a comment on this post.
Here’s a demo video to prove it does at least work on my system: