A Script for Automated Backtracing in IDA

Last week I stumbled upon a piece of malware that uses different techniques for synchronization. The malware would create named Events, named Mutexs and CriticalSections. After enumerating all the named events and mutexs, I started to wonder if there was a way to show relationships between them. For example where did an event get created, what and where was the catalyst for closing or releasing the event. I realized I would need to get the arguments for CreateMutex*, CreateEvent* and a number of other APIs. Since I'm getting the arguments I should probably track how these are being referenced or moved around.  backtrace.py is a script to aid in that process of tracing how register and arguments are used. For more information on the script please see the Table of Code Contents below. Please download the code from the BitBucket Repo. Bux fixes will not be updated on this post.

BitBucket Repo - LINK - If you come across a bug or an error please let me know through an email or the issue tracker. A password protected zipped up executable with a description of the error would be very helpful.

from idaapi import * 
import idautils
import idc
import sys

'''
Name: 
    backtrace.py
 Version: 
    0.1
 Author
    alexander<dot>hanel<at>gmail<dot>com

Table of Code Contents
Summary:
    backtrace.py is a class that can be used to backtrace reference of registers and args in IDA. 
    The class relies on string and function argument parsing. Using strings of disassembly 
    presents some problems because we have to account for every scenario of how the data is
    moved. Needless to say I did not account for all of these. I wrote this script to aid 
    break the script. It's still u    in tracking arguments being passed to API calls. There will 
    be plenty of scenarios that should be useful though. 
    
Quick Code Summary:
    Use backtrace(address, operand) to get the backtrace of a register in a function. The first 
    operand value starts at 0 and then 1. "mov     eax, ecx", eax = 0, ecx = 1. Don't use 0 when 
    first calling backtrace() unless there is only 1 operand. The first time backtrace() is called
    it does not check if the values are being moved. "push    eax" is a safe situation when to use 0. 
    To see verbose output enable classobject.verbose = True. This code only account for the selected
    function and it's parents. If we wanted to check for an argument that was passed to the child, 
    which was also passed as an argument to the parent that will need to be coded up by the user. 
    Assume _cdecl & _stdcall

5 Second Demo
    .text:0040EA90 i___MainOfInjectedCode proc near        ; CODE XREF: sub_40ED40+5 p
    .text:0040EA90                                         ; _main_inject+5 p
    .text:0040EA90
    .text:0040EA90 arg_0     = dword ptr  4
    .text:0040EA90
    .text:0040EA90           push    ebx
    .text:0040EA91           push    ebp
    .text:0040EA92           mov     ebp, [esp+8+arg_0]    ; base address
    .text:0040EA96           push    esi
    .text:0040EA97           push    edi
    .text:0040EA98           call    _GetBaseAddressOfKernel32
    .text:0040EA9D           mov     esi, eax              ; eax = base address of kernel32
    .text:0040EA9F           test    esi, esi
    .text:0040EAA1           jz      short _return         ; something broke
    .text:0040EAA3           push    esi
    .text:0040EAA4           mov     ebx, offset aLoadlibrarya ; "LoadLibraryA"
    .text:0040EAA9           call    i_LookUpAPIAddr
    .text:0040EAAE           push    esi
    .text:0040EAAF           mov     ebx, offset aGetprocaddress ; "GetProcAddress"
    .text:0040EAB4           mov     edi, eax              ; assign address of LoadLibrary
    .text:0040EAB6           call    i_LookUpAPIAddr
    .text:0040EABB           xor     esi, esi
    .text:0040EABD           add     esp, 8
    .text:0040EAC0           cmp     edi, esi
    .text:0040EAC2           jz      short _return         ; something broke
    .text:0040EAC4           cmp     eax, esi
    .text:0040EAC6           jz      short _return         ; something broke
    .text:0040EAC8           push    eax                   ; GetProcAddress
    .text:0040EAC9           push    edi                   ; edi = LoadlibaryA
    .text:0040EACA           mov     ebx, ebp                <- HERE(), we are choosing ebp which is operand 1 
    .text:0040EACC           call    i_RebuildImportTable

Output
    Python>s = Backtrace()
    Python>s.verbose = True
    Python>s.getAll(1)
        0x40eaca mov     ebx, ebp
        0x40ea92 mov     ebp, [esp+8+arg_0]    ; base address

    ****** Argument Details ******
    [INFO] Selection in i___MainOfInjectedCode 
    0x40ea92 mov     ebp, [esp+8+arg_0]    ; base address
    [INFO] Selection is argument 1 in i___MainOfInjectedCode
    Called by sub_40ED40
        0x40ed44 push    eax
        0x40ed40 mov     eax, [esp+arg_0]

    Called by _main_inject
        0x40ed54 push    eax
        0x40ed50 mov     eax, [esp+arg_0]
    
Functions:    
   * retApiRef(String_API_Name) - For working with code refs. Pass the string of an API name
       and it will return a list of the address of all cross-references to the API Name. 

   * retxrefs(Data_Address) - For working with data refs. Pass the address of the data and
       it will return in a list all the addresses that xref the data. 
       - Example:
            Python>s.retxrefs(here())  <- here() = .rdata:004120FC byte_4120FC db 48h  
               [4214883, 4214892] <- all xrefs to data address

   * getArgs(FUNCTION_CALL_ADDRESS, STACK_ARG_NUMBER) - For getting the address and operand
       of an argument pushed on to a function. Pass the address of where the funciton is being called
       from and the count of the argument. Uses a simple string 'push' string search. Will check up to
       10 previous lines. After that is will return None if not found. 
       - Example:
            .text:0040E495           push    eax
            .text:0040E496           add     ecx, 8
            .text:0040E499           call    sub_404FE0
            Python>s.getArgs(0x040E499, 1)
            (4252821, 'eax') <- address of the push and the operand

    * getStack(FUNCTION_ADDRESS) - For getting all variables used on the stack in a function. Returns
        a list that contains each objects used on the stack. 
        - Example:
            .text:0040E1E0 sub_40E1E0 proc near                    ; CODE XREF: sub_40EA50+2A p
            .text:0040E1E0
            .text:0040E1E0 var_10    = dword ptr -10h
            .text:0040E1E0 var_C     = dword ptr -0Ch
            .text:0040E1E0 var_8     = dword ptr -8
            .text:0040E1E0 var_4     = dword ptr -4
            .text:0040E1E0 arg_0     = dword ptr  4
            .text:0040E1E0 arg_4     = dword ptr  8
            .text:0040E1E0 arg_8     = dword ptr  0Ch
            .text:0040E1E0
            .text:0040E1E0           sub     esp, 10h
            .text:0040E1E3           mov     ecx, [esp+10h+arg_8]
            Python>s.getStack(0x0040E1E0)
            [('var_10', 4, '0x20000400'), ('var_C', 4, '0x20000400'), ('var_8', 4, '0x20000400'), 
            ('var_4', 4, '0x20000400'), (' r', 4, '0x400'), ('arg_0', 4, '0x20000400'), ('arg_4', 4, '0x20000400'),
            ('arg_8', 4, '0x20000400')] 
            
    * getStackArgs(STACK) - For getting arguments pushed on to the stack. The STACK is the list retured from the 
        function getStack (see above). 
        - Example:
            .text:0040E1E0 sub_40E1E0 proc near                    ; CODE XREF: sub_40EA50+2A p
            .text:0040E1E0
            .text:0040E1E0 var_10    = dword ptr -10h
            .text:0040E1E0 var_C     = dword ptr -0Ch
            .text:0040E1E0 var_8     = dword ptr -8
            .text:0040E1E0 var_4     = dword ptr -4
            .text:0040E1E0 arg_0     = dword ptr  4
            .text:0040E1E0 arg_4     = dword ptr  8
            .text:0040E1E0 arg_8     = dword ptr  0Ch
            .text:0040E1E0
            .text:0040E1E0           sub     esp, 10h
            .text:0040E1E3           mov     ecx, [esp+10h+arg_8]
            Python>s.getStackArgs( s.getStack(0x0040E1E0))
            [('arg_0', 4, '0x20000400'), ('arg_4', 4, '0x20000400'), ('arg_8', 4, '0x20000400')]
    * printlog() - Used to print when a varaible gets reassinged. This is logged in the list self.refsLog.
    
    * backtrace(ADDRESS, OPERAND_PLACEMENT) - For getting the first reference of a register or stack object
        of a selected operand.  The return is a tupple with the address and the dism. This uses simple string 
        parsing to track when a operand/register/stack object is moved around in a function. Referred to as lastRef
        IMPORTANT please verify the correct number of the operand when selecting. No error checking is done 
        to verify the size. For the following 'mov     eax, ecx', eax is operand 0 and ecx is operand 1. 
        - Example:
            .text:0040E1E0           sub     esp, 10h
            .text:0040E1E3           mov     ecx, [esp+10h+arg_8]
            .text:0040E1E7           push    ebx
            .text:0040E1E8           push    ebp
            .text:0040E1E9           push    esi
            .text:0040E1EA           mov     esi, [esp+1Ch+arg_0]
            .text:0040E1EE           lea     ebx, [esi+28h]
            .text:0040E1F1           lea     eax, [ebx+854h]
            .text:0040E1F7           mov     [esp+1Ch+var_10], eax
            .text:0040E1FB           mov     eax, ecx  <- ecx is chosen. 
            .text:0040E1FD           and     eax, 4]
            Python>s.backtrace(0x0040E1FB, 1)
            (4252131, 'mov     ecx, [esp+10h+arg_8]') <- address and string
            To see the printlog and referenes set class_instance.verbose = True
            Python>s.backtrace(0x0040E1FB, 1)
            0x40e1fb mov     eax, ecx
            0x40e1e3 mov     ecx, [esp+10h+arg_8]
            (4252131, 'mov     ecx, [esp+10h+arg_8]')
        NOTE: Don't choose operand 0 for MOVs during first selection. The code does not check for MOV from 
        operand 0 to operand 1 in the first selection. Might update later. 
        
    * questionBackItUp(RETURN_FROM_backtrace) - For checking if the return from backtrace is passed as an argument
        on the stack. Labled as a lastRef in the code. It returns the stack arugment number if the lastRef was an
        argument. Returns None if it isn't. 
        - Example:
            .text:0040E1E0           sub     esp, 10h
            .text:0040E1E3           mov     ecx, [esp+10h+arg_8]
            .text:0040E1E7           push    ebx
            .text:0040E1E8           push    ebp
            .text:0040E1E9           push    esi
            .text:0040E1EA           mov     esi, [esp+1Ch+arg_0]
            .text:0040E1EE           lea     ebx, [esi+28h]
            .text:0040E1F1           lea     eax, [ebx+854h]
            .text:0040E1F7           mov     [esp+1Ch+var_10], eax
            .text:0040E1FB           mov     eax, ecx  <- ecx is chosen. 
            .text:0040E1FD           and     eax, 4]
            Python>s.questionBackItUp( s.backtrace(0x0040E1FB, 1))
            3 <- argument count.

    * strip(string) - Used for splitting up and parsing operands. I couldn't figure out how to get the one to one
        name between the dism '[esp+1Ch+arg_0]' and the stack name of arg_0. 
        
    * backItUp(RETURN_FROM_backtrace) - For getting all argument address and their operands to a function. This checks 
        all xrefs to a function. The return type is a list that contains a tupple of (address and operand). 
        - Example: 
            The following function is called twice. 
            Python>s.backItUp( s.backtrace(0x0040EACA, 1))
            [(4255044, 'eax'), (4255060, 'eax')]
            To get more details we can enable the verbose which displays more info
            Python>s.backItUp( s.backtrace(here(), 1))
            0x40eaca mov     ebx, ebp
            0x40ea92 mov     ebp, [esp+8+arg_0]    ; base address

            ****** Argument Details ******
            [INFO] Selection in i___MainOfInjectedCode 
            0x40ea92 mov     ebp, [esp+8+arg_0]    ; base address
            [INFO] Selection is argument 1 in i___MainOfInjectedCode
            Called by sub_40ED40
                0x40ed44 push    eax
                0x40ed40 mov     eax, [esp+arg_0]

            Called by _main_inject
                0x40ed54 push    eax
                0x40ed50 mov     eax, [esp+arg_0]

            [(4255044, 'eax'), (4255060, 'eax')]
            
    * getAll(OPERAND_NUMBER) - Simple wrapper for backItUp() and backtrace(). It uses the selected address of the
        cursor. Verbose will need to be enabled.     
        
TO DO List
    1. Forward Trace
    2. Cross-refernce check
    3. Clean up the output. 
'''

class Backtrace():
    def __init__(self):
            self.AlexCanNotDrawAStack = True
            self.registers = ['eax', 'ebx', 'ecx', 'edx', 'esi', 'edi', 'esp', 'ebp']
            self.verbose = False
            self.refsLog = []
            self.maxDepth = 25
            
    def retApiRef(self, apiname):
        'get all xrefs to an api by name'
        apiAddrList = []
        for address in CodeRefsTo(LocByName(apiname),0):
            apiAddrList.append(address)
        return apiAddrList

    def retxrefs(self, address):
        'get xrefs from data return address'
        xrefsList = []
        for xrefs in DataRefsTo(address):
            xrefsList.append(xrefs)
        return xrefsList

    def getArgs(self, address, count):
        'get specified (by count) argument and address'
        pushcount = 0
        instructionMax = 10 + count
        currAddress = PrevHead(address,minea=0)
        while pushcount <= count and instructionMax != 0:
            if 'push' in GetDisasm(currAddress):
                pushcount += 1
                if pushcount == count:
                    return currAddress, GetOpnd(currAddress,0)
            instructionMax -= 1
            currAddress = PrevHead(currAddress,minea=0)
        return None, None 

    def getStack(self,address):
        'gets variables used on the stack'
        # Related reads and/or pointers on stacks in IDA
        # http://zairon.wordpress.com/2008/02/15/idc-script-and-stack-frame-variables-length/
        # https://github.com/moloch--/IDA-Python/blob/master/Stack_Calculator.py
        stackFrame = GetFrame(address)
        lastEntry = GetLastMember(stackFrame)
        count = 0
        stack = []
        while count <= lastEntry:
                localName = GetMemberName(stackFrame,count)
                size = GetMemberSize(stackFrame, count)
                flag = GetMemberFlag(stackFrame, count)
                if localName == None or size == None or flag == -1:
                        count += 1
                        continue 
                stack.append((localName, size, hex(flag)))
                count += size
        # returns stack variable list[tuple(localName, size, hex(flags))]
        return stack

    def getStackArgs(self, stack):
        'returns the args seen locally in a function returned from getStack()'
        # can use len(getStackArgs(stack)) to get arg count 
        for c,x in enumerate(stack):
            if x[0] == ' r':
                # returns stack variable list[tuple(localName, size, hex(flags))]
                return stack[c+1:]
        return None 

    def printlog(self):
        'print log'
        if self.verbose == True:
            for ref in self.refsLog:
                 print '\t', hex(ref[0]), ref[1]
            print
        self.refsLog = []
    
    def backtrace(self, address, operand):
        'find the initial assignment'
        lastRef = (address, GetDisasm(address))
        self.refsLog.append(lastRef)
        funcStart = GetFunctionAttr(address, FUNCATTR_START)
        var = GetOpnd(address, operand)
        currentAddress = PrevHead(address)
        dism = GetDisasm(currentAddress)
        while(currentAddress >= funcStart):
            # check for reg being used as a pointer [eax]
            for reg in self.registers:
                ptreg = '['+reg+']'
                if var == ptreg:
                    var = reg
            if var.isdigit() == True:
                tmp = NextHead(currentAddress)
                lastRef = (tmp, GetDisasm(tmp))
                self.refsLog.append(lastRef)
                self.printlog()
                return lastRef
            if 'call' in dism and var == 'eax':
                lastRef = (currentAddress, GetDisasm(currentAddress))
                self.refsLog.append(lastRef)
                self.printlog()
                return lastRef
            if var in dism:
                #lastRef = (currentAddress, GetDisasm(currentAddress))
                mnem = GetMnem(currentAddress)
                if 'mov' in mnem or 'lea' in mnem:
                    if GetOpnd(currentAddress,0) == var:
                        var = GetOpnd(currentAddress,1)
                        lastRef = (currentAddress, GetDisasm(currentAddress))
                        self.refsLog.append(lastRef)
            currentAddress = PrevHead(currentAddress)
            dism = GetDisasm(currentAddress)
        self.printlog()
        return lastRef

    def questionBackItUp(self, lastRef):
        'checks if last ref is an argument'
        argNum = None
        address, dism = lastRef
        # get value, assuming mov* and lea, check backtrace() for the logic
        dism = dism[:dism.find(';')-1] # used to strip out comments
        value = dism.split()[-1]
        var = self.strip(value)
        args = self.getStackArgs(self.getStack(address))
        for each in var:
            for index, arg in enumerate(args):
                if each == arg[0]:
                    argNum = index + 1
                    if self.verbose == True:
                        print '[INFO] Selection is argument %s in %s' % (argNum, GetFunctionName(address))
                    return argNum
        return argNum
   
    def strip(self, string):
        'sorry could not figure out to get the stack name vs the ascii name in ida'
        for x in '[]+.*':
            string = string.replace(x, ' ')
        for x in self.registers:
            string = string.replace(x, ' ')
        return string.split()
                           
    def backItUp(self,lastRef):
        allArgRefs = []
        address, dism = lastRef
        if self.verbose == True:
            print '****** Argument Details ******'
            print '[INFO] Selection in %s ' %  GetFunctionName(address)
            print hex(address), dism      
        xrefs = ''
        arg = self.questionBackItUp(lastRef)
        # if arg returns None, the lastRef is not an argument
        if arg != None:
            for funcAddr in CodeRefsTo(GetFunctionAttr(address, FUNCATTR_START),0):
                if funcAddr == None:
                    continue 
                argAddr, oper = self.getArgs(funcAddr, arg)
                if argAddr == None or oper == None:
                    if self.verbose == True:
                        print '[ERROR]: Could not get args for', hex(funcAddr)
                    continue
                allArgRefs.append((argAddr, oper))
                if self.verbose == True:
                    print 'Called by %s' % GetFunctionName(funcAddr)
                self.backtrace(argAddr, 0)
        else:
            if self.verbose == True:
                print 'Selection is not an argument'
        return allArgRefs
    
    def getAll(self, op):
        self.backItUp(self.backtrace(here(),op))
            
'''    
if __name__ == "__main__":
    s = Backtrace()
    s.verbose = True
    print 'updated'
'''

Backtrace POC - Stack Strings

Example 1 Hex View

There are a number of tools that cover char strings in IDA. If you are not familiar with char strings it's a low hanging obfuscation technique to thwart analyst from viewing the strings inside of an executable. Some notable tools and posts on this topic are [1] & [2]. In the image above you can see the string DBG. Odds are if we were viewing the executable in a hex editor or using strings this wouldn't stick out.

Example 1 Assembly View

If we were watching the stack of the executable at run time we would see something constructed similar to the string/comment above.

Example 2

 The code can be run in two modes the first is by selecting the code and the double clicking the script in IDA (ALT+F9). In the example above we can see the string "W32Time". My code attempts to reconstruct the stack memory. The buffer can be accessed via a list object.str_buff. In the Output window above you can see the content of the buffer dumped to standard out. This makes it easy to format the data and access it via an index. The commented data is an example of how the string would look on the stack in Ollydbg. The second way to execute the code is to pass an address within a function to object.run( address ). This will try to rebuild the stack for the whole function. All of this is done statically. Char strings that are populated via registers (such as mov [ebp+var_c], bl when bl is 0x4f in the example 1 image) are traced back using backtrace.py. For more details on backtrace please see the the following link.

As previously mentioned this topic has already been covered. I'm posting this code because it's a good example of using backtrace.py. I had fun working on this one. The code handles all examples I have found so far. There is an issue with formatting constructed wide char strings. Not exactly sure of the best approach. I tried to keep the data flexible so it should be easy to write a function to format the data.

[1]. Automatic Recovery of Constructed Strings in Malware by Jay Smith of FireEye - link
[2]. Finding Byte Strings using IDAPython by Jason Jones of Arbor Networks - link 

Repo - Link

Code for reviewing

"""
Author:
    Alexander Hanel 
Date:
    20140902
Version:
    1  - should be good to go.
Summary:
    Examples of using the backtrace library to rebuild strings

TODO:
    * How to deal with printing wide char strings?
    * What is the size of the frame buffer if GetFrameSize returns something
      smaller than the frame/stack index or the IDA does not recognize the function?

Notes:
    idaapi.o_phrase # Memory Ref [Base Reg + Index Reg]
    o_phrase   =  idaapi.o_phrase    #  Memory Ref [Base Reg + Index Reg]    phrase
    o_displ    =  idaapi.o_displ     #  Memory Reg [Base Reg + Index Reg + Displacement] phrase+addr

Useful Reads
    http://smokedchicken.org/2012/05/ida-rename-local-from-a-script.html
    http://zairon.wordpress.com/2008/02/15/idc-script-and-stack-frame-variables-length/
"""
import sys, os, logging, copy
from binascii import unhexlify
# Add the parent directory to Python Path
sys.path.append(os.path.realpath(__file__ + "/../../"))
# import the backtrace module
from backtrace import *

class Frame2Buff:
    def __init__(self):
        self.verbose = False
        self.func_start = idc.SelStart()
        # SelEnd() returns the following selected instruction
        self.func_end = SelEnd()
        self.esp = False
        self.ebp = False
        self.comment = True
        self.frame_size = None
        self.bt = None
        self.str_buff = None
        self.comment = True
        self.formatted_buff = ""
        self.format = True

    def run(self, func_addr=None):
        """ run and create Frame2Buff"""
        # check if code is selected or if using the whole function
        if self.func_start == BADADDR or self.func_end == BADADDR:
            if func_addr == None:
                if self.verbose:
                    print "ERROR: No addresses selected or passed"
                return None
        if func_addr:
            self.func_start = idc.GetFunctionAttr(func_addr, FUNCATTR_START)
            self.func_end = idc.GetFunctionAttr(func_addr, FUNCATTR_END)
        if self.func_start == BADADDR:
            if self.verbose:
                print "ERROR: Invalid address"
        self.frame_size = GetFrameSize(self.func_start)
        try:
            self.bt = Backtrace()
            self.bt.verbose = False
        except ImportError:
            print "ERROR: Could not import Backtrace - aborting"
        self.func_end = PrevHead(self.func_end)
        self.populate_buffer()
        if self.format:
            self.format_buff()
        if self.comment:
            self.comment_func()

    def populate_buffer(self):
        curr_addr = self.func_start
        self.str_buff = list('\x00' * self.frame_size)
        while curr_addr <= self.func_end:
            index = None
            idaapi.decode_insn(curr_addr)
            # check if instr is MOV, [esp|ebp + index], variable
            if idaapi.cmd.itype == idaapi.NN_mov and idaapi.cmd.Op1.type == idaapi.o_displ:
                if "bp" in idc.GetOpnd(curr_addr, 0):
                    # ebp will return a negative number
                    index = (~(int(idaapi.cmd.Op1.addr) - 1) & 0xFFFFFFFF)
                    self.ebp = True
                else:
                    index = int(idaapi.cmd.Op1.addr)
                    self.esp = True
                if idaapi.cmd.Op2.type == idaapi.o_reg:
                    # value needs to be traced back
                    self.bt.backtrace(curr_addr, 1)
                    # tainted means the reg was xor reg, reg
                    # odds are being used to init var.
                    if self.bt.tainted != True:
                        last_ref = self.bt.refsLog[-1]
                        idaapi.decode_insn(int(last_ref[0]))
                        data = idaapi.cmd.Op2.value
                    else:
                        # tracked variable has been set to zero by xor reg, reg
                        curr_addr = idc.NextHead(curr_addr)
                        continue
                elif idaapi.cmd.Op2.type != idaapi.o_imm:
                    curr_addr = idc.NextHead(curr_addr)
                    continue
                else:
                    data = idaapi.cmd.Op2.value
                if data:
                    try:
                        hex_values = hex(data)[2:]
                        if hex_values[-1] == "L":
                            hex_values = hex_values[:-1]
                        if len(hex_values) % 2:
                            hex_values = "0" + hex_values
                        temp = unhexlify(hex_values)
                    except:
                        if self.verbose:
                            print "ERROR: Unhexlify Issue at %x %s (not added)" % (curr_addr, idc.GetDisasm(curr_addr))
                        curr_addr = idc.NextHead(curr_addr)
                        continue
                else:
                    curr_addr = idc.NextHead(curr_addr)
                    continue
                # GetFrameSize is not a reliable buffer size
                # If so append to buffer if index is less than
                # 2 * frame size. If more likely an error
                if self.ebp or self.esp:
                    cal_index = index + len(temp)
                    if cal_index > self.frame_size:
                        if cal_index < (self.frame_size * 2):
                            for a in range(cal_index - self.frame_size):
                                self.str_buff.append("\x00")
                                if self.verbose:
                                    print "ERROR: Frame size incorrect, appending"
                if self.ebp:
                    # reverse the buffer
                    temp = temp[::-1]
                    for c, ch in enumerate(temp):
                        try:
                            self.str_buff[index - c] = ch
                        except:
                            if self.verbose:
                                print "ERROR: Frame EBP index invalid: at %x" % (curr_addr)
                if self.esp:
                    for c, ch in enumerate(temp):
                        try:
                            self.str_buff[index + c] = ch
                        except:
                                print "ERROR: Frame ESP index invalid: at %x" % (curr_addr)
            curr_addr = idc.NextHead(curr_addr)
        # reverse the buffer to match index
        if self.ebp == True:
            self.str_buff = self.str_buff[::-1]
            self.str_buff.pop()



    def format_buff(self):
        self.formatted_buff = ""
        temp_buff = copy.copy(self.str_buff)

        if self.ebp == True:
            temp_buff = temp_buff[::-1]
            temp_buff.pop()

        if self.str_buff:
            for index, ch in enumerate(temp_buff):
                try:
                    if ch == "\x00" and temp_buff[index + 1] != "\x00":
                        self.formatted_buff += " "
                except:
                    pass
                if ch != "\x00":
                    self.formatted_buff += ch

    def comment_func(self):
        idc.MakeComm(self.func_end, self.formatted_buff)

"""
Example:
    Create a buffer of the whole function

x = Frame2Buff()
x.run(here())  # func adddr

"""
x = Frame2Buff()
x.run() # select data

backtrace.py version 0.3

backtrace.py version 0.3 has been pushed out to it's repo. A couple of notable features have been added. The previous version only tracked the use of the MOV instruction. This is kind of useful..I guess..well at least it was fun to code.  The current version tracks whenever a register(ECX) or it's sub-register (CX) are manipulated. The old version relied on string comparisons. For example if we back trace from the highlighted code up we would see al is referenced then EAX, then byte_1003B03C, then dl, etc..

.text:10004E99                 mov     byte_1003B03C, al
.text:10004E9E                 movsx   ecx, byte_1003B03C
.text:10004EA5                 imul    ecx, 0A2h
.text:10004EAB                 mov     byte_1003B03C, cl
.text:10004EB1                 movsx   edx, byte_1003B03C
.text:10004EB8                 xor     edx, 0A4h
.text:10004EBE                 mov     byte_1003B03C, dl
.text:10004EC4                 movsx   eax, byte_1003B03C
.text:10004ECB                 cdq
.text:10004ECC                 mov     ecx, 0C8h
.text:10004ED1                 idiv    ecx
.text:10004ED3                 mov     byte_1003B03C, al
.text:10004ED8                 xor     eax, eax
.text:10004EDA                 jmp     short loc_10004F01
.text:10004EDC ; ---------------------------------------------------------------------------
.text:10004EDC                 movsx   edx, byte_1003B03C
.text:10004EE3                 or      edx, 0D2h
.text:10004EE9                 mov     byte_1003B03C, dl
.text:10004EEF                 movsx   eax, byte_1003B03C
.text:10004EF6                 imul    eax, 0C1h
.text:10004EFC                 mov     byte_1003B03C, al

The old version did not know that AL is the lower address of EAX due to the use of string comparison. The new version does a simple check of the register name and it's purpose. Note: there will be some issues if AH is moved into AL or other similar operations. I didn't code that logic in. If we were to back trace the code above we would have the following output.

Python>s.backtrace(here(),1)
 0x10004efc mov     byte_1003B03C, al
 0x10004ef6 imul    eax, 0C1h
 0x10004eef movsx   eax, byte_1003B03C
 0x10004ee9 mov     byte_1003B03C, dl
 0x10004ee3 or      edx, 0D2h
 0x10004edc movsx   edx, byte_1003B03C
 0x10004ed3 mov     byte_1003B03C, al
 0x10004ec4 movsx   eax, byte_1003B03C
 0x10004ebe mov     byte_1003B03C, dl
 0x10004eb8 xor     edx, 0A4h
 0x10004eb1 movsx   edx, byte_1003B03C
 0x10004eab mov     byte_1003B03C, cl
 0x10004ea5 imul    ecx, 0A2h
 0x10004e9e movsx   ecx, byte_1003B03C
 0x10004e99 mov     byte_1003B03C, al

The code also tracks how some general purpose instructions manipulate different registers. Most of them are simple due to the x86 standard of instruction destination source format. Not all of them are though. I spent a good amount of time wondering what variables to back trace when following instructions such as DIV. Is EAX or the DIV operand more important back trace? I went with the operand but in the future I plan on creating back split trace that will track EAX and the operand passed to DIV.  Odds are there are still more general purpose instructions I need to check for. XADD is a pretty cool instruction. The shortest Fibonacci can be written using XADD. 

This version was written in order for me to crack an obfuscation technique that I have seen lately. Using backtrace.py and the last line of the dead code blocks I'm able to identify most of the junk code and variables. I'm sure there are flaws (like not tracing push or pops...future release) but so far it is working well for me. I hope the code is of use to others. If you have any recommendations, thoughts, etc please shoot me an email (line 20 of the source code) or ping me on twitter.

IDA Thread Analysis Script

In a recent post, I talked about renaming subroutine-blocks and identifying them in IDA. This technique is very helpful for identifying large block of functions where the parent and child functions are self-contained. Library code is a good example of code that is self contained. Take for example a zip library. The zip library is linked to an executable file. To call it we would pass a buffer to the zip function, the buffer would get passed to the child functions, the zip functions will do their magic and a compressed buffer is returned (simplified version). All of the child functions that were responsible for doing the zip magic would be considered a sub-routine block. They are all related and self-contained. What about a set of functions that are related but not self-contained? A perfect example of this would be a thread.

Usually threads are designed to serve one purpose. A thread could contain functions that are called from other functions, which would not make it self contained. Due to the single purpose all the functions would be related. By identifying the code path of a specific thread we might be able to help with enumerating the threads functionality. First we will need to get the address for each call to CreateThread. This can be done using LocByName("CreateThread",0).

def retlistofCreateThreadAddr():
    addr = []
    for x in CodeRefsTo(LocByName("CreateThread"),0):
        addr.append(x)
    return addr

We will then need to get the offset that is pushed on to the stack for the thread's function start address (lpStartAddr). This is the third argument.

push    eax             ; lpThreadId
push    ebx             ; dwCreationFlags
push    esi             ; lpParameter
push    offset StartAddress_SearchFiles ; lpStartAddress
push    ebx             ; dwStackSize
push    ebx             ; lpThreadAttributes
call    ds:CreateThread
cmp     eax, ebx
jz      short loc_1000341E

From MSDN

HANDLE CreateThread(
  LPSECURITY_ATTRIBUTES lpsa,
  DWORD cbStack,
  LPTHREAD_START_ROUTINE lpStartAddr,
  LPVOID lpvThreadParam,
  DWORD fdwCreate,
  LPDWORD lpIDThread
);

 // lpStartAddr: [in] Long pointer to the application-defined function of type
 // LPTHREAD_START_ROUTINE to be executed by the thread; represents the starting 
 // address of the thread. For more information on the thread function, see ThreadProc.

IDA is usually good at identifying lpStartAddr. If we rely on IDA, we can back trace a number of instructions from the address found in retlistofCreateThreadAddr() until we find the string "lpStartAddr" in the comments. Once we have the address we just need to read a Dword for the threads function start address. There are a couple of flaws to this approach. One is that we are relying on IDA for comments and another is we are relying on lpStartAddr to be a Dword address. The function address could be chosen at runtime. If this is the case we won't be abel to find lpStartAddr. The code will then need to be manually analyzed. An easy way to determine if we were able to receive the lpStartAddr is to check if it's a valid address using GetFunctionName(lpStartAddr).

def getStartAddr(ct_addr):
    # backtrace to find string "lpStartAddress"
    count = 0
    addr = PrevHead(ct_addr,minea=0)
    while count < 10:
        if 'lpStartAddress' in str(Comment(addr)):
            return Dword(addr+1) 
        count = count + 1
        addr = PrevHead(addr,minea=0)
        continue 

Once we have the lpStartAddr we just need to get all of it's child functions. For this we can use a modified version of the stolen code (graph_down function) from Carlos G. Prado. The modification returns a list that contains the function name and the address of where a call is at. This list can be used to get the depth of each function. Once we have the depth we can display each threads and all of it's child functions in a structured output in IDA.

This script will be included in the upcoming release of IDAScope. In the previous post we used  a script to rename all functions in a subroutine block. The same script can be used for renaming all child functions in a thread. For the IDAScope release of this script it will be in it's own window.  Plus, the script will have an option to add a repeating comment to all child functions in a thread. Constant appending to the function name starts to clutter it up. Dan has been doing some awesome work on IDAScope (makes my updates look like sophomore programing examples).

 For anyone who wants the code now the script and code can be found below.

Note: Calculating the depth is probably the slowest part of the code. I tried to figure out away to get the depth from inside the graph_down function but I had no luck. I spent a good amount of time reviewing others code in graphing down and graphing up. Cody Pierce has some great code on Tipping Point's blog but it's for graphing up. If anyone has any thoughts please shoot me an email. My address is in the comments of the script.

Source code of an_threads.py, Download

## an_threads.py is a script that can be used to help with analyzing threads
## and their child functions. Usage IDA > File > Script file.. > Select an_threads.py
## The output will be displayed to the Output Window. IDA > View > Output Window
## Created by alexander.hanel@gmail.com, version 0.01 

from idaapi import * 
import idautils
import idc
import sys

def format_depth(x):
    # Get's the depth of each function from the parent/root function
    for index in range(0, len(x)):
        if x[index][1] == None:
            x[index].append(0)
            continue
        if index == 1:
            x[index].append(1)
            continue
        # Indent Child Function 
        if x[index][0] == x[index-1][1]:
            x[index].append(x[index-1][2]+1)
            continue
        # No Indent same function 
        if x[index][0] == x[index-1][0]:
            x[index].append(x[index-1][2])
            continue
        if x[index][0] != x[index-1][1] or x[index][0] != x[index-1][0]:
            for v in range(1, index):
                if len(x[index]) == 3: continue 
                if x[index][0] == x[v][0]:
                    x[index].append(x[v][2])
                    continue
                
        if len(x[index]) == 3:
                continue
    # returns list
    # format parent, child, depth 
    return x
    
def print_dep(dep):
    # prints the output
    for line in dep:
        if line[1] == None:
            print GetFunctionName(int(line[0],16)), "(lpStartAddr)"
            
        else:
            space = ' ' * 3 * line[2]
            func_string = GetFunctionName(int(line[1],16))
            if func_string == '':
                 func_string = '* Call ' + GetDisasm(int(line[1],16))[6:-6]
            print space , func_string

    return 
    
def graph_down(ea, depth, graph = {}, path = set([]) ):
    # This function was borrowed from Carlos G. Prado. Check out his Milf-Plugin for IDA on Google Code. 
    graph[ea] = list()    # Create a new entry on the graph dictionary {node: [child1, child2, ...], ...}
    path.add(ea)        # This is a set, therefore the add() method

    # Iterate through all function instructions and take only call instructions
    for x in [x for x in FuncItems(ea) if is_call_insn(x)]:        # Take the call elements
            for xref in XrefsFrom(x, XREF_FAR):                                   
                    if not xref.iscode:
                            continue

                    if xref.to not in path or 'extrn' in GetDisasm(xref.to):
                        depth.append([hex(LocByName(GetFunctionName(x))), hex(xref.to)])
                    
                    if xref.to not in path:        # Eliminates recursions
                            graph[ea].append(xref.to)
                            graph_down(xref.to, depth, graph, path)
    return depth

def retlistofCreateThreadAddr():
    # returns a list of all addresses that call CreateThread
    addr = []
    for x in CodeRefsTo(LocByName("CreateThread"),0):
        addr.append(x)
    return addr

def getStartAddr(ct_addr):
    # backtrace to find string "lpStartAddress"
    # then read and return Dword
    count = 0
    addr = PrevHead(ct_addr,minea=0)
    while count < 10:
        if 'lpStartAddress' in str(Comment(addr)):
            return Dword(addr+1) 
        count = count + 1
        addr = PrevHead(addr,minea=0)
        continue         


## Main()
threads = []   
for x in retlistofCreateThreadAddr():
    # return (CreateFunction Address, StartAddress)
    threads.append((x,(getStartAddr(x)))) 
                   
print "Number of Threads %s" % (len(threads))
for addr in threads:
    print "CreateThread Call %s" % hex(addr[0])
    if GetFunctionName(addr[1]) == '':
        print "[Warning] Could Not Get lpStartAddr [Warning]"
        print 
        continue
    x = graph_down(addr[1], depth=[[hex(LocByName(GetFunctionName(addr[1]))),None]])
    print_dep(format_depth(x))
    print