####################################################################### QuickBMS by Luigi Auriemma e-mail: aluigi@autistici.org web: aluigi.org home: http://aluigi.org/papers.htm#quickbms ####################################################################### 1) Introduction 2) Usage 3) How to create scripts 4) Notes 5) Support 6) Additional credits ####################################################################### =============== 1) Introduction =============== QuickBMS is a multiplatform extractor engine that can be programmed through some simple textual scripts for extracting the files from the archives of any program and moreover games. The script language from which QuickBMS is derived is MexScript documented here: http://wiki.xentax.com/index.php/BMS So QuickBMS is FULLY compatible with that original syntax and all the scripts that were created here: http://forum.xentax.com/viewtopic.php?t=1086 Anyway I improved that language for removing some implied fields (like the file number in some commands) and adding new commands (like Encryption) and behaviours (like negative GoTo) that allow QuickBMS to work with tons of simple and complex formats. The tool is open source under the GPL license and works on both Windows and *nix and on both little and big endian architectures like Intel and PPC. For *nix users there is a Makefile in the src folder, the only requirements are openssl, zlib, bzip2, lzo, ucl libraries. lzo and ucl can be disabled adding -DDISABLE_LZO and -DDISABLE_UCL at compiling. ####################################################################### ======== 2) Usage ======== Simple and quick: - double-click on quickbms.exe - select the script for the type of archive you want to extract - select the archive - select the output folder where extracting the files (you can specify any file, it will be ignored because only the directory is taken) - watch the progress of the extraction That one was the simple "GUI" usage but QuickBMS can do various other things when launched from the console because it supports various command-line options. You can view all the available options simply launching QuickBMS without arguments, example: Usage: quickbms [options] Options: -l list the files without extracting them, you can use . as output folder -f W filter the files to extract using the W wildcard, example -f "*.mp3" -F W as above but works only with the files in the input folder (if used) -o if the output files already exist this option will overwrite them automatically without asking the user's confirmation -v verbose debug informations, useful for verifying possible errors -c quick list of BMS commands and some notes about them and this tool -L F dump the offset/size/name of the files inside the file F -x use the hexadecimal notation in myitoa (debug) Remember that the script, the input archive and the output folder are ever REQUIRED and must be specified at the end of the command-line so if you want to list all the mp3 files in a particular archive you must use: quickbms -l -f "*.mp3" zip.bms myfile.zip . the last dot is the output folder (the current one) but it's ignored when is used the -l option. From version 0.3 QuickBMS supports also a folder as input which means that with a single command is possible to unpack all the archives of a game without using the "find" tool (http://gnuwin32.sourceforge.net/packages/findutils.htm). Imagine to use the zip.bms script with all the zip files located in the c:\windows\temp folder: quickbms -F "*.zip" zip.bms "c:\Program Files" c:\output_folder Except for -l, -f, -F and maybe -o options the other are intended for debugging and so should be ignored by the normal users. ####################################################################### ======================== 3) How to create scripts ======================== Originally the tool was created for myself for making me able to write quick extractors for simple archives immediately without writing a line of C but it revealed to be a so powerful tool that now I use it really for anything included the parsing of some protocols and much more. So, how to write these scripts? Well I guess that giving a look at http://wiki.xentax.com/index.php/BMS is a good first step to understand at least the basis, then it's good to take a look at the examples provided on http://aluigi.org/papers.htm#quickbms and then to the following list of available commands and their description. A programming knowledge and background is not required but it's very useful for entering in the "logic" of the scripts, while it's required the full knowledge of the format to implement: reversing is ever useful for figuring the needed fields. Luckily in the extraction process is not needed to know all the fields of an archive so a field like a CRC just doesn't matter while the important fields are ever the filename, the offset and the size of the files to extract. You can notice all these useless fields in my scripts because I assign them the name DUMMY. Note that I will try to keep the following documentation update as much as I can, in any case refer also to the source code of quickbms.c for any other technical doubt or possible additions. The fields between [] are optionals. --- A quick list of available commands is available also when QuickBMS is launched with the -c option, and REMEMBER to read the notes specified there like the fact that: - everything is considered a variable except if it starts with a number in which case it's considered a numeric constant so when in this text I talk about VAR, STRING and other types of data I refer EVER to both variables and constants because they are EXACTLY the SAME thing - all the commands and the names of the variables are case INsensitive so "get OFFSET long" is the same as "GeT oFfSeT lOnG" - everything works with signed 32 bit numbers (-2147483648 to 2147483647) so QuickBMS at the moment could not work well with files over 2 gigabytes - the so called constant strings (depends by the context of the command) are handled as strings in C notation like "\x12\x34\\hello\"bye\0" and so on, in this case you must know a bit how this representation works. the keyword is "C language escape characters" or escape sequences, they are very simple, take a look here: http://msdn.microsoft.com/en-us/library/h21280bw%28VS.80%29.aspx http://www.acm.uiuc.edu/webmonkeys/book/c_guide/1.1.html - are supported also hexadecimal numbers if they start with 0x so 1234 and 0x4d2 are the same thing - any operation made on fields bigger than 8 bits is controlled by the global endianess, which means that any number and unicode field is read in little endian by default otherwise it's valid the endianess specified with the Endian command File numbers: every file opened in QuickBMS has a number assigned to it and in all the commands that access to files this number is 0 (main file) if not specified. the first opened file is the input archive to which is assigned the 0 number, the others must be opened with the Open command MEMORY_FILEs: this is a particular type of temporary file which resides in memory and works exactly like a normal file. it's extremely useful for doing tons of things and are supported various memory files: MEMORY_FILE, MEMORY_FILE2, MEMORY_FILE3, ... TEMPORARY_FILE: exists also another type of file called TEMPORARY_FILE which instead resides phisically on the hard-disk with that exact name. although it's "temporary" name it's not deleted by the output folder and is created even when it's used the -l option for listing the files which makes it ideal in certains situations like when it's used a chunks based file system. indeed in this case using a MEMORY_FILE is a bad idea because the continuous reallocation of the memory makes it terribly slow. for using the temporary file remember to use it like in the following example: log TEMPORARY_FILE 0 0 # reset it in case it already exists append # enables the append mode ... log TEMPORARY_FILE OFFSET SIZE ... append # disable the append mode open "." TEMPORARY_FILE 1 # open the temporary file on the file number 1 Types of variables supported, also know as datatypes or types: BYTE 8 bit, 0 to 0xff SHORT 16 bit (aka INT), 0 to 0xffff THREEBYTE 24 bit, 0 to 0xffffff LONG 32 bit, 0 to 0xffffffff LONGLONG fake 64 bit, so only 0 to 0xffffffff but takes 8 bytes in Get STRING null delimited string ASIZE special type used to return the size of the opened file, used only with the GET command FILENAME special type used to return the name of the opened file like "myfile.zip", used only with the GET command BASENAME special type used to return the base name of the opened file like "myfile", used only with the GET command EXTENSION special type used to return the extension of the opened file like "zip", used only with the GET command UNICODE special type used for unicode utf16 strings, the endianess of the utf16 is the same used globally in the script (watch the Endian command) BINARY special type used for binary strings in C notation like "\xff\x00\x12\x34", used mainly as a constant LINE special type used for carriage return/line feed delimited string FULLNAME full path of the file, in reality at the moment it returns the same path used in the input filename And now the list of commands in the same order in which they are listed in the QuickBMS source code in the function parse_bms(): ....................................................................... QuickBMSver VERSION checks if the current version of QuickBMS is enough recent to support the script. arguments: VERSION version of QuickBMS for which was created the script examples: QuickBMSver 0.2.4 ....................................................................... Clog NAME OFFSET ZSIZE SIZE [FILENUM] extracts the file decompressing it in real-time, this operation doesn't affect the current position of the input file. the decompression algorithm used in the operation is decided by the ComType command. arguments: NAME name of the output file OFFSET position of the archive where is located the file ZSIZE size of the compressed data in the archive SIZE size of the uncompressed file FILENUM number of the file associated to the archive (0) examples: Clog NAME OFFSET ZSIZE SIZE Clog "dump.dat" 0 ZSIZE 10000000 # the file will have the real size and not 10000000 ....................................................................... FindLoc VAR TYPE STRING [FILENUM] [ERR_VALUE] finds the first occurrency of a given string or number from the current offset of the archive. it's usually used when the format of the archive is not known or it a particular text file... not much used. arguments: VAR the variable which will receive the offset of the occurrency TYPE can be STRING, UNICODE or a number STRING string in C notation or number, depending by the TYPE FILENUM number of the file associated to the archive (0) ERR_VALUE by default FindLoc terminates the script if no string is found but if ERR_VALUE is specified this value will be assigned to VAR without terminating when there are no other occurrencies examples: For FindLoc OFFSET string "filename=" ... Next ....................................................................... For [VAR] [=] [VALUE] [COND] [VAR] ... Next [VAR} a classical "for" loop with initializers, conditions and incrementers. there is also the Break instruction available to break the loop at any moment. Next is the command which delimits the loop and at the same time increments the given variable if specified. practically it means: - give VALUE to VAR - performs a loop till VAR is "condition" than the other VAR - performs all the operations between For and Next - increments VAR arguments: VAR variable to which assign a value = fixed VALUE value to assign to the variable COND condition, the same of the IF command VAR second part of the condition examples: For i = 0 < FILES ... next i ....................................................................... Get VAR TYPE [FILENUM] reads strings and numbers from the open files, it's also the most used command. arguments: VAR variable which will receive the read data TYPE watch the description of the types explained before FILENUM number of the file associated to the archive (0) examples: Get OFFSET long Get NAME string ....................................................................... GetDString VAR LENGTH [FILENUM] reads a defined amount of data from the file and stores it in the given variable. arguments: VAR variable which will receive the read data LENGTH amount of bytes to read FILENUM number of the file associated to the archive (0) examples: GetDString NAME NAME_LENGTH GetDString NAME 0x100 ....................................................................... GoTo OFFSET [FILENUM] go to the given absolute position of the file, seek arguments: OFFSET position to reach, if it's negative it will be considered an "OFFSET bytes from the end of the file" position note that from QuickBMS 0.3.5a only the constants can be negative with GoTo, all the variables will be handled as unsigned (0 to 0xffffffff) FILENUM number of the file associated to the archive (0) examples: GoTo OFFSET GoTo 0x100 GoTo -4 # 4 bytes before the end of the file ....................................................................... IDString [FILENUM] STRING terminates if the the signature at the current position of the file differs than the provided string arguments FILENUM number of the file associated to the archive (0) STRING string in C notation examples: IDString "PK\x03\x04" IDString " KAP" ....................................................................... Log NAME OFFSET SIZE [FILENUM] extracts the file, this operation doesn't affect the current position of the input file arguments: NAME name of the output file OFFSET position of the archive where is located the file SIZE amount of the data to extract FILENUM number of the file associated to the archive (0) examples: Log NAME OFFSET SIZE Log "dump.dat" 0 SIZE ....................................................................... Math VAR OP VAR performs a mathematical operation between the two variables and places the result in the first one arguments VAR variable which acts as input and output OP + sum * multiplication / division - substraction ^ xor & and | or % modulus ! negation of var2 (0 becomes 1 and any other value becomes 0) ~ complement of var2 (like "xor 0xffffffff") < shift left > shift right l rotate left r rotate right s byte swapping w bit swapping = assign var2 to var1 n negative value of var2 a absolute value of var2 VAR other input variable examples: Math SIZE *= 0x100 Math OFFSET <<= 2 Math TMP = SIZE Math TMP ~ TMP Math TMP n TMP Math TMP2 a TMP ....................................................................... Open FOLDER NAME [FILENUM] opens a file for reading, practically assigns a file number/id to an existent file that you want to use arguments: FOLDER FDDE, means that you want to open the file in the same location of the input one which has the extension provided with NAME FDSE, it will consider NAME as a file located in the same folder of the input file (very useful) any other value is considered the folder where is located the file to load so use "." for the current output folder NAME read above, NAME can be also a ? in which case QuickBMS will ask the user to insert the name of the file to open manually FILENUM number of the file associated to the archive (0) examples: Open FDDE DAT 0 Open FDDE IDX 1 Open FDSE "myfile.zip" Open "." TEMPORARY_FILE 1 ....................................................................... SavePos VAR [FILENUM] returns the current position of the file arguments: VAR variable which will contain the offset FILENUM number of the file associated to the archive (0) examples: SavePos OFFSET ....................................................................... Set VAR [TYPE] VAR command for assigning a value to a variable arguments: VAR output variable or memory file TYPE useless type, indeed it can be also not specified in which case will be used the String value. it's useless because in QuickBMS doesn't exist a real difference between numbers and strings. remember that here TYPE can have also the special BINARY type explained before VAR variable or constant to assign examples: Set i long 0 Set TMP long SIZE Set TMPNAME NAME Set MEMORY_FILE binary "\x12\x34\x56\x78" ....................................................................... Do ... While VAR COND VAR another less used type of cycle where is performed the check of the condition at the end of the cycle... really rarely used. arguments: VAR first part of the condition COND condition, check the If command below for additional info VAR second part of the condition examples: Do ... While OFFSET < MAX_OFFSET ....................................................................... String VAR OP VAR the equivalent of the Math command for the strings arguments: VAR input and output variable OP + append the second string to the first one - if the second variable is a positive number the string will be truncated at that amount of bytes from the end if the second variable is a negative number the string will be truncated at that amount of bytes from the beginning otherwise will be removed all the occurrencies of the second string in the variable ^ xoring of the string with the second one (looped if shorter) < var1="thisisastring", var2="4", result="isastring" % truncate the variable at the position obtained by the modulus of its length and the number in the second variable & var1="thisisastring", var2="isa", result="isastring" | var1="thisisastring", var2="isa", result="string" > if the second variable is a number: var1="thisisastring", var2="4", result="thisisast" otherwise: var1="thisisastring", var2="isa", result="this" VAR the second variable or string examples: string FULLPATH += NAME string FULLPATH += \ string NAME -= ".zip" string NAME -= 4 ....................................................................... CleanExit terminates the script ....................................................................... If VAR COND VAR ... [Elif VAR COND VAR] ... [Else] ... EndIf checks various conditions and performes the needed operation when the condition is verified, translated: - If is ever the first condition - Elif is another condition and can be used endless times - Else is the operation to do when no conditions are met - EndIf delimits the If command arguments: VAR first part of the condition COND valid for both strings and numbers: < minor > major != different == equal >= major/equal <= minor/equal & string: var2 is included in var1 number: logical AND ^ string: equal number: logical XOR | number: logical OR % number: modulus / number: division << number: shift left >> number: shift right ! number: negation ~ number: complement ext compares the string after the last dot basename compares the string before the last dot VAR second part of the condition Examples: If NAME != "" ... Endif If MASK & 1 Elif MASK & 2 Elif MASK & 4 Elif MASK & 8 Else Endif ....................................................................... GetCT VAR TYPE CHAR [FILENUM] reads a string till the reaching of the CHAR delimiter. arguments VAR output variable TYPE ignored because doesn't exist a type in this operation CHAR the delimiter character as 8bit number FILENUM number of the file associated to the archive (0) examples: GetCT NAME string 0x0a GetCT NAME string 0x3b GetCT NAME string DELIMITER_BYTE ....................................................................... ComType ALGO [DICT] selects the specified compression algorithm to use with the Clog command. from version 0.3.8 of QuickBMS I have also added the possibility of choosing a number as ALGO for an "utopic" idea about a scanner for being able to guess the possible compression algorithm in an unknown compressed data block: http://aluigi.org/papers/bms/comtype_scan.bms obviously this feature is only for advanced people who knows exactly what they are doing. note that some algorithms work only on Windows arguments: ALGO zlib, RFC 1590 (aka windowbit 15, the data starts with a 'x') DICT supported deflate, RFC 1591 (aka windowbit -15) used for example in the ZIP files DICT supported lzo1a till lzo2a, LZO (remember that the most used is lzo1x) DICT supported lzss, with default configuration (dictionary of 4096 bytes) this particular algorithm can be fully configured setting the EI, EJ and P fields plus another number rarely used. for setting them it's enough to use a DICT equal to something like "12 4 2" which means EI:12 (N:4096), EJ:4 (F:18), P:2 lzx, used by the old (aka jurassic) unlzx tool and on Amiga gzip, automatic handling of the gzip data remember that in this case the uncompressed size is ignored and calculated automatically so in CLog use ZSIZE ZSIZE pkware, the algorithm also known as blast/explode/implode/DCL lzma, 5 bytes + lzma lzma86head, 5 bytes + 8 bytes (size) + lzma lzma86dec, 1 byte + 5 bytes + lzma lzma86dechead, 1 byte + 5 bytes + 8 bytes (size) + lzma lzmaefs, the format implemented in ZIP bzip2 XMemDecompress, Xbox 360 LZX algorithm of xcompress.lib use DICT to specify a custom WindowSize and CompressionPartitionSize like "131072 524288" hex, from "01234567" to 4 bytes: 0x01 0x23 0x45 0x67 base64, from "aGVsbG8=" to "hello", supports also the Gamespy and URL chars uuencode ascii85 yenc COM_LZW_Decompress, used in Vietcong milestone_lzw, the lzw algorithm used in the Milestone games lzxcab, the lzx algorithm used in the handling of the cab files (libmspack 21 0) lzxchm, the lzx algorithm used in the handling of the chm files (libmspack 16 2) rlew, 16 bit RLE algorithm used in AIM Racing lzjb, a compression used in a file system for *nix sfl_block, expand_block from iMatix Standard Function Library sfl_rle, expand_rle from iMatix Standard Function Library sfl_nulls, expand_nulls from iMatix Standard Function Library sfl_bits, expand_bits from iMatix Standard Function Library lzma2, 1 bytes + lzma2 lzma2_86head, 1 bytes + 8 bytes (size) + lzma2 lzma2_86dec, 1 byte + 1 bytes + lzma2 lzma2_86dechead, 1 byte + 1 bytes + 8 bytes (size) + lzma2 nrv2b, UCL nrv2d, UCL nrv2e, UCL huffboh, an unrecognized compression used in the Asura engine uncompress, the lzw algorithm used in the compress utility (the lzw data starts from offset 3 of the .Z files) dmc, Dynamic Markov Compression (DMC) lzhuf, aka LZH/LHA lzari rle7 rle0 rle rlea, another generic rle decompressor use DICT to choose the escape char bpe, byte pair encoding quicklz q3huff, Adaptive Huffman algorithm used in the Quake 3 engine unmeng, algorithm used in DreamKiller lz2k, algorithm used in various games developed by Traveller's Tales darksector, a very basic algorithm used in the game Dark Sector mszh, used in the LossLess Codec Library un49g, used in the games of 49Games unthandor, used in the old game Thandor doomhuff, huffman used in doom, hexen, skulltag and other doom ports the DICT field can be used to specify a custom HuffFreq table (256 float elements) aplib tzar_lzss, used in Tzar of HaemimontGames DICT must contain the name of the variable with the algorithm number to use, example: ComType tzar_lzss MYVAR lzf, aka fastlz clz77, the lz77 algorithm available on http://compressions.sourceforge.net/about.html lzrw1 dhuff, Huffman Decompression in LDS ("lossless datacompression sources" kit 1.1) fin, from LDS lzah (not tested) lzh12, aka -lh4- lzh13, aka -lh5- grzip, aka GRZipII ckrle, Chilkat RLE quad, note that in this version I have removed the handling of the first 32bit number containing the size of the uncompressed data balz, note that in this version I have removed the handling of the first 9 bytes from the files that contains an ID byte and a 64bit field containing the size of the uncompressed data deflate64 shrink (not tested) ppmd, ppmd var.i rev.1 with ZIP specifics so 2 bytes of info followed by the compressed data z-base-32 base32hex base32crockford base32nintendo base???, if ALGO starts with "base" then will be taken its subsequent number (for example 32 if it's "base32") and used for the conversion. the function supports ANY base conversion from 2 to 256. for bases major than 64 will be used a char table starting from byte 0x00 so base128 will have a charset from 0 to 0x7f brieflz paq6, raw data block shcodec hstest_hs_unpack (never tested, could be removed in future) hstest_unpackc (never tested, could be removed in future) sixpack (never tested) ashford (never tested, could be removed in future) jcalg jam lzhlib srank zzip scpack DICT supported (for the SCPACK_TABLE field) rle3 bpe2 bcl_huf, Basic Compression Library bcl_lz, Basic Compression Library bcl_rice, Basic Compression Library you must use DICT to specify the format (1 to 8) bcl_rle, Basic Compression Library bcl_sf, Basic Compression Library scz szip ppmdi1_raw ppmdg ppmdg_raw, requires DICT "par1 par2" ppmdj1 ppmdj1_raw, requires DICT "par1 par2 par3" sr3c huffmanlib sfastpacker, smart+simple mode sfastpacker2, smart-mode only dk2, used in Dungeon Keeper 2 of Bullfrog/EA lz77wii, (use input size as output size in clog) lz77wii_raw darkstone, lz77 compression used in the game DarkStone sfl_block_chunked, as above with automatic handling of the chunks if used DICT an optional C string containing the bytes of the dictionary or particular parameters depending by the chosen algorithm ....................................................................... ReverseLong VAR inverts a variable from 0x44332211 to 0x11223344 and viceversa depending by the current endianess. arguments: VAR variable to flip ....................................................................... Endian TYPE changes the current endianes of the read/written data, the default one is little endian. arguments: TYPE little, intel endianess where 0x11223344 is stored as 44 33 22 11 big, network endianess where 0x11223344 is stored as 11 22 33 44 Examples: print "little endian" endian big print "big endian" endian little print "little endian" ....................................................................... FileXOR SEQ [OFFSET] any read operation on any file will perform also the xoring of the read data with the numbers contained in the given string or in the given variable. the OFFSET field by default is zero which means that if the data must be xored with more than one byte (a "xor key") the first byte of the xor key is the first byte at OFFSET which is 0 by default (beginning of the file). resuming: the FileXOR command works with ANY file access arguments: SEQ sequence of space-separated 8bit numbers, it can be a: - sequence of bytes separated by space like 0x12 or "0x12 0x34 0x56" - a numeric variable at the moment is not possible to use a key in string mode (use the Encryption command for doing it) so if you have a string convert it to a numeric sequence first. set it to 0 or "" for disabling the xor OFFSET needed only for the xor key. if the archive is xored with a xor key this argument is usually not necessary if instead it's the file to extract that is xored, this argument must have the same offset of the file (so just reuse the same OFFSET used in Log) examples: filexor 0xff filexor "0x12 0x34 123 255" filexor MYXORBYTE filexor "0x12 0x34 123 255" OFFSET Log NAME OFFSET SIZE ....................................................................... FileRot SEQ [OFFSET] exactly as for FileXOR but it performs a sum operation. for example if SEQ is 0x01 and the file contains "hello" it will become "ifmmp" while if SEQ is -1 or 0xff it will become "gdkkn". -1 and 0xff are the same because it's a 8 bit number resuming: the FileRot command works with ANY file access watch the previous arguments and examples. ....................................................................... Strlen VAR VAR calculates the length of the second variable and stores it in the first one. the length is the amount of bytes till the reaching of a 0 delimiter. arguments VAR destination variable which will contain the length VAR variable of which calculating the length examples strlen NAME_LENGTH NAME ....................................................................... GetVarChr VAR VAR OFFSET [TYPE] a particular and sometimes very useful command which works exactly like accessing to an array of elements contained in the second variables (so a string or a memory file). this simple but effective method allows the customization of strings and variables for example for creating a custom header (like a DDS) and moreover for performing operations on a piece of the memory (like a custom encryption algorithm). some real examples are my Deer Hunter 2004/2005 scripts. arguments VAR destination variable which will contain the read element VAR variable or memory file from which you want to get the element OFFSET position of the second variable where taking the element TYPE if not specified it's a BYTE so a 8bit number, you can choose any of the available data types and it will go in the destination variable examples: For i = 0 < SIZE GetVarChr TMP MEMORY_FILE i GetVarChr TMP MEMORY_FILE i long GetVarChr TMP MEMORY_FILE i string Next i ....................................................................... PutVarChr VAR OFFSET VAR [TYPE] the "write-mode" alternative of the previous command which allows to perform various complex operations with custom algorithms (like in my Deer Hunter 2004/2005 scripts). note that from version 0.3.3 PutVarChr can be also used as an allocator of memory that could be useful for example in the implementation of custom decompression algorithms. arguments VAR variable or memory file to which you want to put the element OFFSET position of the first variable where placing the element VAR source variable which will contain the element to write TYPE as above examples: For i = 0 < SIZE GetVarChr TMP MEMORY_FILE i Math TMP ^= 0xff PutVarChr MEMORY_FILE i TMP Next i ....................................................................... Debug switch command which enables the -v option for a specific portion of the script, used only for debugging ....................................................................... Padding VAR [FILENUM] when called it performs an automatic GoTo to the next position of the file skipping the padded data. imagine to have a file where it's used an alignment of 4 bytes and your current file offset is 0x39, if you use Padding 4 the offset will be automatically changed to 0x3c. arguments: VAR size of the alignment, for example 4 or 16 and so on FILENUM number of the file associated to the archive (0) examples: Get NAME string Padding 4 get OFFSET long ....................................................................... Append switch command which enables the append mode in the *Log commands, so if the output filename is still the same it will be not overwritten but concatenated. note that the user must ever confirm the appending for security reasons. examples: append Log "dump.dat" 0 0x10 Log "dump.dat" 0x10 0x100 the following is a particular example for allocating a MEMORY_FILE (from version 0.3.3) and using it instead of TEMPORARY_FILE saving space on the disk and performances: math TMP = CHUNKS math TMP *= 0x8000 log MEMORY_FILE 0 0 putvarchr MEMORY_FILE TMP 0 log MEMORY_FILE 0 0 append for i = 0 < CHUNKS ... clog MEMORY_FILE OFFSET ZSIZE 0x8000 next i ....................................................................... Encryption ALGO KEY [IVEC] [MODE] one of the most interesting commands which allow to set a decryption algorithm used for any read operation on the files. resuming: the Encryption command works ONLY with Log and CLog arguments: ALGO aes, Rijndael blowfish des 3des-112 3des-168 rc4 tea xtea idea xor rot charset, the substitution algorithm which uses a charset of 256 chars charset2, as above but the substituition is inverted (confortable in some rare cases) twofish cast5 seed serpent ice icecrypt, ICE algorithm with key implemented as in the homonym program, the difference with "ice" is ONLY in the key rotor, added as experiment, ivec contains the number of rotors (12) ssc, Leverage SSC any other algorithm and variant supported by OpenSSL which are a lot and so can't be listed here, check AUTO_MYCRYPTO in quickbms.c "", disable the encryption KEY the key to use in C notation like "\x11\x22\x33\x44" or "this is my key" from version 0.3.9 this value can be also a variable or a memory file IVEC the ivec to use in C notation, an ivec is an additional key used for increasing the security of encryption algorithms that are usually defined as ECB without ivec and CBC with ivec MODE 0 for decryption (default), 1 for forcing the encryption mode if no ivec is used remember to place a "" at its place examples: Encryption aes "0123456789abcdef" "" 1 # encryption without ivec Log MEMORY_FILE 0 SIZE Encryption aes "0123456789abcdef" # decryption without ivec Log "redecrypted_file.dat" 0 SIZE MEMORY_FILE Encryption aes "\x12\x34\x56\x78" set MEMORY_FILE binary "\x12\x34\x56\x78" Encryption aes MEMORY_FILE Encryption aes MY_VARIABLE ....................................................................... Print MESSAGE prints a string in C notation with the values of the variables if they are specified between two % chars. arguments: MESSAGE C notation string, each %VAR% word is converted to its value examples: print "the variable OFFSET of the file %FILENAME% has the value %OFFSET%" print "this is the first line\nthis is the second line\n" ....................................................................... GetArray VAR ARRAY VAR and PutArray ARRAY VAR VAR experimental commands implemented for a possible future usage but that at the moment have been never used. they work on a dyniamc array where is possible to store data, something like a temporary place or a stack. at the moment this function doesn't have a purpose so ignore it. ....................................................................... CallFunction NAME StartFunction NAME ... EndFunction calling and declaration of a function identified by NAME where the values of the variables are saved till the termination of the function when they are restored. it works very well for recursive archives like those used by "The Void" and "Another Day" although it could result not much immediate in its usage to understand but it's very powerful. so remember the rule: the content of the variables is restored when the function terminates. it's a good idea to place all the functions (StartFunction till EndFunction) at the end of the scripts. arguments: NAME name assigned to the function examples: watch thevoid.bms and another_day.bms ....................................................................... ScanDir PATH NAME SIZE function without a real usage, it simply scans the PATH folder and fills the NAME and SIZE variables with the name and the size of each file found. at the moment this function doesn't have a purpose so ignore it. arguments: PATH must be ".", the current folder NAME output variable which receives the name of the file, it will be "" when there are no other files SIZE output variable which receives the size of the file, it will be -1 when there are no other files examples: For ScanDir "." NAME SIZE if NAME == "" cleanexit endif Next ....................................................................... CallDLL DLLNAME FUNC/OFF CONV RET [ARG1] [ARG2] ... [ARGn] this is the command which allows to use plugins inside QuickBMS. the idea came from the possibility of using the custom decompression/decryption functions (both exported and internals) located in executables and dlls avoiding the boring reversing of all the functions. unfortunately this is not much possible with the functions got from executables where are used static variables due to some technical reasons (memory addresses that don't match), for example if the function uses the memory between 006c0000 and 006d0000 it's highly possible that such range of memory is not allocated or is already in use because the executable has not been loaded (LoadLibrary) in its original address because already occupied. obviously there are no problems with the dlls. arguments: DLLNAME name of the dll or executable where is located the function, example "mylib.dll" QuickBMS can even load raw binary files that contain only the dumped function... very useful FUNC/OFF it can be the name of the function to import in which case it must be exported by the dll/exe or the relative offset where is located the function, remember that the relative offset is NOT the absolute one but it's the offset related to the image base of the exe/dll (so if normally the dll loads at offset 10000000 and the function is at 10012345 then the offset is 0x12345) CONV calling convention: stdcall: aka winapi, used by Visual C cdecl: used by almost any other C/C++ compiler RET the variable which will contain the value returned by the function, use "" if there is not return value [ARGS] all the arguments of the function from version 0.3.5 QuickBMS implements also the pointer to arguments when they are preceded by a & or a * like &SIZE, ZSIZE. it works only with numeric variables examples: idstring LZ2K get SIZE long get ZSIZE long log MEMORY_FILE 0xc ZSIZE putvarchr MEMORY_FILE2 SIZE 0 # like malloc #calldll "TransformersDemo.exe" 0x263c50 cdecl "" MEMORY_FILE MEMORY_FILE2 ZSIZE SIZE # 00663C50 calldll "unlzk.dll" "unlz2k" cdecl SIZE MEMORY_FILE MEMORY_FILE2 ZSIZE SIZE log "dump.dat" 0 SIZE MEMORY_FILE2 ####################################################################### ======== 4) Notes ======== The following are some exceptions in the usage of QuickBMS, not real bugs but things that can't work (at least at the moment) due to the very flexible nature of the tool: - Log "123.txt" OFFSET SIZE creates the file 123 and not 123.txt, this happens because "123.txt" is considered a constant number due to the rule that anything starting with a number (or a '-') is handled as a constant number. this behaviour didn't happen with the previous versions of the tool because wasn't used the number optimization which saves tons of cpu cycles - set NAME string MEMORY_FILE log NAME 0 0 produces no phisical file because it's considered a MEMORY_FILE, it happens because the dumping function receives "MEMORY_FILE" as output file name. at the moment there is no fix anyway it's a very very rare event (never happened to find an archive containing a file with that name) and so not a priority - any Clog operation with a compressed or uncompressed size minor/equal than zero produces a file with a zero size, but this is not a problem of the tool because it's the perfectly logical behaviour in these situations - all the extracted files are loaded completely in memory before being dumped for various technical reasons so if the file to dump has a size of 800 megabytes this is the same size which will be allocated in memory, so it's good to have a good amount of free RAM when handling big archives - QuickBMS uses many third party code (mainly compression) and even if I have tried to adjust them a bit unfortunately many of these external functions were a disaster or missed any security requirement. that's the reason why the comtype scanning feature causes so many crashes with invalid data Other things to know or strange behaviours will be listed when I will figure (or remember) them. ####################################################################### ========== 5) Support ========== QuickBMS, like any other of my projects, is fully supported by me and is ever in continous development for adding new encryption and compression algorithms and sometimes for fixing bugs because I'm the first user of this tool and so I have a direct interest in it. So remember to contact me for any doubt or new idea via: - mail to aluigi@autistici.org - my forum on http://forum.aluigi.org - xentax forum on http://forum.xentax.com ####################################################################### ===================== 6) Additional credits ===================== QuickBMS uses also various public-domain code and code released under GPL/LGPL or other open source and free licences. Compression: - zlib, inflateback9 (for deflate64) and blast of Jean-loup Gailly and Mark Adler http://www.zlib.net - LZO of Markus F.X.J. Oberhumer http://www.oberhumer.com/opensource/lzo/ - LZSS, LZARI, LZHUF of Haruhiko Okumura - unlzx.c of Erik Meusel - LZMA and LZMA2 of Igor Pavlov http://www.7-zip.org - bzip2 of Julian Seward http://www.bzip.org - ascii85 partially derived from http://www.stillhq.com/svn/trunk/ascii85/decode85.c - libmspack of Stuart Caie http://www.cabextract.org.uk/libmspack/ - lzjb from http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/lzjb.c - iMatix SFL compression http://download.imatix.com/pub/ - UCL of Markus F.X.J. Oberhumer http://www.oberhumer.com/opensource/lzo/ - code from the uncompress utility of "The Regents of the University of California" - Dynamic Markov Compression implementation of Gordon V. Cormack http://plg1.cs.uwaterloo.ca/~ftp/dmc/dmc.c - 3 small rle algorithms from ScummVM https://scummvm.svn.sourceforge.net/svnroot/scummvm/scummvm/tags/release-1-0-0rc1/engines/sword1/screen.cpp - bpe of Philip Gage http://www.csse.monash.edu.au/cluster/RJK/Compress/bpd.c - QuickLZ of Lasse Mikkel Reinhold http://www.quicklz.com - Quake 3 Huffman code of Id Software http://www.idsoftware.com - mszh from the LossLess Codec Library - Doom Huffman code from the Doom/Hexen source code - aPLib of Jørgen Ibsen http://www.ibsensoftware.com/products_aPLib.html - LZF of Marc Alexander Lehmann http://home.schmorp.de/marc/liblzf.html - LZ77 of Arkadi Kagan http://compressions.sourceforge.net/about.html - LZRW1 of Ross Williams http://www.ross.net/compression/lzrw1.html - an Huffman implementation of Bill Demas on LDS - the FIN algorithm (useless and very close to LZSS) on LDS - LZAH/LZH12/13 of Dik T. Winter http://homepages.cwi.nl/~dik/english/ftp.html - GRZipII/libGRZip of Grebnov Ilya (only the win32 code is linked to it because it's composed by too much files and it's not a priority) - rle of Chilkat Software http://www.chilkatsoft.com/chilkatdx/ck_rle.htm#source - Quad of Ilia Muraviev http://quad.sourceforge.net - Balz of Ilia Muraviev http://balz.sourceforge.net - unshrink of Info-Zip http://www.info-zip.org/UnZip.html - PPMD algorithms of Dmitry Shkarin http://compression.ru/ds/ - BriefLZ of Jørgen Ibsen http://www.ibsensoftware.com/products_aPLib.html - PAQ6 of Matt Mahoney http://cs.fit.edu/~mmahoney/compression/paq.html#paq6 - shcodec of Simakov Alexander http://webcenter.ru/~xander/ - hstest of tom ehlert - SixPack of Philip G. Gage - ashford of Charles Ashford - JCALG1 of Jeremy Collake http://www.bitsum.com/jcalg1.htm - jam/unjam of W. Jiang - lzhlib of Haruhiko Okumura and Kerwin F. Medina for the adaptation of the code - Srank P M Fenwick http://www.cs.auckland.ac.nz/~peter-f/FTPfiles/srank.c - Zziplib/Zzlib of Damien Debin http://damiendebin.net/archives/zzip/download.php#zzlib - scpack of Philip Gage - rle3 and bpe2: http://musyozoku211.blog118.fc2.com/blog-entry-13.html http://blog-imgs-17.fc2.com/m/u/s/musyozoku211/bpe2.txt http://blog-imgs-17.fc2.com/m/u/s/musyozoku211/rle3.txt - Basic Compression Library of Marcus Geelnard http://bcl.comli.eu - SCZ of Carl Kindman http://scz-compress.sourceforge.net - szip of HDF Group http://www.hdfgroup.org/doc_resource/SZIP/ - sr3c of Kenneth Oksanen http://cessu.blogspot.com - Huffman library of Douglas Ryan Richardson http://huffman.sourceforge.net - SFastPacker of Aleksey Kuznetsov http://www.utilmind.com/delphi3.html - lz77wii of Hector Martin http://wiibrew.org/wiki/Wii.py Encryption: - all the algorithms provided by OpenSSL http://www.openssl.org - xtea from PolarSSL http://www.polarssl.org - some encryption algorithms from GnuPG and libgcrypt http://www.gnupg.org - ICE of Matthew Kwan http://www.darkside.com.au/ice/index.html - Rotor module from the Python source code Notes: - some of the original codes have been modified a bit to make them usable in QuickBMS for the memory2memory (aka in-memory) decompression and for other possible fixes or for reducing the amount of code (for example removing the compression routine leaving only the decompression one) - almost all the algorithms implemented here have been selected because have or could have been used or it has been claimed to have been used in real software/games or they are enough known to deserve their implementation in QuickBMS - if the files have been modified or don't have the original license informations (could happen only with small functions that didn't contain a license header in origin) please follow the provided links for more details - tell me if I have forgiven someone/something in this section, my memory and level of attention are quite ridiculous #######################################################################