Reverse engineering McAfee Virus Scan - Part I: The Installer

August 17, 2023 – 6 min – 1162 words

With this article I want to summarize some details I found while analyzing McAfee Virus Scan software, historically speaking, it is one of the first commercial antivirus. Studying its details and operation might reveal some design choices that show how early antivirus worked. We are especially interested in understanding how malware detection occurred, as well as the particular features that the McAfee product offered.

Screenshot of VirusScan Console

Getting the binary

The McAfee Virus Scan product was one of the first antivirus products developed by the company’s namesake founder John McAfee and later became the product of the McAfee Corporation. Although the product was commercialized very early (we are talking about 1988 as the first year of release), it is still possible to find some copies online. To retrieve one of the early binaries, I went to web.archive.org which has a wide range of archives of materials including software, but also books, documents, movies and more.

The binary we are interested in is called “McAfee Virus Scan” and is the installer of the antivirus product, version released on October 10, 2002. Since we are interested in inspecting its contents, we download the binary locally and begin the software analysis. The downloaded binary VSHENU702D30.exe is a self-extracting archive called “InstallShield self-extracting archive.”

InstallShield is a platform for creating software installation and distribution programs. Developed by Flexera Software, it is used by companies to create installers for software products. Commonly speaking, an installer consists of two main sections. The first section is the set of information that the binary needs to extract the content (and thus actually install the software), while the second section is given by the content itself (the files that the program needs to install). In this case we are interested in extracting the contents of the McAfee suite from the program.

How is an InstallShield binary composed?

A PE (Portable Executable) binary created with InstallShield consists of several elements:

Examples of data structures we can find within the InstallShield binary:

typedef struct IS_HEADER {
    char signature[14];
    uint16_t num_files;
    uint32_t type; // "ISSetupStream" format uses this
    uint8_t unknown[26];
} IS_HEADER;

where:

Another important structure is the file attributes.

typedef struct IS_FILE_ATTRIBUTES {
	char file_name[_MAX_PATH]; 
	uint32_t encoded_flags;
	uint32_t x3;
	uint32_t file_len;
	uint8_t  unknown_2[8];
} IS_FILE_ATTRIBUTES;

where:

The contents of each file are compressed using the LZMA algorithm, Lempel Ziv Markov, an algorithm used for data compression. The LZMA algorithm uses encoding via a dictionary to store occurrences and references to a pattern of common strings. The idea is to have a dictionary representing <String, NumberOfOccurrences> pairs. A number of auxiliary data structures are used to store the occurrence offset. To learn more about the algorithm specification I recommend reading “LZMA specification”.

File extraction is done by this simple algorithm (derived via IDA Free ❤️ ):

GetTempFileNameA(lpPathName, PrefixString, 0, TempFileName);
v3 = (void *)sub_401756(TempFileName, 33025);
result = -1;
if ( v3 != NULL ){
	sub_40182F(v3, lpBuffer, nNumberOfBytesToWrite);
	sub_401851(v3);
	v5 = LZOpenFileA(TempFileName, &ReOpenBuf, 0);
	v6 = LZOpenFileA(lpFileName, &v9, 0x1001u);
	v7 = LZCopy(v5, v6);
	LZClose(v5);
	LZClose(v6);
	DeleteFileA(TempFileName);
	return v7;
}

The steps of the algorithm are as follows:

  1. We obtain the name of the file contained in the temporary folder
  2. We create a new file inside the temporary folder using TempFileName and the sub_401756 routine.
  3. We write the (compressed) contents of the file through the sub_40182F routine.
  4. We open the compressed file (v5) and the file that will be used to unpack the contents (v6)
  5. We extract the contents from buffer v5 to buffer v6
  6. We close the relevant handles
  7. We delete the temporary file (it contained the compressed file included in the binary).

You can, of course, extract the contents of the file simply by starting any debugger and setting several breakpoints on the functions of interest (in this case LZOpenFileA and LZCopy).

Since we are working on an installer created many years ago (almost 21!) the installer data structures may not be the same as those used in the latest version of InstallShield. To extract the contents of the binary, it is not necessary to do anything extraordinarily difficult. Simply run the 7z x name-binary command using the 7zip software: in this way we can derive the binaries that should be installed inside the C:\Programs (x86)\McAfee Virus Scan folder. The contents of the binaries are as follows:

[5]SummaryInformation'   Binary.EpocDetect.dll                                      Icon.VsMain.exe
'!ActionText'             Binary.Exclaim                                            '!IniFile'
'!AdminExecuteSequence'   Binary.McAfeeCA.dll                                       '!InstallExecuteSequence'
'!AdminUISequence'        Binary.McAfeeCA.Dll.03723396_538B_11D4_B355_00B0D04BB45E  '!InstallShield'
'!AdvtExecuteSequence'    Binary.McAfeeCA.Dll.E2288E6B_6CA7_11D4_B37B_00B0D04BB45E  '!InstallUISequence'
'!AppSearch'              Binary.NAILogo_Top.bmp                                    '!LaunchCondition'
'!Binary'                 Binary.SetAllUsers.dll                                    '!Media'
 Binary.ACCEPTLICENSE     Binary.side16                                             '!ModuleComponents'
 Binary.AxdistEXE         Binary.VSCLogo_Left.bmp                                   '!ModuleDependency'
 Binary.Binary20          Binary.VSCTop_NoLogo.bmp                                  '!ModuleSignature'
 Binary.Binary21         '!CheckBox'                                                '!MoveFile'
 Binary.Binary22         '!Class'                                                   '!ProgId'
 Binary.Binary23         '!_Columns'                                                '!Property'
 Binary.Binary24         '!Component'                                               '!RadioButton'
 Binary.Binary25         '!Condition'                                               '!Registry'
 Binary.Binary26         '!Control'                                                 '!RegLocator'
 Binary.Binary27         '!ControlCondition'                                        '!RemoveFile'
 Binary.Binary28         '!ControlEvent'                                            '!RemoveRegistry'
 Binary.Binary29         '!CreateFolder'                                            '!SelfReg'
 Binary.Binary30         '!CustomAction'                                            '!ServiceControl'
 Binary.Binary31          Data.Cab                                                  '!ServiceInstall'
 Binary.Binary32         '!Dialog'                                                  '!Shortcut'
 Binary.Binary33         '!Directory'                                               '!Signature'
 Binary.Binary34         '!DrLocator'                                               '!_StringData'
 Binary.Callca_vsc       '!Error'                                                   '!_StringPool'
 Binary.Callca_vsc_nt    '!EventMapping'                                            '!_Tables'
 Binary.Callimmediate    '!Feature'                                                 '!TextStyle'
 Binary.Callmsi_inst     '!FeatureComponents'                                       '!TypeLib'
 Binary.Callritual       '!File'                                                    '!UIText'
 Binary.Callwiseapi      '!Icon'                                                    '!Upgrade'
 Binary.DELETE.EXE        Icon._B291A24037AB_11D7_BAF5_00B0D0C6893B.exe             '!_Validation'
 Binary.Dll_.ini          Icon._C050255A3944_11D7_99DE_00605B102CFB.exe              VSC.msi
 Binary.DLLWRAP.DLL       Icon.Icon63CB76201.exe                                     
 Binary.DontInstall.ico   Icon.Icon63CB7620.exe

As you can see, 7zip has extracted the entire contents of the binary file, including some secondary information that specifies certain settings to the installer. These settings are encoded in a series of files that we are not interested in (e.g. '!Dialog or !File). These files are denoted by the initial of the file name '!: we can then proceed to move/delete these files. To delete them we can use the rm \!* command. Instead, all that can be obtained are the files representing the McAfee Virus Scan program.

Next we can filter the binaries by type: we do not want our analysis to focus on files that allow graphics (icons, images). We can then run file_name to better understand the nature of the file. Executable files, libraries, and native code on Windows usually are contained within the PE file format and are labeled much like:

PE32 executable (DLL|GUI) Intel 80386, for MS Windows

In the next article we will focus on first reverse engineering McAfee’s internal file and directory scanning engine ``F324_SCAN.EXE`. If you have any comments, curiosities or criticisms, you can email me.