PhD(Tech.), associate professor Chastikova V.A., Ostapov D.S.

Kuban State Technological University, Russia

ANALYSIS AND RISK MINIMIZATION OF SOFTWARE HACKING

At present many users prefer to download illegal program from the Internet than to buy legal program in the shop. This problem is very typical in the Russian Federation.

Many developers protect their programmers by packing or encrypting and adding some keys. Keys are located in Exe-file or other file. But these actions cannot provide protection of high quality, because it is very easy to view file’s byte code with Hex editor. By this reason developers should protect their programs by other algorithm. Some developers protect their programs by using hardware components. Sentinel HASP is the protection system which protects computer programs using popular protection algorithms against disassemblers and debuggers, using local keys and hardware key-identifiers. But hardware key-identifiers are very expensive and it is very uncomfortable to use programs, which were protected by them.

Hackers use many programs for their criminal actions. These programs are debuggers, disassemblers, decompilers, Hex editors, unpackers, dumpers, resource editors, spy programs and monitors.

Comprehensive ring system of program protection against illegal use was developed. This system consists of 4 Rings of protection: Ring 0, Ring 1, Ring 2 and Ring 3. It is multilevel and comprehensive system.

Ring 0 is the main part of this system. This ring checks up on how many users use one program. This Ring detects illegal users. The main purpose of this ring is control of unique computer parameters. When a user connects to the Internet, Ring 0 will write serial numbers of user’s devices and user’s IP address to the database. Then a number of checks are carried out to find illegal users. If parameters in this session are not the same as the parameters in the previous session the program will write count of changes in database. If changes happen very frequently, Ring 0 will block this program and show a message [2]. This message contains information that the program is illegal and user should buy legal program and call technical support.

The next ring of protection is Ring 1. Ring 1 protects against debuggers. There are two ways to realize Ring 1.

The first way consists of the use of Trap Flag. Processors x86 have special flag TF. If TF equal to 1, special interrupt is generated after each directive. This interrupt is handled by debugger. Trap Flag is the part of the flag register. This flag can be saved in the stack by commands pushf/pushfd. This algorithm can protect program against unskilled hackers. But professional hacker will detect this set of commands and will disarm it by using breakpoint after this set of commands. Thus, if developer wants to protect program against professional hackers, he should modify this code by using SEH-handler. At first developer should load “1” into Trap Flag and call SEH-handler. All controls should be located inside SEH-handler. If program is run by debugger, SEH-handler will not be called [2].

The second way of using Ring 1 consists in calculation of cycle count. When program is run by debugger, processor will execute more cycles than when program is not run by debugger. After carrying out these actions register EAX will contain information about cycle count executed by processor between rdtsc directives. These two ways of debugger-protection can be used one by one or together [2].

Ring 2 protects program against disassemblers. At this moment there are many programs which can provide protection against dissemblers. ExeCryptor is one of the most popular programs which can provide this protection.

If a hacker opens program in disassembler without using protection, disassembler will show the whole code. But if a hacker opens protected program in disassembler, disassembler will show a code without any logical sense.

The last ring is Ring 3. This ring should check program integrity. If any byte has been changed, Ring 3will detect it.

The whole code of protected file should be written in two arrays: double_array_sum and double_mas_xor. These arrays must contain control sums by operations “+” and “xor”. Two operations are chosen because if only one operation is used, Ring 3 will not able detect change of bytes. For example if there are two bytes of 90h and 70h and a hacker changes them into 8Eh and 72h, control sum by operation “+” before and after changes will be equal to 100h. But control sum by operation “xor” before and after changes will be equal to 0E0h and 0Fch accordingly. Thus, in this example control sum by operation “+” cannot detect all changes, but control sum by operation “xor” can detect any changes.

The result of the sum of 16777216 (224) bytes should be written in each cell of the array. In this algorithm only 24 are used because 28 * 224 is maximum result of the sum by operation “+” and it should not be more than 232. If Ring 3 detects any changes in the bytes, it will block protected program and show message with information that program is illegal and user should buy legal program and call technical support.

But Ring 3 code should not be located in one place. There are files 1.exe, 2.exe. 3.exe, … M.exe. Ring 3 code should be located these files. Ring 3 code should be rewritten every time set up by developer. This code should move between PE-files. To do this there should be free space in these files. This space in PE-files consists of NOP (no operation) commands. When Ring 3 code is re- written in any file, it should be encoded by algorithm AES-128. SetFileTime function call is the last step in the work of Ring 3 [1].

The developed algorithm will help to minimize the possibility of illegal use of computer programs.

References:

1 Joan Daemen, Vincent Rigmen. AES Proposal: Rijndael, 1999. – 47p.

2 Chastikova V.A. Comprehensive Ring System Of Program Protection Against Using By Illegal Users / Chastikova V.A., Ostapov D.S // Scientist magazine KubSAU [Online resource]. Krasnodar, KubSAU, 2012, - № 82(08).