P.h.D. Kryuchin O.V.

Tambov State University named after G.R. Derzhavin

Artificial neural networks, computer clusters and the universal simulator

Now artificial neural networks (ANN) are used in different branches of science. Especially it occurs in such science as psychology, sociology and economics. The reason of it is the hardship of the objects simulation in this sphere because these objects cannot be defined by this province rules. But the ANN usage needs the long time expense because neural network models have the large structure and many parameters. One of this problem solutions is the parallel algorithms and the computer clusters usage [1-2].

The aim of this work is to analyse products of ANN sphere, to develop parallel algorithms of ANN training, and to develop the information system with parallel algorithms of ANN.

Now there are not real neural networks simulator for computer clusters. There are simulators for PC (JavaNNS, NeuroShell, NNC, Mathlab neural network tools and etc.) but they can't be executed on computer clusters. There are tools for computer clusters (T-System and etc.) but they don't consider specific of the neural network simulation. And there are hardware solutions (Nimfa and etc.). So there are not products which synthesizes the artificial neural network technology and the parallel algorithms technology [1, 3].

The main task of the neural network training is the target function (1) minimization. It consists of three levels. These are the structure training level, the level of the neurons activation functions training and the level of the weight coefficients training. The pic. 1 shows the general algorithm of the artificial neural network training.

, (1)

where is the input pattern row, is the output pattern row, is the ANN output value calculation function, is the weight coefficients vector, is the activation functions vector, is pattern row number and is the ANN output neurons number.

Pic. 1. The scheme of training of artificial neural network.

It is possibly to develop the parallel algorithm of the training on each level. And it is possible to develop the parallel training on the level of the target function value calculation.

If serial the training algorithm elapses hours then ideal parallel algorithm time expense is calculated by formula

, (2)

where is the processors number. But it is impossible because the synergetic effect is able in particular occurrences only. So real time expense of the parallel training is calculated by formula

. (3)

Here is the time expense for prepare and sending data.

So the efficiency coefficient may be calculated by formula

, (4)

where , are time expense of serial and parallel algorithms and is the processors number.

The processors count increase involves to the training speedup. But after several moment time expenses begin to rise. The reason of it is the large time expenses for transfer data from one processors to other. So there is the optimal count of processors for each task. This count dependences of calculation waiting expense (blue series in pic. 2) and expense of computer cluster using (black series in pic. 4). The red series in pic. 2 shows total expense.

Pic. 2. Dependence of expense from processors count.

Pic. 3. Information system architecture.

Described algorithms were developed in the information system. This system consists of three components group (pic. 3). The first group is neural network simulators. They are located on calculates nodes of the computer cluster. The second group consists of the simulators server and logging component. They are located on computer cluster master node. And third group is the clients program. They are located on the terminal computer.

So this paper brings together the technology of artificial neural networks and the technology of parallel algorithms. Aning information system with parallel algorithms of the ANN training was developed.

Literature

1. Крючин О.В., Арзамасцев А.А, Королев А.Н., Горбачев С.И., Семенов Н.О. Универсальный симулятор, базирующийся на технологии искусственных нейронных сетей, способный работать на параллельных машинах. // Вестн. Тамб. ун-та. Сер. Естеств. и техн. науки. – Тамбов, 2008, Т. 13. Вып. 5. С. 372 – 375.

2. Kryuchin O.V., Arzamastsev A.A., Troitzsch K.G. A universal simulator based on artificial neural networks for computer clusters [Электронный ресурс] — Электрон. дан. // Arbeitsberichte aus dem Fachbereich Informatik Nr. 2/2011. Koblenz. 2011. 13 p. — http://www.uni-koblenz.de/~fb4reports/2011/2011_02_Arbeitsberichte.pdf

3. Крючин О.В., Козадаев А.С., Аразамасцев А.А. Обзор нейросимуляторов для персональных компьютеров и кластерных систем // Материалы XVII общероссийской научной конференции «Державинские чтения» // Вестник Тамбовского Университета, Серия: Естественные и технические науки, - Т. 17, Вып. 1, 2012 - С. 168-172.