International Journal "Information Theories & Applications" Vol.14 / 2007
265
[Кириченко, Крак, Полищук,2004] Н.Ф.Кириченко, Ю.В. Крак.А.А. Полищук Псевдообратные и проекционные матрицы
в задачах синтеза функциональных преобразователей.// Кибернетика и системный анализ –2004.–№3.
[Donchenko, Kirichenko,2005.] V.S Donchenko, N.F. Kirichenko Generalized Inverse in Control with the constraints.//
Cybernetics and System Analysis.– v.39 (6) November-December– 2003.–p.854-861(USA).
[Линник,1962] Ю.В.Линник Метод наименьших квадратов и основы математико-статистической теории обработки
наблюдений. Изд. 2-е. – М.:Физматгиз.– 1962.– 349 c.
[Вапник,1979] В.Н. Вапник. Восстановление зависимостей по эмпирическим данным. – М.: Наука. – 1979. – 447 с.
[Ивахненко,1969] А.Г. Ивахненко Самоорганизующиеся системы распознавания и автоматического управления. – К.:
Техника. – 1969. – 395 с.
Authors’ Information
Volodymyr Donchenko – Kyiv National Taras Shevchenko University (Ukraine), Professor,
e-mail: voldon@unicyb.kiev.ua
Mykola Kirichenko – Institute of the Cybernetics, National Academy of Sciences (Ukraine), Professor
Yuriy Krivonos – Institute of the Cybernetics, National Academy of Sciences (Ukraine), Member Correspondent
of the National Academy of Sciences.
NEURAL NETWORK BASED OPTIMAL CONTROL WITH CONSTRAINTS
Daniela Toshkova, Georgi Toshkov, Todorka Kovacheva
Abstract: In the present paper the problems of the optimal control of systems when constraints are imposed on
the control is considered. The optimality conditions are given in the form of Pontryagin’s maximum principle. The
obtained piecewise linear function is approximated by using feedforward neural network. A numerical example is
given.
Keywords: optimal control, constraints, neural networks
ACM Classification Keywords: I.2.8 Problem Solving, Control Methods, and Search
Introduction
The optimal control problem with constraints is usually solved by applying Pontryagin’s maximum principle. As it
is known the optimal control solution can be obtained computationally. Even in the cases when it is possible an
analytical expression for optimal control function to be found, the form of this function is quite complex. Because
of that reason the possibilities of using neural networks for solving the optimal control problem are studied in the
present paper.
The ability of neural networks to approximate nonlinear function is central to their use in control. Therefore it can
be effectively utilized to represent the regulator nonlinearity. Other advantages are their robustness, parallel
architecture.
Lately, different approaches are proposed in the literature treating the problem of constrained optimal control for
using neural networks. In [Ahmed 1998] a multilayered feedforward neural network is employed as a controller.
The training of the neural network is realized on the basis of the so called concept of Block Partial Derivatives. In
[Lewis 2002] a closed form solution of the optimal control problem with constraints is obtained solving the
associate Hamilton-Jacobi-Bellman (HJB) equation. The solution of the value function of HJB equation is
approximated by using neural networks.
International Journal "Information Theories & Applications" Vol.14 / 2007
266
In the present paper the problem of finding the optimal control with constraints is considered. A numerical
example is given.
Problem Statement
The control system, described by following differential equations is considered:
∑=
+ =
n
1 j
i i ij
i u b x a
dt
dx
(i = 1, 2, …, n) (1)
where xj are phase coordinates of the system, function u describes the control action and aij are constant
coefficients. The admissible control u belonging to the set U of piecewise linear functions is constrained by the
condition
1 ) t ( u ≤ (2)
Following problem for finding the optimal control is formulated. To find such a control function u(x1,…, xn) for the
system (1) among all the admissible controls that the corresponding trajectory (x1(t),…, x2(t)) of the system (1)
starting from any initial state (x1(0),…, x2(0)) to tend to zero at t → ∞ and the performance index
∫ ∑
∞
=
⎟ ⎟⎠
⎞
⎜ ⎜⎝
⎛
+ =
0
n
1 i
2 2
i i dt ru x q J (3)
to be converging and to take its smallest possible value. The coefficient qi and r are positive weight constants.
Optimality Conditions
The notation is introduced [Pontryagin 1983]:
( ) ∑=
+ =
n
1 j
2 2j
j n 1 0 ru x q u , x ..., , x f (4)
( ) ∑=
+ =
n
1 j
i
2j
ij n 1 i u b x a u , x ..., , x f (i = 1,…,n) (5)
One more variable θ0 is added to the state variables (x1,…, xn) of the system (1) [Chjan 1961]. It is a solution of
the following equation
( ) u , x ..., , x f
dt
dx
n 1 0
0 = (6)
and initial condition x0(0) = 0. Then the quantity J according to (9) becomes equal to the boundary of x(t) when
t → ∞. The system of differential equation, which are adjoint to the system (7) is composed with new variables
{ }n 1 0 ..., , , Ψ Ψ Ψ = Ψ :
∑=
α
α
α = Ψ
∂
∂
− =
Ψ n
0 0
0 0
x
f
dt
d
(7)
∑ ∑
=α =
α
α Ψ − θ − = Ψ
∂
∂
− =
Ψ n
0
n
1 j
j ij i i
0
i a q 2
x
f
dt
d
(i = 1,…,n) (8)
After that the Hamilton function is composed:
( ) ( ) ∑ ∑ ∑ ∑ ∑
= = = = α
α α
α
= α
α ⎟
⎟
⎠
⎞
⎜ ⎜
⎝
⎛
+ Ψ + ⎟
⎟⎠
⎞
⎜ ⎜⎝
⎛
+ Ψ = Ψ = Ψ = Ψ θ
n
1 i
n
1 j
i j ij i
n
1 i
2 2
i i 0
n
0
n 1
n
0
u b x a ru x q u , x ,..., x f
dt
dx u , , H (9)
In the right-hand side of Eq. (9) the quantity u is contained in the expression
International Journal "Information Theories & Applications" Vol.14 / 2007
267
∑=
Ψ + Ψ =
n
1 i
i i
2
0 1 ) t ( b ) t ( u ) t ( u ) t ( r H (10)
Because of that the condition for maximum of H coincide with the condition
⎪⎭
⎪⎬ ⎫
⎥⎦
⎤
⎢⎣
⎡
Ψ
Ψ
−
⎪⎩
⎪⎨ ⎧
⎥⎦
⎤
⎢⎣
⎡
Ψ
Ψ
+ Ψ =
= ⎥⎦
⎤
⎢⎣
⎡
Ψ + Ψ =
∑ ∑
∑
= = ≤
= ≤ ≤
2
1 0
2
1 0
0
1 | |
1
2
0 1 | | 1 1 | |
) (
4
1 ) (
2
1 ) ( ) ( max
) ( ) ( ) ( ) ( max max
n
i
i i
n
i
i i
u
n
i
i i u u
t b
r
t b
r
t u t r
t b t u t u t r H
(11)
Having in mind condition (7) the quantity Ψ0
is a constant. As its value can be any negative number it is set to
Ψ0 = -1.
After placing this value in Eq. (11) the maximum of the expression in the square brackets will be reached when
the first negative addend becomes zero if it is possible or takes its minimal absolute value. The expression
2 n
1 i
i i ) t ( b
r 2
1 ) t ( u ⎥⎦
⎤
⎢⎣
⎡
Ψ − ∑=
(12)
will take its minimal absolute value if on condition |u| ≤ 1 a value of the following kind is chosen for u
⎪ ⎪ ⎪ ⎪ ⎪
⎩
⎪⎪ ⎪ ⎪ ⎪
⎨
⎧
− ≤ Ψ −
≥ Ψ
≤ Ψ Ψ
=
∑
∑
∑ ∑
=
=
= =
1 b
r 2
1 at 1
1 b
r 2
1 at 1
1 b
r 2
1 at b
r 2
1
) t ( u
n
1 i
i i
n
1 i
i i
n
1 i
i i
n
1 i
i i
(13)
The values of ψс(t) can be determined if the adjoint equations (7), (8) are solved. This leads to the requirement
the initial values of ψ0(0) to be found beforehand.
First u(t) is assumed not to reach its boundary values. Then after placing the upper expression from (13) instead
of u(t) in Eqs. (1), (7) и (8) one obtains
∑
∑ ∑
=
= =
Ψ − =
Ψ
= Ψ + =
n
1 j
j ji i i
i
n
1 j
n
1 j
j j
i
j ij
i
a x q 2
dt
d
) n ,..., 1 i ( b
r 2
b x a
dt
dx
(14)
This system of equations has to be solved with the initial conditions x1(0),…, xn(0) as well as with the final
(boundary) conditions
( ) () () 0 t x lim ... t x lim t x lim n t 2 t 1 t
= = = =
∞ → ∞ → ∞ → (15)
It is necessary the appropriate initial conditions ψ1(0), … , ψn(0) to be selected in such a way that the initial and
the final conditions for x1(t),…, xn(t) to be satisfied.
The relationship between xi(0) and ψi(0) has the following form [4]:
∑= χ
=
Ψ
n
1 j
i ij i ) 0 ( x ) 0 ( (i =1, …,n) (16)
These relationships have to be kept in any time, for which one can always assume to be the initial one. Therefore
the optimal control u within the boundaries is determined and it has the following form:
International Journal "Information Theories & Applications" Vol.14 / 2007
268
∑=
=
n
1 i
i i x k
r 2
1 u (17)
where ∑=
χ =
n
1 j
ji j i b k
The expression (17) holds only in the cases when the absolute value of the sum ( ) n n 1 1 x k ... x k
r 2
1 + + is not
greater than one. When ( ) 1 x k ... x k
r 2
1
n n 1 1 > + + the optimal control passes on the boundary i.e. |u| = 1, if the right
hand boundary conditions are satisfied i.e. the solution of the system (1), which became nonlinear in connection
to the nonlinear relationship between u and x1,…, xn, tends to zero. In other words the solution of the system has
to be asymptotically stable. Thus the optimal control is defined by the expression
⎪ ⎪ ⎪ ⎪
⎩
⎪ ⎪ ⎪ ⎪
⎨
⎧
− ≤ −
≥
≤
=
∑
∑
∑ ∑
=
=
= =
1 x k
r 2
1 at 1
1 x k
r 2
1 at 1
1 x k
r 2
1 at x k
r 2
1
) t ( u
n
1 i
i i
n
1 i
i i
n
1 i
i i
n
1 i
i i
(18)
Structure and Training of the Neural Network
For the control function realization a feed forward neural network with one hidden layer is used. Thus the
necessity of solving a large number of equations for determining the coefficients ki drops off.
The neural network consists of three layers – an input, output and hidden one. The input and hidden layers have
five neurons and the output layer – one. The activation function of the output neuron is piecewise linear. The
neural network output is
( )
( ) ( )
( ) ⎪⎩
⎪⎨
⎧
≤ ν ϕ −
≤ ν ϕ ν ϕ
≥ ν ϕ +
=
1 1
1
1 1
y (19)
where ν = wTz. The neural network input is denoted z and w is the neural network weight. The neural network
output represents the control u, x – the state vector and weights are the coefficient k.
The neural network is trained according to the back-propagation algorithm. Let the training sample N
1 n d(n)} {z(n), =
be given where z(n) are the system states and d(n) is the corresponding control, which are known preliminarily.
The neural network is trained according to the back-propagation algorithm [Haykin 1999].
Simulation Results
In order to verify the suggested approach for solving the optimal control problem following system is considered:
u x 2 x
dt
dx
x
dt
dx
2 1
2
2
1
+ − − =
=
International Journal "Information Theories & Applications" Vol.14 / 2007
269
and the control is constrained by
|u| ≤ 1
The performance index to be minimized is of the form:
( ) ( ) ( )dt t u t x t x [ 2 22
0
2
1 + + ∫
∞
The problem is solved by using Pontryagins principle and neural networks. The results, which are obtained by
both approaches, are compared. In Fig. 1 the optimal control, obtained by using neural networks is shown. Fig. 2
depicts the corresponding states trajectory. In Fig. 3 and Fig. 4 the optimal control, obtained by applying the
maximum principle and the corresponding trajectory are given respectively. By 1 and 2 are denoted x1 and x2
respectively.
0 2 4 6 8 10 12 14 16
1.5
1
0.5
0
-0.5
-1
-1.5
time,s
control
Fig. 1. Optimal control, obtained by using
the suggested neural network based approach
1.5
1
0.5
0
-0.5
-1
-1.5
0 2 4 6 8 10 12 14 16
time,s
control
Fig. 2. Optimal control, obtained by applying
the maximum principle
1
2
0 2 4 6 8 10 12 14 16
-15
-10
-5
0
5
10
15
time,s
state
Fig.3 Optimal trajectory (neural network based approach)
2
1
0 2 4 6 8 10 12 14 16
time,s
15
10
5
0
-5
-10
-15
state
Fig.4 Optimal trajectory (Pontryagin’s maximum principle)
International Journal "Information Theories & Applications" Vol.14 / 2007
270
Conclusion
In the present paper an approach for optimal constrained control based on using of neural networks is suggested.
On the basis of the simulation experiments one can say that the proposed approach for optimal control is
accurate enough for the engineering practice. The suggested approach can be applied for optimal control in real
time, where the control is constrained.
Bibliography
[Ahmed 1998] M.S. Ahmed and M.A. Al-Djani. Neural regulator design. Neural Networks, Vol. 11, pp. 1695-1709, 1998
[Chjan 1961] Zh.-V.Chjan. A problem of optimal system synthesis through the maximum principle. Automatization and
telemechanics, Vol. XXII, No.1, 1961, pp. 8-12
[Haykin 1999] S. Haykin. Neural Networks: A Comprehensive Foundation., 1999, 2nd ed, Macmillan College Publishing
Company, New York
[Lewis 2002] F.L. Lewis and M. Abu-Khalaf. A Hamilton-Jacobi setup for constrained neural network control. Proceedings of
the 2003 IEEE International Symposium on Intelligent Control Houston, Texas *October 5-8. 2003
[Pontryagin 1983] L.S. Pontryagin, V.G. Boltyanskiy, R.V. Gamkrelidze and E.F.Mishtenko. Methematical theory of the
optimal processes. Nauka, Moskow, 1983, in Russian
Authors’ Information
Georgi Toshkov –e-mail: g_toshkov2006@abv.bg
Daniela Toshkova – е-mail: daniela_toshkova@abv.bg
Technical University of Varna, 1, Studentska Str, Varna, 9010, Bulgaria
Todorka Kovacheva – Economical University of Varna, Kniaz Boris Str, e-mail: todorka_kovacheva@yahoo.com
LINEAR CLASSIFIERS BASED ON BINARY DISTRIBUTED REPRESENTATIONS
Dmitri Rachkovskij
Abstract: Binary distributed representations of vector data (numerical, textual, visual) are investigated in
classification tasks. A comparative analysis of results for various methods and tasks using artificial and real-world
data is given.
Keywords: Distributed representations, binary representations, coarse coding, classifiers, perceptron, SVM, RSC
ACM Classification Keywords: C.1.3 Other Architecture Styles - Neural nets, I.2.6 Learning - Connectionism
and neural nets, Induction, Parameter learning
Introduction
Classification tasks consist in assigning input data samples to one or more classes from a predefined set [1].
Classification in the inductive approach is realized on the basis of a training set containing labeled data samples.
Usually, input data samples are represented as numeric vectors. Vector elements are real numbers (e.g., some
measurements of object characteristics or their function) or binary values (indicators of some features in the input
data).
This vector information often isn’t explicitly relevant to the classification, therefore some kind of transformation is
necessary. We have developed methods for transformation of input information of various kinds (such as
numerical [2], textual [3], visual [4]) to binary distributed representations. These representations can then be
 davido.extraxim@gmail.com