Adaptive Optimization in Stochastic Systems via the Variational Technique

This paper deals with the stochastic adaptive linear quadratic optimal control problems which have been an active area of research for many years. It has been known that these problems could be treated by dynamic programming. However, it has been conceded that explicit solution of the dynamic programming equations for these problems is generally not possible and that numerical solution of these equations is a difficult computational procedure. This has led to many approximation techniques. In the paper, a variational approach is used to obtain optimality conditions for the stochastic linear quadratic adaptive control problems. These conditions lead to an algorithm for computing optimal control laws which differs from the dynamic programming algorithm. If the unknown parameters enter into the state equation additively, and the prior distribution of the unknown parameters is normal, the algorithm can be carried out in closed form. The examples are given to illustrate the proposed technique.