Optimal Design, Lagrangian and Linear Model Theories: a Fusion
Ben Torsney (University of Glasgow, UK)
We consider the problem of optimizing a criterion of several variables, subject to them satisfying several linear equality constraints. Lagrangian Theory requires that, at an optimum, all partial derivatives be exactly linear in a set of Lagrange Multipliers. It seems we can argue that the partial derivatives, viewed as response variables, must exactly satisfy a Linear Model with the Lagrange Multipliers as parameters. This then is a model without errors implying a fitted model with zero residuals. A special case is an approximate optimal design problem with the single “summation to one” constraint. In this case one formula for residuals defines “vertex directional derivatives”. In our general problem any formula for residuals appears to play the role of directional derivatives.Further, if all variables are nonnegative, we can extend the multiplicative algorithm formulated for finding optimal design weights. This comprises two steps: a multiplicative step, under which we multiply each variable by a positive increasing function of its (vertex directional) derivative; and a scaling step, under which we scale these products to meet the “summation to one” constraint.The multiplicative step naturally extends to our more general problem and we believe that we have discovered a generalisation of the scaling step. Numerical results will be reported.