One-dimensional optimisation, Victorian style
For many years, I taught an undergraduate course on nonlinear optimisation. It was a subject close to my heart, as it formed a significant part of my PhD thesis. Nonlinear optimisation is concerned with finding the location (parameter values) and objective function value of a function which is not a linear expression. School calculus teaches you to differentiate the function and find where the gradient is zero. All very well, until the function is only found by a computer evaluation of a series of expressions, or by experiment, or the solution of the derivative equation(s) is difficult/impossible. Then, the only way forward is to search.
At the time, the 1970s, there was a great deal of interest in the numerical analysis of algorithms to be efficient in terms of both the number of evaluations of the objective and the computer storage needed for the algorithm (How times have changed!). Many methods relied on a series of linear searches - I used to illustrate this with the aid of …
At the time, the 1970s, there was a great deal of interest in the numerical analysis of algorithms to be efficient in terms of both the number of evaluations of the objective and the computer storage needed for the algorithm (How times have changed!). Many methods relied on a series of linear searches - I used to illustrate this with the aid of …
