1.   Optimization

Optimization is central to any problem involving decision making, whether in engineering or in economics. The task of decision making entails choosing between various alternatives. This choice is governed by our desire to make the “Best” decision. The measure of goodness of the alternatives is described by an objectives function or performance index. Optimization theory and methods deal with selecting the best alternative in the sense of the given objective function.

The area of optimization has received enormous attention in recent years, primarily because of the rapid progress in computer technology, including the development and availability of user-friendly software, high-speed and parallel processors, and artificial neural networks. A clear example of this phenomenon is the wide accessibility of optimization software tools such as the Optimization Toolbox of MATLAB and the many other commercial software packages.

The field of optimization is still a very active research area. In recent years, various new approaches to optimization have been proposed. In this lecture, we have tried to reflect at least some of the flavor of recent activity in the area. For example, a discussion of generic algorithms, a topic of increasing importance in the study of complex adaptive systems.

Additionally, the effort required to make a piece of software completely optimal — incapable of any further improvement — is almost always more than is reasonable for the benefits that would be accrued; so the process of optimization may be halted before a completely optimal solution has been reached. Fortunately, it is often the case that the greatest improvements come early in the process.

2.  Levels of optimization

Optimization can occur at a number of “levels”:

At the highest level, the design may be optimized to make best use of the available resources. The implementation of this design will benefit from a good choice of efficient algorithms and the implementation of these algorithms will benefit from writing good quality code. The architectural design of a system overwhelmingly affects its performance. The choice of algorithm affects efficiency more than any other item of the design and, since the choice of algorithm usually is the first thing that must be decided, arguments against early or “premature optimization” may be hard to justify.

In some cases, however, optimization relies on using more elaborate algorithms, making use of “special cases” and special “tricks” and performing complex trade-offs. A “fully optimized” program might be more difficult to comprehend and hence may contain more faults than un-optimized versions.

3.       Time taken for optimization

Sometimes, the time taken to undertake optimization in itself may be an issue. Optimizing existing code usually does not add new features, and worse, it might add new bugs in previously working code (as any change might). Because manually optimized code might sometimes have less “readability” than un-optimized code, optimization might impact maintainability of it as well. Optimization comes at a price and it is important to be sure that the investment is worthwhile.

4.  Mathematical Optimization

An optimization problem consists of maximizing or minimizing a real function by systematically choosing input values from within an allowed set and computing the value of the function. The generalization of optimization theory and techniques to other formulations comprises a large area of applied mathematics. More generally, optimization includes finding “best available” values of some objective function given a defined domain, including a variety of different types of objective functions and different types of domains.

5.  Computational optimization techniques

To solve problems, researchers may use algorithms (Algorithms can be expressed in many kinds of notation, including natural languages, pseudocode, flowcharts, programming languages or control tables (processed by interpreters) that terminate in a finite number of steps, or iterative methods (If an equation can be put into the form f(x) = x, and a solution x is an attractive fixed point of the function f, then one may begin with a point x1 in the basin of attraction of x, and let xn+1 = f(xn) for n ? 1, and the sequence {xn}n ? 1 will converge to the solution x) that converge to a solution (on some specified class of problems), or heuristics (heuristic designates a computational method that optimizes a problem by iteratively trying to improve a candidate solution with regard to a given measure of quality) that may provide approximate solutions to some problems.