The solution of many problems requires us to compute derivatives. **Complex step differentiation** is a method of computing the first derivative of a real function, which circumvents the problem of roundoff error found with typical finite difference approximations.

Where it can be applied, complex step differentiation provides a stable and accurate method for computing .

**Finite differences and derivatives **

We define the derivative of a function as the limit

For finite , we can use Taylor’s theorem to estimate the accuracy:

By making small, we can get a precise estimate of the derivative. But there is a problem: the limit on the precision of numbers in the computer means that, as becomes smaller, the difference looses precision and, ultimately, becomes indistinguishable from , giving a zero value for the approximation.

*We may allow the step to be complex:* taking a pure imaginary increment, we have

Taking the imaginary part and dividing by gives us

This yields a second order accuracy. Moreover, since no difference between function values is required, it is free from the roundoff errors that contaminate (1) for small .

** A Numerical Example **

In the figure, we show the logarithm of the error of approximations to the derivative, at , of . The dashed red line is for the first-order finite difference approximation (1). For the error decreases linearly with consistent with the first-order accuracy of (1). For the error actually gets larger as decreases. This is due to roundoff errors in computing the difference .

The solid blue line in the figure is for the second-order complex step approximation (2). The error has a slope of 2, consistent with the second-order accuracy of (2). It decreases rapidly as the step size is decreased. Since there is no difference calculation in (2), roundoff error is not a problem. For , the error is obliterated by rounding and the value of is exact. Rounding actually leads to an improvement in this case! For double precision floating point computing, with about 16 digits accuracy, we may choose , so that error is at the level of machine precision.

** Cauchy-Riemann Equations **

The magical formula (2) can be understood in terms of the theory of holomorphic functions. We can regard as the real part of a function of a complex variable :

We will be concerned with functions that are real for real :

If is holomorphic, the real and imaginary parts are related by the Cauchy-Riemann equations

Since is real-valued on the real line, and . Therefore, on the real line, and also so that for real . Thus,

We approximate by taking a finite step , noting that , whence

which is consistent with (2).

** Conclusion**

In cases where we can easily calculate but have no simple expression for , the complex step approximation gives us an accurate and numerically robust means of computing the derivative. It has been applied with great effect in a wide variety of circumstances.

** Sources **

Higham, N. J., 2018: Differentiation With(out) a Difference. *SIAM News*, **51** (5), p. 2. URL.

Squire, William and George Trapp, 1998: Using complex variables to estimate derivatives of real functions, *SIAM Review* **40**, 110–112.

Wikipedia article: *Numerical differentiation.*

* * * * *

**That’s Maths II: A Ton of Wonders**

by Peter Lynch now available.

Full details and links to suppliers at

http://logicpress.ie/2020-3/

>> Review in *The Irish Times <<*

* * * * *