Numerical Methods for Engineers

The Engineering Problem

Most engineering equations cannot be solved analytically. The Navier–Stokes equations, the heat equation, the equations of non-linear structural mechanics — these require numerical approximation. Understanding the nature of that approximation is essential, because a numerical result is not an exact answer. It is an approximation whose quality depends on the method, the discretisation and the problem itself.

Analytical vs Numerical Solutions

An analytical solution is an exact mathematical expression — a closed-form formula that satisfies the equation exactly. A numerical solution is an approximation computed at discrete points. The difference between them is the error introduced by the numerical method. Understanding, controlling and verifying this error is the core of computational engineering.

Discretisation and Truncation Error

Numerical methods replace continuous problems with discrete approximations. A derivative becomes a finite difference. An integral becomes a sum. A continuous domain becomes a mesh. Each of these replacements introduces truncation error — the difference between the exact mathematical operation and its numerical approximation. Truncation error generally decreases as the discretisation is refined (smaller step, finer mesh), but it never reaches zero.

Every numerical method introduces approximation — the engineering question is whether that approximation is controlled.

Numerical Differentiation

The finite difference approximation replaces a derivative with a ratio of differences. The forward difference is the simplest approximation. Its truncation error is proportional to the step size h, but for very small h, round-off error from finite-precision arithmetic grows. There is an optimal range of h where the total error (truncation plus round-off) is minimised.

Forward difference:

df/dx  ≈  [ f(x + h) − f(x) ] / h

Truncation error:  O(h)
Round-off error:   O(ε/h)   where ε ≈ machine precision

More decimal places do not automatically mean a better numerical approximation. A smaller step reduces truncation error but increases round-off error. The useful range is in the middle.

Floating-Point Numbers

Computers represent most real numbers approximately using floating-point arithmetic. A 64-bit floating-point number (double precision) has approximately 15–16 significant decimal digits. This is sufficient for most engineering calculations, but it is not infinite precision. The limitations of floating-point arithmetic affect every numerical computation.

• Round-off — most real numbers cannot be represented exactly; the stored value is the nearest representable number
• Comparison — exact equality (==) on floating-point numbers is unreliable; use tolerance-based comparison
• Subtraction of near-equal numbers — cancellation can lose significant digits
• Very large and very small scales — mixing values with widely different magnitudes can lose precision
• Accumulation — summing millions of numbers can accumulate round-off differently depending on order

Numerical precision is finite even when the screen displays many decimal places. A result printed to 12 decimal places may only be accurate to 6.

Numerical Integration

Numerical integration replaces a continuous integral with a weighted sum of function values at discrete points. The trapezoidal rule is the simplest method. Higher-order methods — Simpson’s rule, Gaussian quadrature — achieve greater accuracy for the same number of evaluation points by choosing optimal locations and weights.

Trapezoidal rule:

∫ f(x) dx  ≈  Σ [ f(xᵢ) + f(xᵢ₊₁) ] · Δx / 2

Simpson's rule:

∫ f(x) dx  ≈  Σ [ f(xᵢ) + 4·f(xᵢ₊₁) + f(xᵢ₊₂) ] · Δx / 3

Iterative Methods and Convergence

Many numerical methods are iterative — they produce a sequence of approximations that should converge to the solution. An iterative solver for a linear system, a Newton–Raphson iteration for a non-linear equation, a time-marching scheme for a transient problem — all produce successive approximations. Convergence means the sequence approaches the solution; the rate of convergence determines how many iterations are needed.

• Convergence criterion — when to stop iterating (tolerance on residual or change)
• Convergence rate — linear, quadratic, superlinear
• Stability — whether the iteration converges at all for the given problem
• Divergence — the iteration moves away from the solution; usually indicates a problem

Stability

A numerical method is stable if it does not amplify errors as the computation proceeds. Stability is distinct from accuracy — a method can be stable but inaccurate, or accurate but unstable for a particular problem. Explicit time integration schemes, for example, are conditionally stable: they are stable only if the time step is below a critical value (the Courant condition).

NUMERICAL CHECK: Does changing the discretisation (step size, mesh density, time step) materially change the result? If so, the discretisation is not sufficiently refined for the intended use.

Root Finding

Finding the roots of an equation — the values of x where f(x) = 0 — is a common engineering task. Methods include bisection (robust but slow), the secant method (faster but less reliable) and Newton–Raphson (fast but sensitive to initial guess and derivative quality). The choice depends on the smoothness of the function, the availability of derivatives and the required reliability.

When a Simpler Method Is Better

A common engineering error is using a high-order numerical method when a simpler one would suffice. If a trapezoidal integration gives sufficient accuracy for the engineering purpose, Simpson’s rule adds complexity without value. If a hand calculation can answer the question, a numerical method adds computational overhead and verification burden. The simplest method that provides a controlled, sufficiently accurate answer is the best engineering choice.

Verification

Numerical methods should be verified against analytical solutions where available, against published benchmark results, and by convergence studies — refining the discretisation and checking that the result converges to a stable value.

• Compare against an analytical solution where one exists
• Perform a convergence study — refine discretisation and check convergence
• Check that the result is insensitive to further refinement
• Verify floating-point precision is adequate for the problem

Key Takeaways

• Every numerical method introduces approximation — the question is whether it is controlled
• Truncation error decreases with finer discretisation; round-off error increases — there is an optimal range
• Floating-point precision is finite; more decimal places do not mean more accuracy
• Stability and accuracy are distinct — a method can be stable but inaccurate
• Use the simplest method that provides a controlled, sufficiently accurate answer