How Computers Calculate Trigonometric Functions

MathematicsProgrammingCompetitive Programming

Translated from Korean by Claude · Read the original

Note: This post is easier to follow if you know a bit of calculus. I've still included some proofs for the sake of explanation, so feel free to skip the parts you already know.

Right angles and rectangles always feel comfortable to us. Humanity invented the Cartesian coordinate system. And most computer monitors have pixels arranged in a grid of hundreds to thousands of pixels across and down, which you can address with Cartesian coordinates. I suppose it's because drawing the screen on a Cartesian grid makes it easiest and most flexible to manage each individual pixel.

Suppose someone is developing a game with an engine that only provides the four basic arithmetic operations. There's some shape drawn on a Cartesian plane, and they need to rotate it. How would they do it?

Actually, math already knows the answer. For a pixel at coordinates (x,y)\left ( x, y\right ), the rotated coordinates (x′,y′)\left ( x′, y′\right ) are

[cos⁡θ−sin⁡θsin⁡θcos⁡θ][xy]=[x′y′]\begin{bmatrix}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}=\begin{bmatrix}x′\\ y′\end{bmatrix}


in other words,

{x′=xcos⁡θ−ysin⁡θy′=xsin⁡θ+ycos⁡θ\begin{cases}x′=x \cos \theta-y\sin \theta\\ y′=x \sin \theta+y \cos \theta \end{cases}

which is self-evident. Do this once for the (x,y)\left ( x, y\right ) of every pixel in the original image, and you get the rotated image. Easy, right?

You want me to compute trig functions with nothing but the four basic operations?

Since we're discussing how computers calculate trigonometric functions, we should only use operations a computer can do. On a processor that supports only very basic operations, that is.

So how on earth can we approximate trigonometric functions? Once we have sin⁡\sin and cos⁡\cos, we can get tan⁡\tan and the like by division, so let's focus on finding sin⁡\sin and cos⁡\cos.

Couldn't we just approximate trig functions with polynomials?

Polynomials can be computed with just the four basic operations, so we could build a polynomial that looks like the graph of a trig function and plug values into that!

But how do we actually find that polynomial? This is where the Taylor series comes in.

Taylor Series

With the Taylor series, you can approximate any differentiable function with a polynomial. Does such black magic really exist? Yes, amazingly, it does!

Proof

First, let's take the definite integral of some function f′(x)f^\prime\left(x\right ) from aa to xx. By the definition of calculus, it can be written as follows.

∫axf′(t)dt=f(x)−f(a)\int_{a}^{x}f^\prime\left(t\right )\mathrm{d}t=f\left(x\right )-f\left(a\right )


Now let's rearrange the equation above.

∫axf′(t)dt=∫ax(−1)(−f′(t))dt\int_{a}^{x}f^\prime\left(t\right )\mathrm{d}t=\int_{a}^{x}\left(-1\right )\left(-f^\prime\left(t\right )\right )\mathrm{d}t


Now let's integrate −1-1 and differentiate −f′(t)-f^\prime\left(t\right ) to apply integration by parts. If f(x)f\left(x\right ) is infinitely differentiable, we can apply integration by parts infinitely many times too. So

⋯=(−(x−t)f′(t)−(x−t)22f′′(t)−(x−t)36f′′′(t)−⋯ )∣ax\cdots = \left.\begin{matrix}\left (-\left ( x-t \right )f^\prime\left ( t \right )- \dfrac{\left( x-t \right )^2}{2}f^{\prime\prime}\left ( t \right )- \dfrac{\left( x-t \right )^3}{6}f^{\prime\prime\prime}\left ( t \right )- \cdots\right )\end{matrix}\right|_{a}^{x}


Here, xx, which doesn't depend on the variable of integration tt, is treated as a constant. Expanding this equation gives

⋯=(x−a)f′(a)+(x−a)22!f′′(a)+(x−a)33!f′′′(a)+⋯\cdots =\left ( x-a \right )f^\prime\left ( a \right )+ \dfrac{\left( x-a \right )^2}{2!}f^{\prime\prime}\left ( a \right )+ \dfrac{\left( x-a \right)^3}{3!}f^{\prime\prime\prime}\left ( a \right )+ \cdots\\


=f(x)−f(a)= f\left ( x \right )-f\left ( a \right )


Finally, moving f(a)f\left ( a \right ) to the other side gives

f(x)=f(a)+(x−a)f′(a)+(x−a)22!f′′(a)+(x−a)33!f′′′(a)+⋯f\left ( x \right ) =f\left ( a \right )+ \left ( x-a \right )f^\prime\left ( a \right )+ \dfrac{\left( x-a \right )^2}{2!}f^{\prime\prime}\left ( a \right )+ \dfrac{\left( x-a \right )^3}{3!}f^{\prime\prime\prime}\left ( a \right )+ \cdots


That is, if f(n)f^{\left( n \right)} is the function obtained by differentiating ff nn times,

f(x)=∑n=0∞f(n)(a)n!(x−a)nf\left ( x \right ) = \sum_{n=0}^{\infty}\dfrac{f^{\left ( n \right )}\left ( a \right )}{n!}\left( x-a \right )^n


is derived. This approximation works for any value you put in for aa.

So what does this have to do with trig functions!

Trigonometric functions are infinitely differentiable. So if we put a trig function in place of ff, we can approximate the trig function with a polynomial. For a trig function f(x)f\left ( x \right ), substituting 00 for aa gives

f(x)=f(0)+(x−0)f′(0)+(x−0)22!f′′(0)+(x−0)33!f′′′(0)+⋯f\left ( x \right ) =f\left ( 0 \right )+ \left ( x-0 \right )f^\prime\left ( 0 \right )+ \dfrac{\left( x-0 \right )^2}{2!}f^{\prime\prime}\left ( 0 \right )+ \dfrac{\left( x-0 \right )^3}{3!}f^{\prime\prime\prime}\left ( 0 \right )+ \cdots


=f(0)+xf′(0)+x22!f′′(0)+x33!f′′′(0)+⋯=f\left ( 0 \right )+ x f^\prime\left ( 0 \right )+ \dfrac{x^2}{2!}f^{\prime\prime}\left ( 0 \right )+ \dfrac{x^3}{3!}f^{\prime\prime\prime}\left ( 0 \right )+ \cdots


and letting f(x)=sin⁡xf\left ( x \right ) = \sin x and substituting,

sin⁡x=sin⁡(0)+xsin⁡′(0)+x22!sin⁡′′(0)+x33!sin⁡′′′(0)+⋯\sin x =\sin\left ( 0 \right )+ x \sin^\prime\left ( 0 \right )+ \dfrac{x^2}{2!}\sin^{\prime\prime}\left ( 0 \right )+ \dfrac{x^3}{3!}\sin^{\prime\prime\prime}\left ( 0 \right )+ \cdots


=0+(x⋅1)+(x22!⋅0)+(x33!⋅−1)+(x44!⋅0)+(x55!⋅1)+⋯=0+ \left ( x \cdot 1 \right )+ \left ( \dfrac{x^2}{2!}\cdot 0 \right )+ \left ( \dfrac{x^3}{3!}\cdot -1 \right )+ \left ( \dfrac{x^4}{4!}\cdot 0 \right )+ \left ( \dfrac{x^5}{5!}\cdot 1 \right )+ \cdots


Simplifying,

∴sin⁡x=x−x33!+x55!−x77!+x99!+⋯\therefore \sin x =x- \dfrac{x^3}{3!}+ \dfrac{x^5}{5!}- \dfrac{x^7}{7!}+ \dfrac{x^9}{9!}+ \cdots


Likewise,

cos⁡x=1−x22!+x44!−x66!+x88!+⋯\cos x =1-\dfrac{x^2}{2!}+\dfrac{x^4}{4!}-\dfrac{x^6}{6!}+\dfrac{x^8}{8!}+ \cdots


Compared to the steps in between, it turned into a really simple formula. What's more, we've achieved our ultimate goal of expressing trig functions with only the four basic operations! (Factorials are just repeated multiplication, after all!)

But the formula is, well… infinite. What should we do about that? We all know that floating-point variables aren't all that accurate anyway. So we can compute the formula up to a certain point and just ignore the error beyond that.

Let's compute the polynomial we found above only up to the term of degree nn and ignore the rest. Let's call this the degree-nn approximation. From here on, I'll raise the degree step by step and plot the graph of the degree-nn approximation.

Graphs

▶ Degree-1 approximation: f(x)=xf\left ( x \right ) = x

… Doesn't look similar at all. Next!

▶ Degree-3 approximation: f(x)=x−x33!f\left ( x \right ) = x-\dfrac{x^3}{3!}

It seems to have gotten the hang of it near the start, but it starts getting a little off before even reaching π2\dfrac{\pi}{2}, and then it nosedives. Next!

▶ Degree-5 approximation: f(x)=x−x33!+x55!f\left ( x \right ) = x-\dfrac{x^3}{3!} + \dfrac{x^5}{5!}

Now it's similar at π2\dfrac{\pi}{2} too! Next!

▶ Degree-7 approximation: f(x)=x−x33!+x55!−x77!f\left ( x \right ) = x-\dfrac{x^3}{3!} + \dfrac{x^5}{5!}- \dfrac{x^7}{7!}

It's pretty close even around pipi now. Let's do a few more.

▶ Degree-9 approximation: f(x)=x−x33!+x55!−x77!+x99!f\left ( x \right ) = x-\dfrac{x^3}{3!} + \dfrac{x^5}{5!}-\dfrac{x^7}{7!} + \dfrac{x^9}{9!}

It's almost identical to the original function. Honestly, from here on, it looks like you could just use it as is for 0<x<π0 < x < \pi without much error. Since sinsin is periodic and symmetric anyway, as long as it's accurate on 0<x<π20 < x < \dfrac{\pi}{2}, you can compute other ranges using the properties of the function.

▶ Degree-11 approximation: f(x)=x−x33!+x55!−x77!+x99!−x1111!f\left ( x \right ) = x-\dfrac{x^3}{3!} + \dfrac{x^5}{5!}- \dfrac{x^7}{7!} + \dfrac{x^9}{9!}- \dfrac{x^{11}}{11!}

It's getting hard to tell how much they differ just by looking. Let me plot just one last one, the degree-13 approximation.

▶ Degree-13 approximation: f(x)=x−x33!+x55!−x77!+x99!−x1111!+x1313!f\left ( x \right ) = x- \dfrac{x^3}{3!} + \dfrac{x^5}{5!}- \dfrac{x^7}{7!} + \dfrac{x^9}{9!}- \dfrac{x^{11}}{11!} + \dfrac{x^{13}}{13!}

Within the range of the graph, the curves are practically identical.

Implementations in Programming Languages

The native code of OpenJDK's StrictMath implements trig functions using a degree-13 approximation. For accuracy, it doesn't use the degree-13 approximation as is, but modifies it slightly like this.

sin⁡x∼x−x33!+x55!−x77!+x99!−x1111!+x1313!\sin x \sim x- \dfrac{x^3}{3!} + \dfrac{x^5}{5!}- \dfrac{x^7}{7!} + \dfrac{x^9}{9!}- \dfrac{x^{11}}{11!} + \dfrac{x^{13}}{13!}


sin⁡xx∼1−x23!+x45!−x67!+x89!−x1011!+x1213!\frac{\sin x}{x} \sim 1- \dfrac{x^2}{3!} + \dfrac{x^4}{5!}- \dfrac{x^6}{7!} + \dfrac{x^8}{9!}- \dfrac{x^{10}}{11!} + \dfrac{x^{12}}{13!}


So when rr is defined like this,

r=x3(15!+x2(−17!+x2(19!+x2(−111!+x2⋅113!))))r = x^3 \left(\dfrac{1}{5!} + x^2 \left(-\dfrac{1}{7!} + x^2 \left(\dfrac{1}{9!} + x^2 \left(-\dfrac{1}{11!} + {x^{2}} \cdot \dfrac{1}{13!}\right)\right)\right)\right)


sin⁡x\sin x can be expressed like this.

sin⁡x∼x+x3⋅(−13!+x2r)\sin x \sim x + x^3 \cdot \left(-\dfrac{1}{3!} + x^2 r \right)


Let's look at an example. When the sin function calls __kernel_sin, if ∣x∣≺π/4|x| \prec \pi/4 in the code, the values of y and iy are both 0. For other ranges, it shifts the input into an appropriate range and computes the value of __kernel_sin or __kernel_cos.

#include "fdlibm.h"
#ifdef __STDC__
    static const double
#else
    static double
#endif
half = 5.00000000000000000000e-01, /* 0x3FE00000, 0x00000000 */
S1 = -1.66666666666666324348e-01, /* 0xBFC55555, 0x55555549 */
S2 = 8.33333333332248946124e-03, /* 0x3F811111, 0x1110F8A6 */
S3 = -1.98412698298579493134e-04, /* 0xBF2A01A0, 0x19C161D5 */
S4 = 2.75573137070700676789e-06, /* 0x3EC71DE3, 0x57B1FE7D */
S5 = -2.50507602534068634195e-08, /* 0xBE5AE5E6, 0x8A2B9CEB */
S6 = 1.58969099521155010221e-10; /* 0x3DE5D93A, 0x5ACFD57C */

#ifdef __STDC__
    double __kernel_sin(double x, double y, int iy)
#else
    double __kernel_sin(x, y, iy) double x,y; int iy; /* iy=0 if y is zero */
#endif
{
    double z,r,v; int ix;
    ix = __HI(x)&0x7fffffff; /* high word of x */
    if(ix < 0x3e400000) /* |x| < 2**-27 */ {
        if((int)x==0) return x;
    } /* generate inexact */
    z = x*x;
    v = z*x;
    r = S2+z*(S3+z*(S4+z*(S5+z*S6)));
    if(iy==0) return x+v*(S1+z*r);
    else return x-((z*(half*y-v*r)-y)-v*S1);
}

In the source, you can see that the coefficients of the approximation are approximated and hardcoded as
S1=−13!≈−1.66666667×10−1S1 = -\dfrac{1}{3!} \approx -1.66666667 \times {10}^{-1}
S2=15!≈8.33333333×10−2S2 = \dfrac{1}{5!} \approx 8.33333333 \times {10}^{-2}
S3=−17!≈−1.98412698×10−4S3 = -\dfrac{1}{7!} \approx -1.98412698 \times {10}^{-4}
S4=19!≈2.75573137×10−6S4 = \dfrac{1}{9!} \approx 2.75573137 \times {10}^{-6}
S5=−111!≈−2.50521084×10−8S5 = -\dfrac{1}{11!} \approx -2.50521084 \times {10}^{-8}
S6=113!≈1.58969099×10−10S6 = \dfrac{1}{13!} \approx 1.58969099 \times {10}^{-10}
. It also computes z=x2z=x^2 and v=zx=x3v=zx=x^3. If you used the degree-13 approximation as is, you'd need 1, 3, 5, …, 13 multiplications for each term, but treating x2x^2 as a new constant cuts down on multiplications, which I think is probably why they used this approach.

Are there other ways to calculate them?

Of course there are. For example, you can also approximate trig function values using the CORDIC (COordinate Rotation DIgital Computer) algorithm, devised by Volder in 1956.

To explain it briefly: at the very top, I introduced the matrix we multiply by to rotate as

[cos⁡θ−sin⁡θsin⁡θcos⁡θ]\begin{bmatrix}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta\end{bmatrix}


but the CORDIC algorithm expresses this using only the tan⁡\tan function, transforming it into

Ri=11+tan⁡2(γi)[1−tan⁡(γi)tan⁡(γi)1]R_i = \dfrac{1}{\sqrt{1 + \tan^2 \left(\gamma_i\right)}}\begin{bmatrix}1 & -\tan \left(\gamma_i\right) \\\tan \left(\gamma_i\right) & 1\end{bmatrix}


and, so that computers can calculate it quickly, hardcodes the values of γi\gamma_i such that tan⁡(γi)=±2−i\tan \left(\gamma_i\right) = \pm2^{-i}, i.e. arctan⁡(2−i)\arctan\left(2^{-i}\right), along with the corresponding values of 11+tan⁡2(γi)\dfrac{1}{\sqrt{1 + \tan^2 \left(\gamma_i\right)}}. Then, increasing ii one by one until it gets close to the desired angle, it carries out the matrix computation while
adding or subtracting arctan⁡(2−i)\arctan\left(2^{-i}\right).

This algorithm is mainly used in calculators, and it has also been used in Intel's 8087 through 80486 CPUs.

And that's how computers calculate trigonometric functions.

4 comments