4.13 · AHL · HL onlyPaper 1Paper 2Paper 3

Non-linear regression and R²

In a minute

Real data often follow a curve, not a line. Your GDC finds the best curve of the type you choose, and you judge the fit with the and R2R^{2}. A high R2R^{2} never settles the choice alone: the context matters too.

You'll be able to

  • Fit quadratic, cubic, exponential, power and sine models with your GDC, and write the model in context.
  • Find and the sum of square residuals for a model, and use SSresSS_{\text{res}} to compare fits.
  • Find R2R^{2} with your GDC and interpret it as the proportion of the variation in yy that the model accounts for.
  • Use R2=r2R^{2} = r^{2} for a linear model, and see why R2=1R^{2} = 1 when SSres=0SS_{\text{res}} = 0.
  • Choose between models using context, the shape of the data and , not R2R^{2} alone.
Before you start3 quick questions on what this builds on
Check yourself
A linear regression gives r=−0.93r = -0.93. Which description fits the data?
Check yourself
A model is y=120×0.8ty = 120 \times 0.8^{t}. Find yy when t=3t = 3. Give your answer to 3 s.f.
Check yourself
Data plotted as ln⁡y\ln y against xx lie on the line ln⁡y=0.2x+1.5\ln y = 0.2x + 1.5. Find yy when x=0x = 0, to 3 s.f.

Revise first: 1.3 Geometric sequences and series, 2.2 Function concepts, domain and range, 2.5 Modelling with functions, 2.6 Fitting and testing models, 2.10 Logarithmic scales and linearizing data and 4.4 Correlation and regression (y on x)

Fitting curves with your GDC

In 4.4 you fitted a straight line. Many real relationships curve instead: revenue that peaks, a drug that wears off, daylight that rises and falls through the year.

You choose the type of model. Your GDC then finds its by : of all curves of that type, it picks the one that passes closest to the points, measured vertically. For exponential and power models, the GDC does this for ln⁡y\ln y instead of yy.

The model types in exam questions, with the pattern that suggests each:

1. Linear y=ax+by = ax + b: a steady rate of change. 2. Quadratic y=ax2+bx+cy = ax^{2} + bx + c: one peak or one trough. 3. Cubic y=ax3+bx2+cx+dy = ax^{3} + bx^{2} + cx + d: up to two turning points.

4. Exponential y=abxy = ab^{x}: the same percentage change for each step in xx, with a>0a > 0, growth if b>1b > 1 and decay if 0<b<10 < b < 1. 5. Power y=axby = ax^{b}: yy varies with a power of xx, as mass varies with length. 6. Sine y=asin⁡(bx+c)+dy = a\sin(bx + c) + d: a repeating cycle, such as tides.

Worked exampleA bakery tries six prices pp (dollars) for its sourdough loaf, one per week, and records the weekly revenue RR (dollars). pp: 3.0,3.5,4.0,4.5,5.0,5.53.0, 3.5, 4.0, 4.5, 5.0, 5.5. RR: 612,695,726,748,712,671612, 695, 726, 748, 712, 671. Use your GDC to find the least squares quadratic model for RR in terms of pp. Hence find the price that gives the greatest revenue according to the model, to the nearest cent.

Try it first, then reveal one step at a time.

  1. The revenue rises and then falls: one peak, so a quadratic suits the shape.
Worked exampleThe caffeine concentration CC (mg per litre) in a person's blood is measured tt hours after a strong coffee. tt: 1,2,3,4,5,61, 2, 3, 4, 5, 6. CC: 7.4,6.3,5.2,4.5,3.7,3.27.4, 6.3, 5.2, 4.5, 3.7, 3.2. Use exponential regression to find a model C=abtC = ab^{t}, and interpret bb in context. Give aa and bb to 3 s.f.

Try it first, then reveal one step at a time.

  1. Each hour CC falls by a similar fraction: 6.37.4≈0.85\frac{6.3}{7.4} \approx 0.85 and 5.26.3≈0.83\frac{5.2}{6.3} \approx 0.83. That suggests exponential decay.

Some models add a known constant, such as T=abt+20T = ab^{t} + 20 for a drink cooling in a room at 20∘C20^{\circ}\text{C}. The GDC's exponential regression has no added constant. So subtract 2020 from every TT first, then run the regression on T−20T - 20 against tt.

Check yourself
Apples of diameter dd cm have mass mm g. dd: 4,5,6,7,84, 5, 6, 7, 8. mm: 30,61,101,162,24330, 61, 101, 162, 243. Use power regression to fit m=adbm = ad^{b}. Find bb to 3 s.f.
Check yourself
Sine regression on daylight hours HH for day tt of the year gives H=3.6sin⁡(0.0172t−1.39)+12.2H = 3.6\sin(0.0172t - 1.39) + 12.2, with angles in radians. Find the period of the model in days, to 3 s.f.
Check yourself
A harbour records the depth of water every hour for three days. Which type of model should you fit?

Residuals and the sum of square residuals

How close is a model to the data? Look at the vertical gap between each point and the curve. In 4.4 the made the squares of these gaps add up to as little as possible.

x y observed y predicted residual negative residual model
Residuals are vertical gaps: positive above the curve, negative below it.

Positive and negative residuals partly cancel, so their plain sum tells you little. Squaring makes every gap count, and big gaps count most.

Learn this
SSres=∑i=1n(yi−y^i)2SS_{\text{res}} = \sum_{i=1}^{n} \left(y_{i} - \hat{y}_{i}\right)^{2}
Worked exampleA gardener models a seedling's height hh cm after ww weeks by h=2.5×1.3wh = 2.5 \times 1.3^{w}. She measures h=3.1,4.4,5.3,7.2h = 3.1, 4.4, 5.3, 7.2 at w=1,2,3,4w = 1, 2, 3, 4. Find SSresSS_{\text{res}} for this model, to 3 s.f.

Try it first, then reveal one step at a time.

  1. Predicted heights: 2.5×1.3=3.252.5 \times 1.3 = 3.25, 2.5×1.32=4.2252.5 \times 1.3^{2} = 4.225, 2.5×1.33=5.49252.5 \times 1.3^{3} = 5.4925, 2.5×1.34=7.140252.5 \times 1.3^{4} = 7.14025.
Worked exampleA road-safety study models braking distance dd m against speed vv m/s by d=kv2d = kv^{2}. Its data: v=10,15,20,25v = 10, 15, 20, 25 give d=7.8,17.0,31.5,48.9d = 7.8, 17.0, 31.5, 48.9. Write SSresSS_{\text{res}} as a quadratic in kk, and hence find the least squares value of kk to 3 s.f.

Try it first, then reveal one step at a time.

  1. Predicted distances: 100k100k, 225k225k, 400k400k and 625k625k.
Check yourself
Two models are fitted to the same 1010 readings of a phone's battery temperature. Model A has SSres=14.2SS_{\text{res}} = 14.2 and model B has SSres=9.6SS_{\text{res}} = 9.6. Which statement is correct?

The coefficient of determination

The yy-values in a data set vary. Part of that variation follows the model: as xx changes, the model's yy changes too. The rest is scatter that the model misses.

Close to 11: the points lie close to the curve. Close to 00: the model accounts for little of the variation. Interpret it with the context's variable names.

Worked exampleFor eight one-bedroom flats, xx is the distance in km from a city centre and yy is the monthly rent in euros. xx: 0.8,2.1,3.5,5.2,6.4,8.0,9.7,11.30.8, 2.1, 3.5, 5.2, 6.4, 8.0, 9.7, 11.3. yy: 1480,1420,1290,1310,1150,1180,1010,10201480, 1420, 1290, 1310, 1150, 1180, 1010, 1020. Find R2R^{2} for the linear regression model of yy on xx and interpret it.

Try it first, then reveal one step at a time.

  1. Enter the lists and run linear regression. The GDC gives r=−0.96563…r = -0.96563\ldots and r2=0.93244…r^{2} = 0.93244\ldots
Worked exampleA seed lab keeps batches of seeds at temperature TT (°C) and records the percentage GG that germinate. TT: 10,14,18,22,26,30,3410, 14, 18, 22, 26, 30, 34. GG: 31,52,70,78,74,61,3831, 52, 70, 78, 74, 61, 38. Use your GDC to find R2R^{2} for the quadratic regression model of GG on TT, and interpret it.

Try it first, then reveal one step at a time.

  1. GG rises and then falls, so try a quadratic. On a TI-84, turn on stat diagnostics first or R2R^{2} is not shown.

Where R2R^{2} comes from (useful for understanding; the exam does not test this formula). Compare SSresSS_{\text{res}} with the total variation of yy about its mean yˉ\bar{y}:

Learn this
R2=1−SSresSStot,SStot=∑i=1n(yi−yˉ)2R^{2} = 1 - \frac{SS_{\text{res}}}{SS_{\text{tot}}}, \qquad SS_{\text{tot}} = \sum_{i=1}^{n} \left(y_{i} - \bar{y}\right)^{2}

SStotSS_{\text{tot}} is the SSresSS_{\text{res}} of the flat model y=yˉy = \bar{y}, which ignores xx completely. If every point lies on the curve, SSres=0SS_{\text{res}} = 0, so R2=1R^{2} = 1.

Check yourself
A linear model of a used car's value against its age has R2=0.7225R^{2} = 0.7225. Find Pearson's rr, to 3 s.f.
Check yourself
A model has SSres=18.4SS_{\text{res}} = 18.4 and SStot=152SS_{\text{tot}} = 152. Find R2R^{2}, to 3 s.f.
Check yourself
A quadratic model of wheat yield against rainfall has R2=0.64R^{2} = 0.64. Which statement is correct?

Choosing between models

A high R2R^{2} says the curve is close to the points you have. It doesn't say the model suits the situation, or that predictions outside the data can be trusted.

Before you trust a model, check: 1. Context. Does it behave sensibly? Sales can't be negative, and a cooling drink levels off at room temperature.

2. Extrapolation. A prediction outside the range of the data assumes the pattern carries on. Quadratic and cubic models often turn sharply just outside the data.

3. Extra terms. On the same data a cubic never has a lower R2R^{2} than a quadratic, because a quadratic is a cubic with a=0a = 0. So a slightly higher R2R^{2} from a cubic is weak evidence.

4. Like with like. For exponential and power regression your GDC fits a straight line to ln⁡y\ln y, and the r2r^{2} it shows belongs to that line. Don't rank it directly against a quadratic's R2R^{2}.

0 5 10 15 20 0 20 40 60 80 100 120 week sales (thousands) beyond the data quadratic exponential
Both models fit weeks 11 to 88. Only one makes sense beyond them.
Worked exampleWeekly sales SS (thousands) of a new video game in weeks t=1t = 1 to 88 are 52,41,33,27,22,18,15,1352, 41, 33, 27, 22, 18, 15, 13. Fit a quadratic model, use it to predict sales in week 2020, and comment on the prediction.

Try it first, then reveal one step at a time.

  1. Quadratic regression gives S=0.673t2−11.4t+62.0S = 0.673t^{2} - 11.4t + 62.0, with R2=0.998R^{2} = 0.998: a very close fit to weeks 11 to 88.
Worked exampleA forester measures a tree's height hh m at ages 2,4,6,8,10,12,142, 4, 6, 8, 10, 12, 14 years: 1.2,2.6,4.2,5.7,6.5,7.1,7.41.2, 2.6, 4.2, 5.7, 6.5, 7.1, 7.4. A quadratic model has R2=0.996R^{2} = 0.996 and a cubic model has R2=0.998R^{2} = 0.998. She picks the cubic because its R2R^{2} is higher, and uses it to predict the height at 2525 years. Comment on her choice.

Try it first, then reveal one step at a time.

  1. A cubic is a quadratic plus one extra term, so on the same data its R2R^{2} is never lower. A rise from 0.9960.996 to 0.9980.998 is no real evidence.
Check yourself
A museum fits V=−0.8t3+12t2+5t+40V = -0.8t^{3} + 12t^{2} + 5t + 40 to daily visitors VV on days 0≤t≤120 \le t \le 12 of an exhibition. Find the model's value of VV at t=18t = 18.
Check yourself
On the same data, model P has R2=0.97R^{2} = 0.97 and model Q has R2=0.95R^{2} = 0.95. Which conclusion is justified?

Common mistakes

  • Fitting the wrong model: linear regression on curved data, or yy against xx when the question asks for ln⁡y\ln y against xx. Use the form and variables the question names.
  • Working out a residual as predicted minus observed. A residual is observed minus predicted, so a point above the curve has a positive residual.
  • Rounding parameters to 2 s.f., or using rounded parameters for later predictions. Give 3 s.f. and use the GDC's stored values.
  • Stating which model is better with no reason. Quote both values (R2R^{2} or SSresSS_{\text{res}}), compare them, and add a reason from the context.
  • Ignoring a known added constant. For y=abx+20y = ab^{x} + 20, subtract 2020 from the yy-values before running exponential regression.
  • Trusting a high R2R^{2} far outside the data. Check what the model predicts there and whether it makes sense.

Exam tips

  • Find the model: write which regression you ran, then the equation with each parameter to 3 s.f., using the question's variables (such as CC and tt, not yy and xx).
  • Find SSresSS_{\text{res}}: show the predicted values and the sum of squared differences before the final value. Those lines earn the method marks.
  • Interpret R2R^{2}: say what proportion of the variation in the named variable the model accounts for, such as 'about 93%93\% of the variation in rent'.
  • Justify which model is better: compare the two values, then give a reason from the context, such as long-term behaviour. A choice with no reason earns no marks.

Key takeaways

Learn thisChoose the model type from the context and the scatter diagram (or the form the question names); the GDC fits the parameters. Give them to 3 s.f.
Learn thisResidual =y−y^= y - \hat{y}, observed minus predicted: positive above the curve, negative below.
Learn thisSSres=∑i=1n(yi−y^i)2SS_{\text{res}} = \sum_{i=1}^{n} \left(y_{i} - \hat{y}_{i}\right)^{2}
Learn thisFor models fitted to the same data, a smaller SSresSS_{\text{res}} means a closer fit; the least squares curve of a type has the smallest SSresSS_{\text{res}} of that type (the GDC's exponential and power regressions minimise it for ln⁡y\ln y instead).
Learn thisR2R^{2} is the proportion of the variation in yy that the model accounts for. For a linear least squares model, R2=r2R^{2} = r^{2}.
Learn thisR2R^{2} alone does not choose a model: check the context, the shape and any extrapolation. On the same data, a cubic regression's R2R^{2} is never lower than a quadratic regression's.

Practise this

15 past-paper questions on Non-linear regression and linearizing in the AI HL bank, each with a mark scheme and a worked explanation.

Practise Non-linear regression and linearizing →