Sloppy Math and the Austerity Debate

In 2010, two Harvard economists, Carmen Reinhart and Kenneth Rogoff, circulated a paper demonstrating that GDP growth is negatively correlated to public debt (debt-to-GDP ratio to be more precisely).  Their paper was highly influential and has been used to support the global austerity agenda.  For instance, it was cited in the “Paul Ryan Budget,” p. 80.  But the problem is that the paper is rife of basic math errors, as pointed out in a recent paper by Thomas Herndon, Michael Ash and Robert Pollin of the University of Massachusetts, Amherst.  On April 19, Paul Krugman wrote an op-ed piece entitled “The Excel Depression” for the New York Times publicizing an embarrassing Excel error in Reinhart and Rogoff’s spreadsheet.  (You can see the Excel screen capture here; instead of AVERAGE(L30:L49), Reinhart and Rogoff entered AVERAGE(L30:L44), excluding Australia, Austria, Belgium, Canada, and Denmark from their calculation.)  A week later, Reinhart and Rogoff contributed their own op-ed piece; they suggested that their mistake was inconsequential and continued their claim that “growth is about 1 percentage point lower when debt is 90 percent or more of gross domestic product.”  In their online appendix, the chart below was shown to make such a point.

NYTRR

Source: Reinhart and Rogoff’s New York Times online appendix.

You can find extensive discussions over the Reinhart-Rogoff controversy on the internet.  Here I just want to highlight some of the technical issues involving merely elementary arithmetic.  In fact, Thomas Herndon, a PhD candidate, discovered RR’s Excel error through a course assignment.  Students were supposed to replicate the findings of a famous paper in order to learn econometric techniques.  As RR’s work involves essentially calculating the means and medians, Herndon’s professor almost didn’t let him take on the project.  (See this New York Magazine article and this Colbert Report interview.)

A casual reader might get an impression of downward slide in GDP growth based on the above chart, but a careful reader might have noticed that the chart compares medians and arithmetic averages.  Students who have taken an introductory statistics course know that medians and means (often loosely referred to as averages) can be quite different if the distribution is skewed.  Furthermore, means and medians without accompanying measure of how much the data is spread out (such as standard deviation or interquartile range) provide incomplete information.

I wrote to Professor Ash and he kindly supplied additional information.  He and Professor Pollin published a response to Reinhart and Rogoff in the Times, with a comprehensive technical supplement.  They also made the data public to allow a close examination.  To illustrate the effect of Reinhart and Rogoff’s Excel error, one can make a comparison between the New York Times chart and a corrected one based on the inclusion of Australia, Austria, Belgium, Canada, and Denmark.  This inclusion alone increases median GDP growth by 0.3 percentage points in the 90 percent and above public debt/GDP category.

RRc1

Median figures for blue line are from the NY Times; median figures for red line are from Table 1 of technical supplement by Ash and Pollin, “RR calculation method but with corrected spreadsheet.” The Excel error alone is responsible for a difference of 1.9-1.6=0.3 percentage points in RR’s above 90% debt/GDP category.

It is unclear why RR excluded available data of earlier years for Australia, Canada and New Zealand.  By including these three countries, it further increases median GDP growth in RR’s highest debt/GDP category to 2.5%, which is only 0.4 percentage points lower than that in the next highest debt/GDP category.

RRc2

Median figures for blue line are from the NY Times; median figures for red line are from Table 1 of technical supplement by Ash and Pollin, “recalculation with both corrected spreadsheet calculations and inclusion of Australia, Canada and New Zealand early years.” The difference in median GDP growth is only 2.5-2.9=-0.4 percentage points between 60 to 90% category and above 90% category.

While most economists acknowledged some correlation between high debt and low GDP growth, the above 2 charts illustrate that the claim “growth is about 1 percentage point lower when debt is 90 percent or more of gross domestic product” by Reinhart and Rogoff is based on sloppy math and is unsubstantiated.

In the Times online appendix, Reinhart and Rogoff stated that they “gave significant weight to the median estimates, precisely because they reduce the problem posed by data outliers.”  Yet two paragraphs later, when reporting their findings in a paper published in 2012 (joined by Vincent Reinhart), they gave the means only.  They compared the mean from 1800 to 2011, 2.3 percent, to the mean of Herndon et al. from 1945 to 2009, 2.2 percent.  If we just cherry pick numbers, let us compare the tainted result that Reinhart and Rogoff published in the Times with means from 2000 to 2009 only.  Contrary to Reinhart and Rogoff’s central claim, GDP growth in the over 90 percent public debt/GDP category has actually outperformed GDP growth in the 60 to 90% public debt/GDP category.

RRHAP2009

In this chart, the median figures for blue line are again from the NY Times, and the mean figures are from Table 4 of technical supplement by Ash and Pollin. Data from 2000 to 2009 contradict Reinhart and Rogoff’s claim.

But the above chart is not particularly meaningful, because the standard deviation of each GDP growth figure (as shown in Ash and Pollin’s supplement) is at least 0.3.  A more responsible way to present the means is the following graph, taken from their supplement.

gdpgrowth2009

Source: technical supplement by Ash and Pollin, Figure 2.

People who have working experience with data know that mean or median conceals a lot of information.  In this case, it is evident that there is a wide range of economic performance outcomes in a given category.  Using the file containing data from 1946 to 2009 (the basis of the New York Times line chart above), one can make a scatterplot.  It is apparent that the relationship between public debt and GDP growth varies significantly.

reproducedRR

Source: The file RR-processed.dta posted in U Mass website. Working spreadsheet was provided by RR; the U Mass group corrected errors and cleaned up the data.

By now, you should have realized that the same data can be graphed in different ways, which can lead to a vastly different impression.  The Reinhart-Rogoff affair offers educators a great opportunity to introduce many other quantitative reasoning topics, such as which measures of central tendency to use, correlation does not imply causation (does high debt cause low growth, or the other way around?), and so on.  It is quite feasible to ask students to perform their own analyses and make their own inferences, and I strongly encourage you to do so.

This entry was posted in Uncategorized. Bookmark the permalink.

46 Responses to Sloppy Math and the Austerity Debate

  1. Mp3 Download says:

    This Is Awesome Guys.. Keep it up

  2. SallieStevens says:

    opportunity to help each other. We are hoping to have the same goal for the future of ours. ole777 mobile

  3. Finally i have gotten a site that solved my problem! thank you

  4. totosafeguidecom says:

    Esto es realmente interesante. Eres un blogger muy hábil.
    Me uní a tu feed y espero seguir buscando más de tu excelente publicación.
    Además, ¡he compartido su sitio web en mis redes sociales!

    https://totosafe-gguidecom.jouwweb.nl/

  5. CG Board Examination every year more than laks of students. CG Board 12th Exam are conducted by Chhattisgarh Board of Secondary Education Only, Boardmodelpaper.com Provide Latest Study Material Hindi, English Medium Pdf Format, CGBSE 12th Exam Pattern 2022 Students can Start Higher Secondary (12th Class) Exam Preparation, CGBSE HS Model Paper 2022 to help with the Preparation, Study Material Available for both Regular and Self- Study Students as a part of their Preparation methods.

  6. tiger388 says:

    Nice site, i like this site

  7. Mp3 Download says:

    I have learnt alot. Thank you for this!

  8. The most difficult calculations i have ever come across

  9. Teknik menggertak atau bluffin ini dapat dikatakan sebagai salah satu cara ampuh untuk memenangkan permainan

  10. طراحی آپارتمان

  11. kualitasqq says:

    awesome, thanks for share

  12. love this site, thanks

  13. Teknik menggertak atau bluffin ini dapat dikatakan sebagai salah satu cara ampuh untuk memenangkan permainan

  14. sedakadeh says:

    hey how can i contact with you for ad ?

  15. cactusmusic says:

    There is no fear in life
    We must learn not to be afraid of anything in life
    If we are to succeed, we must learn to be brave

  16. shahrMusic says:

    This article is very helpful. keep sharing such information.

  17. Teknik menggertak atau bluffin ini dapat dikatakan sebagai salah

  18. You can find extensive discussions over the Reinhart-Rogoff controversy on the internet. Here I just want to highlight some of the technical issues involving merely elementary arithmetic

  19. zahra says:

    Thank you for raising your level of knowledge

  20. melika says:

    It was great and first class was really helpful thanks to your good team

  21. maryam says:

    Thank you for your efforts to produce such engaging content
    Thank You

  22. i love this site, thanks

  23. Nice site, thanks for shareing

  24. mehrab says:

    Teknik menggertak atau bluffin ini dapat dikatakan sebagai salah

  25. mehrab says:

    Teknik menggertak atau bluffin ini dapat dikatakan sebagai salah

  26. Dari permulaan permainan yang berlangsung baiknya anda perhatikan

  27. pasarqq says:

    The people in your life may or may not have values and habits that are similar to your own

  28. Teknik menggertak atau bluffin ini dapat dikatakan sebagai salah satu cara ampuh untuk memenangkan permainan

  29. hobiqq says:

    Ada juga taktik memperhatikan cara bermain dari lawan anda. Dari permulaan permainan yang berlangsung baiknya anda perhatikan

  30. interqq says:

    Namun jangan terlalu sering menggunakan teknik ini

  31. sumoqq says:

    Cara ini memang bagus untuk menaikan peluang kemenangan anda

  32. dewaqq says:

    Teknik menggertak atau bluffin ini dapat dikatakan sebagai salah satu cara ampuh untuk memenangkan permainan

  33. Whether you’re going through a break up, you’ve just lost your job, or you simply woke

  34. Pingback: Stencila – Spreadsheet-like Live Reactive Programming Environment | 神刀安全网

Comments are closed.