Lessons from 125 Years of US Bond and Stock Returns

US investors face the prospect of low returns on their main financial assets: stocks and bonds. Is there any reason to believe that stocks are a worse investment than bonds in the future? The analysis here suggests that stocks will likely yield higher returns than bonds, as they have done in most years. However, there is a significant chance—about 20 percent—that stocks will perform worse than bonds over the next ten years. Investing in stocks is not for the faint of heart.

The excess return to holding stocks relative to holding bonds is called the equity premium (EP). This premium compensates investors for the extra risk of holding stocks. The EP over any period in the past is an ex post EP. The expected future EP is referred to as the ex ante EP. The question is whether the ex ante EP is a good predictor of the ex post EP: In other words, should one buy stocks when the ex anteEP is high and buy bonds when the ex ante EP is low?

The analysis here is based on the dataset pioneered by Robert Shiller of Yale.¹ It includes dividends, earnings, and prices of the S&P 500 composite index, yields on the 10-year Treasury note, and consumer prices going back to January 1871. This posting focuses on 10-year holding periods, consistent with a long-term outlook for most investors and coinciding with the maturity of the bonds in our data.

The ex post EP is constructed as the nominal rate of return to holding the S&P 500 index and reinvesting all dividends for ten years minus the yield on the 10-year Treasury note at the start of the holding period.² The ex ante EP is constructed as the cyclically-adjusted earnings-price ratio of the S&P 500 minus the inflation-adjusted yield on the 10-year Treasury note.³ Inflation expectations are proxied by a 10-year average of past inflation.

Figure 1 displays the ex ante and ex post EPs over the sample.⁴ Ex post EP is dated from the start of the holding period so that it corresponds to the same holding period as the ex ante EP that was observed at that time. Ex post EP is not available after 2006 because stock price and dividend data are not available beyond January 2016.⁵ The average value of the ex post EP from 1881 through 2006 is 4.2 percent, which is somewhat lower than the average ex ante EP of 5.1 percent.

Both measures of the EP are affected by swings in stock prices. If stock prices are high at the beginning of the holding period or low at the end, the ex post EP is low. Both conditions held for the very low ex post EP in 1999 and 2000, reflecting the dot-com boom of 1999–2000 and the Great Recession of 2008–09. High stock prices also tend to reduce the ex ante EP, because they are in the denominator of the earnings-price ratio. Thus, the ex ante EP also was low in 2000, during the dot-com boom.

In January 2016, the ex ante EP was 3.8 percent, somewhat lower than its historical average. The S&P 500 index rose about 7 percent from its January average through March 31, suggesting that the ex anteEP currently is about 3.5 percent.

There is a clear positive correlation between the two measures of the equity premium. Figure 2 divides the sample into three buckets according to low, medium, and high values of the ex ante EP. For each subset of ex ante EP, the values of ex post EP are shown in histograms. Low values of ex ante EP are associated with low or medium values of ex post EP. Medium values of ex ante EP are associated with a wide range of ex post EP outcomes. High values of ex ante EP are associated with medium or high values of ex post EP.

A negative value of the ex post EP denotes a 10-year period in which stocks performed worse than bonds. Over the sample, the ex post EP was negative in 23 years.⁶ Out of these 23 years, 17 were associated with low values of ex ante EP and 5 with medium values of ex ante EP. In only one year, 1923, was a high ex ante EP associated with an ex post EP below zero.

To further explore the predictive power of the ex ante EP, the ex post EP was regressed on the ex anteEP. In order to check for the influence of structural changes, such as the introduction of income taxes, exiting the gold standard, changes in accounting standards, and better measures of inflation expectations, the regression was run on the second half of the sample. Table 1 shows that ex ante EP is a highly significant predictor of ex post EP. In the full sample, a 1 percentage point increase in ex anteEP is associated with a 0.6 percentage point increase in ex post EP. In the second half, this coefficient rises to 0.8.

Figure 3 projects ex post EP starting in 2007 based on the full-sample regression. Figure 4 projects ex post EP based on the second half regression. Both projections show an ex post EP continuously above zero, indicating that stocks are expected to have higher returns than bonds. However, the 95 percent confidence intervals around these projections are very wide. According to the full-sample regression, US stocks are expected to outperform US bonds by 3.5 percentage points (annualized average) over the ten years starting in January 2016, but there is a roughly 20 percent chance that stocks will perform worse than bonds.⁷

Note: *** and ** respectively denote statistical significance at the 1 and 5 percent level. AR(1) denotes a first-order autoregressive residual and MA(10) denotes a tenth-order moving average error. This specification proved to be a parsimonious way to allow for the MA(1-9) errors induced by the construction of ex post EP as a 10-year moving average of returns, but the results are not sensitive to using MA(1-9) errors.

Sources: Shiller’s database and authors’ calculations.

Notes

1. Shiller provides updated data from his book, Irrational Exuberance (Princeton University Press, 2000),on his website.

2. Implicitly, the yield is assumed to be on a zero-coupon bond to avoid the issue of how coupons are reinvested. For years in which both are available, par-coupon and zero-coupon yields are reasonably close. These returns are not adjusted for inflation. Ex post inflation is identical for bond and stock investors and would cancel out in the calculation of the EP.

3. The earnings-price ratio is effectively an inflation-adjusted rate of return on stocks because the value of corporate assets, and hence both future earnings and the equity price, increase with inflation. Shiller’s definition of cyclically-adjusted earnings is used, which is the 10-year backward average of earnings after adjusting earnings for inflation. The cyclically-adjusted ex ante EP is a slightly better predictor of the ex post EP than the ex ante EP based on the standard, four-quarter earnings-price ratio, but the difference is not large.

4. The figure starts in 1881 because ten years of inflation data are needed to construct ex ante EP.

5. Shiller’s annual data are based on observations for January of each year.

6. These 23 years are a subset of the 41 observations in the dark bars in figure 2.

7. The confidence interval is a bit narrower for the projection based on the second half regression. If the regression is run as the change in ex post EP regressed on the change in ex ante EP, the projections for the ex post EP look similar to those in figures 3 and 4. However, the confidence intervals are wider.