In my last post I briefly mentioned Downside Risk-Adjusted Success (D-RAS) and the Coverage Ratio. Both of those come from Javier Estrada and Mark Kritzman. I noticed that they have a new paper out in the beginning of 2019 and reviewing that serves as a convenient platform to explain those two metrics a bit more.
The Coverage Ratio is Estrada & Kritzman’s current best metric. It is the culmination of a journey that Estrada in particular has been on for several papers.
A common complaint about failure rates is that “it assumes investors are indifferent between running out of money early in retirement or near the end of retirement”. That is, running out of money in Year 17 is worse than running out of money in Year 29 (when you are almost certainly dead anyway). But they are treated identically by failure rate metrics.
In a 2017 paper, “Refining the Failure Rate”, Estrada took the Dimson-Marsh-Staunton database of global returns fro 1900–2014 and calculated “shortfall years” for all the countries in the dataset.
Already this new metric gives a few interesting insights. If you look solely at failure rates, Sweden failed 16% of the time and the Netherlands 23% of the time. Clearly Sweden “wins”?
But the shortfall in the Netherlands is only 4.7 years on average and the “sustained percentage” was 84%, meaning 84% of all possible retirement years were fully funded. In Sweden the shortfall was 7.3 years on average and the sustained percentage was 75%.
When we look at the size of the shortfall, the Netherlands looks better and Sweden looks worse.
Risk-Adjusted Success (RAS)
The next was a paper the following year, “From Failure to Success: Replacing the Failure Rate”, which looked at shortfall years across a variety of asset allocations and introduced the RAS metric.
Looking at USA data the 80/20 portfolio had the fewest shortfall years. When it failed, it failed (on average) after 28 years. A 60/40 portfolio failed after 27 years. A 100/0 portfolio failed after 25.3 years.
Looking at the World, the 100/0 portfolio did best. When it failed, it failed after 25.2 years. A 60/40 portfolio failed after 21.6 years.
Next Estrada introduces RAS, which is a more complicated metric.
RAS: Years Sustained
First we want to calculate the “years sustained”. When a portfolio fails, the number is easy. We just subtract the shortfall years from the target length of retirement. If our target was 30 years and we had a 3 year shortfall then “years sustained” is 27.
But what about when the portfolio lasts the full 30 years? Estrada’s idea is to take the bequest, what’s left at the end of the 30 years, and count it as “extra” years. You calculate this by dividing the (inflation-adjusted) final portfolio value by the (inflation-adjusted) withdrawal amount. (Remember he’s using constant dollar withdrawals.) If you withdraw $40,000 a year and the final portfolio value was $236,000 then 236000/40000 = 5.9 extra years. That makes the “years sustained” = 30 + 5.9 = 35.9.
This is actually a pretty nice way of taking the unifying “final portfolio size” and “magnitude of failure”, which I hadn’t seen done before.
RAS: Expected Value
We can put together all of “years sustained” to form an “expected value”. Expected value is just a calculation that takes probability into account. If I have a 10% chance of winning $100 and a 90% chance of losing -$50 then my expected value is (.10 x 100) + (.90 x -50) = (10) + (-45) = -35.
Here is Estrada’s formula for how expected value of a portfolio is calculated:
This is straightforward:
- Calculate how long the portfolio lasts (on average) when it fails. e.g. the average shortfall is 3.7 years. So the average “years sustained” when the portfolio fails is 26.3 years.
- Then we multiply that by the failure rate. (e.g. continued) The failure rate is 3.5%. 0.035 x 26.3 = 0.9205.
- Calculate how long the portfolio lasts (on average) when it succeeds. (e.g. continued) We look at the final portfolio size and find that, on average, it resulted in 6.7 extra years. So the “years sustained” is 36.7.
- Then we multiply that by the success rate. (e.g. continued) The success rate is 96.5%. 0.965 x 36.7 = 35.4155.
- Then we add the two numbers together. 0.9205 + 35.4155 = 36.336.
So a retiree can “expect” that their portfolio will last 36.336 years.
RAS: standard deviation of years sustained
The next step is calculate the standard deviation of all those years sustained. This is the volatility of how long the portfolio lasts.
Here’s how it is calculated. Let’s assume there are just 3 results: a retirement that lasted 27 years, one that lasted 35 years, and one that lasted 32 years. So 1 failure and 2 successes.
The “expected value” is
=(0.3333 * 27) * (0.6666 * 33.5)
And the standard deviation is
a = (0.3333 * (27 - 31.3333))
b = (0.6666 * (33.5 - 31.3333))
= sqrt((a*a) + (b*b))
This is just volatility. So a low number means you are “more certain” that you’ll hit your expected value (whatever that expected value is).
RAS: Putting it all together
The standard deviation along only tells us about volatility, not about how long the portfolio lasts. You could have a strategy that always lasts exactly 26 years. Low volatility but also low expected value.
Out final step in building RAS is to take the ratio of the “expected value” and the “standard deviation of years sustained”.
From our example above that would be:
=31.333 / 3.064
A higher number is better. RAS can be high for two reasons:
- The expected value is high. That is, on average, the portfolio lasts longer.
- The standard deviation is low. That is, there is more certainty that we’ll end up close to the expected value.
There we go, we have Risk-Adjusted Success. It is a lot like the Sharpe Ratio, except applied to retirement outcomes. However it also shares the shortcomings of the Sharpe Ratio.
- It treats upside & downside as symmetric. But investors clearly don’t think that way. No one says “my retirement was a failure, my portfolio ended up twice as big as I expected”.
- A portfolio with very low volatility is going have a very high RAS, regardless of how low the expected value is. Our example above, of the strategy that always lasts exactly 26 years has an amazingly good RAS, infinite in fact, even though few people would want to risk using a strategy with such a low expected value.
If we look again at the US results, we can see problem #1 in action.
If we just look at shortfall years, we’d pick the 80/20 allocation. But if we look at RAS, we’d pick the 70/30 allocation. But why does RAS prefer 70/30?
80/20 lasts longer when the portfolio succeeds: 89.6 years versus 80.2 years.
80/20 lasts longer when the portfolio fails: 28.0 years versus 27.3 years.
It seems like we’d always prefer 80/20, right? But 80/20 has more volatility on the upside, giving it a higher standard deviation of expected value, resulting in a lower RAS.
If we look at Spain, we can see an example of problem #2. This isn’t to say that RAS is right or wrong. Just that metrics are tricky and being overly focused on a single one can sometimes lead us astray.
If we just look at failure rates, then 100/0 is the clear winner. If we look at shortfall years, then 30/70 is the clear winner. If we look at RAS, then 0/100 is the clear winner. In this case:
100/0 has an expected value of 45.6 years but a standard deviation of 21.8.
0/100 has an expected value of 25.6 years (44% lower) but a standard deviation of only 7.2 (67% lower). The standard deviation fell by more than the expected value, so the ratio increases. Yet few people would choose a 0/100 portfolio that has an expected value of only 25.6 years.
Downside Risk-Adjusted Success (D-RAS)
Another paper the same year, “Replacing the Failure Rate: A Downside Risk Perspective”, set out to address problem #1. We shouldn’t treat upside and downside volatility the same.
The proposed fix is straightforward: instead of using the standard deviation of expected value, only use the standard deviation of shortfall years.
Here’s an example of how that change makes a difference: let’s imagine we have 4 retirements and they lasted 28, 29, 40, and 45 years. Our previous definition of standard deviation, of expected value, gave us a result of 7. But just looking at it, we can tell that most of the volatility was on the upside. Our new definition of standard deviation — which only cares about downside volatility — gives a result of just 1.06.
Armed with D-RAS, let’s go back and look at our US results.
Now that we’re ignoring upside volatility, D-RAS prefers the 80/20 portfolio over the 70/30 portfolio.
Overall, by not penalizing upside volatility, D-RAS selects much more aggressive portfolios than RAS. This seems like a great example of why it is so important to dig into the details of volatility and decide whether it really matters to you in your personal situation.
The next step came in the paper “Evaluating Retirement Strategies : A Utility-Based Approach” where Estrada teamed with Mark Kritzman for the first time. Their calculation is performed in two steps:
- The coverage ratio is defined as the ratio of the “years sustained” and the target length of retirement. If a portfolio last 28 years out of 30 then its coverage ratio would be 28/30 = 0.9333. If a portfolio lasted 43 out of 30 then its coverage ratio would be 43/30 = 1.43333
- Next apply a utility function. This discounts the value of outperformance and penalizes underperformance.
Here are a few examples of how the utility function changes things in practice:
# a portfolio that lasts exactly 30 years, as planned.
u(30 / 30) = 0.0# a portfolio that lasts 1 year extra only get a small bonus
u(31 / 30) = 0.0327
# the bonus DOESN'T increase linearly.
u(40 / 30) = 0.2877# a portfolio that falls short by 1 year has a large penalty
u(29 / 30) = -0.3334
# the penalty DOES continues to increase linearly
u(20 / 30) = -3.333
Utility functions are pretty common in economics. There’s a decreasing value of money, which you’ve probably noticed in your own life. The first $1 million is really nice. The second $1 million isn’t bad…but not as nice.
It also isn’t symmetric. Missing your rent by $1 is more bad than having $1 left over is good.
This is nice because it takes into account the final portfolio value, the chance of shortfall, and the amount of shortfall — all in a single number. We lose out on the measure of volatility that D-RAS was giving us.
The 100/0 allocation has the best coverage ratio in most circumstances. Estrada & Krizman write,
Our approach results in the selection of relatively aggressive strategies, with an average allocation of 91% to stocks and 9% to bonds. In over half of the markets, including the U.S. and the world market, the strategy selected is the most aggressive of those considered; 100% stocks. The most conservative strategy selected, in only two countries (Portugal and Sweden), is a portfolio with 60% in stocks
“Toward Determining the Optimal Investment Strategy for Retirement”
Finally we turn to their most recent paper, which came out December 2018/early 2019.
They take their Coverage Ratio and utility function and further explore a comment in the conclusion of their previous paper:
All of these metrics seem to favor 100/0 portfolios, or something nearly as heavily weighted. That seems in stark contrast to standard advice. Is this heavy tilt an artifact of the 30-year horizon chosen? After all, we know that (generally, though not always) over a period that long stocks outperform bonds. Are all of these metrics just a fancy, complicated way of saying that again (but maybe fooling ourselves that we’ve actually learned something new)?
First they run a Monte Carlo simulation to see how things fall out with different returns & volatility for stocks.
If you believe Bogle that equity returns will be 4% going forward, then this is evidence that a 60/40 portfolio is a better choice than a 100/0 portfolio.
These simulated results may provide better guidance for investors who believe that the historical record represents a relatively favorable pass through history that is unlikely to recur.
They also look at how things work out over shorter periods of time. Yes, retirement is (theoretically) a multi-decade affair. And, in the long run, stocks usually do better. But we also know that you shouldn’t “set & forget” your retirement. You should be re-evaluating it every year or so.
Here Estrada & Kritzman’s findings seem to support the conventional wisdom.
If we hold constant both the expected return and standard deviation, the optimal allocation to stocks falls as the retirement period becomes shorter.
They just…kind of leave that there without a deeper analysis, especially since it seems to be contrary to the results of their previous paper and, indeed, virtually all of Estrada’s previous papers, which argue for heavy-equity allocations throughout retirement and, in particular, against glide paths.
The Coverage Ratio is a nice addition to the toolbox. Because it takes into account the final portfolio value it adds an extra dimension to the evaluation. It isn’t really suitable for use with variable withdrawal schemes — it can really only be used for Constant Dollar withdrawals. In the same way the failure rate treats “success or failure” as binary without caring how much you succeeded or failed by, the Coverage Ratio behaves the same way about yearly income. If you withdraw $40,000 that’s success but withdrawing $39,999 is failure. And withdrawing $39,999 is treated the same as withdrawing $22,000…both are failures.
I do feel that Coverage Ratio, like MSWR itself, is mostly just a proxy for “stocks do well on average, so hold lots of stocks”. I’m not sure we’ve found a metric yet that can really disentangle that well.
D-RAS is interesting in that it directly incorporates volatility, which the Coverage Ratio doesn’t. Though Estrada & Kritzman argue that the utility function plays the same role:
The coverage ratio we introduced in this article is a simpler and more intuitive version of the years sustained variable, and the utility function we consider here accounts for risk as both RAS and D-RAS aim to do but further takes into account an investor’s attitude toward risk.
Estrada is one of the few people currently putting out regular interesting retirement research and he is also one of the few people who regularly makes use of the Dimson-Marsh-Staunton database of global returns in his papers. It’ll be interesting to see if his next paper does a deeper analysis & explanation of the apparent disconnect between this paper (“hold fewer equities as your horizon shortens”) and his previous ones.