Where we left off:
We looked at a bunch of aggregate stats & charts trying to answer two questions:
- Is OmegaNot any better than simple annual rebalancing?
- Is one particular variant of OmegaNot better than any other?
For #2 we didn’t see any clear evidence, so let’s set it aside for the moment. Going back to #1, is there is a point to OmegaNot at all, we didn’t see any clear answers. Some things pointed to yes. Some things pointed to no. A summary of comparison against high-equity annual rebalancing strategies (e.g. 80/20 or 90/10)
- Failure rates: tie
- Maximum Safe Withdrawal Rate (MSWR): tie
- Maximum Perpetual Withdrawal Rate: tie
- Shortfall years: OmegaNot
- Downside-Risk Adjusted Success: mixed, slightly favoring OmegaNot
- Coverage Ratio: Annual Rebalancing
- Ulcer Index: OmegaNot (though the advantage is slight)
- Certainty Equivalent Withdrawals (average): Annual Rebalancing
- Certainty Equivalent Withdrawals (minimum): mixed, slightly favoring OmegaNot
- Harvest-Rate Efficiency (average): OmegaNot (though the advantage is slight)
Given the mixed results above, it is hard to recommend one over the other: OmegaNot doesn’t look better but it also doesn’t look worse.
The problem with averages
However, I wasn’t entirely truthful with you. I expected that we wouldn’t see anything particularly compelling in the previous tests. We already knew that just looking at averages can, in the worst cases, be downright misleading. What we really want to look at are distributions of results, especially the left hand of the distribution (the less-good cases).
Let’s take another look at some of the stats but this time, instead of looking at averages, we’re going to get into the weeds, and see if that brings up any more clarity.
Unless otherwise specified: all portfolios are 60/40, a 40-year retirement is assumed, and 5% constant dollar withdrawals are used. Yes 5% is higher than “normal” but we want to see how the strategy behaves under stress, not when things are going perfectly.
(As a reminder: we are comparing OmegaNot with a 75/25 portfolio because, even though OmegaNot starts out at 60/40, we found that its average bond holdings were 75/25.)
Shortfall Years
Shortfall years was almost the only metric that showed a clear & not-just-marginal improvement: OmegaNot performed better. Shortfall years seems like a good proxy for what retirees care about, so the outperformance on this is intriguing. Maybe it is a sign that OmegaNot is better, if only we can find the right metric, or set of metrics, to highlight the improvements?
(Remember, a lower number is better, since it means fewer years without money.)
What we’ve done here is group everything into quantiles. What’s the 5th percentile result? That 10th percentile? Etc. This makes it easier to see how things do in the worst cases or the best cases.
In general, what we’d like to see is something like: the worse cases improve, the median cases stay around the same, and the best cases only degrade a little.
And that’s exactly what we see here. OmegaNot is a clear improvement from around the 60th percentile & up. If all we looked at was shortfall years, we’d call OmegaNot a clear winner.
To provide a bit of supporting evidence for the quantile chart, we also will compare the results during the 1965–1975 years, which included several very tough years to retire. Again, we see that OmegaNot improved things for these bad years and never made things worse.
Maximum Safe Withdrawal Rate (MSWR)
The evidence with MSWR is less conclusive — the improvements are very minimal — but overall the trend is positive:
Improvements in the worst cases (the left side of the quantile chart), the median holding steady, and the best cases seeing a degradation. And looking at the 1965–1975 chart reinforces that.
Coverage Ratio
The results from the Coverage Ratio are even less impressive than for MSWR but we still see minor improvements in the worst cases (everything from about 40th percentile downward).
Interestingly, there are large improvements…but only for the 60th and 65th percentiles. No idea what’s up with that anomaly.
The coverage ratio, like most utility functions, is a bit hard for me to intuitively interpret in left-tail scenarios. What’s the difference between -10.7 and -11.2 really like? Is that meaningful? Or not?
Still, the Coverage Ratio remains supportive of the case for OmegaNot.
Ulcer Index (Portfolio)
Remember that lower values are better for the Ulcer Index, that it is a bit like standard deviation.
It is less than clear what the relevance of the Ulcer Index is. After all, shouldn’t be willing to tolerate slightly larger drawdowns if that means we are able to support our standard of living longer?
So I think the real value of the Ulcer Index, when applied to portfolio values, is as a sanity check in a ceteris parabis kind of way. Are we drawing down portfolios too much for that stable income? If two strategies seem otherwise similar, does one of them generate shallower drawdowns?
The evidence is a bit mixed here: when we look at the quantiles, OmegaNot seems to have slightly bigger drawdowns for the lower quantiles. But when we look at the detailed yearly charts we see slight improvements for all years, especially in the 1970–1974 range.
Certainty Equivalent Withdrawals
Another set of charts in favor of OmegaNot. Improvements at the low end. Though the drop-off at the high end is a little concerning, but we remind ourselves that (looking at the y-axis) we’re still talking about $60,000-$70,000 a year.
I imagine that most retirees would be willing to make that tradeoff. Though, the improvement is quite modest, which is especially clear when we look at the yearly results. At the 5th percentile we see an improvement from $33,400 to $34,900. In 1969 we went from $32,200 to $34,900.
Still, a 4–5% improvement, an extra $100 a month, might mean “don’t have to cancel our cable TV subscription” or “don’t have to cut out my three-times-a-week Starbucks trip”.
Harvesting-Rate Efficiency (HREFF-4)
The Harvest-Rate Efficiency metric is attractive because it directly incorporates the notion of a spending floor. If we drop below $40,000, that’s when we really worry.
Unfortunately, HREFF-4 can’t really capture any differences here. That’s because so much of our spending is under $40,000 that the penalties and the utility function math just kinda smear everything together.
For the year 1969 annual rebalancing has an HREFF-4 of 0.0002 and OmegaNot has an HREFF-4 of 0.00136. OmegaNot is clearly better and by a factor of almost 10x. But everything is so close to 0 that, on an absolute instead of relative basis, they are indistinguishable.
Harvesting-Rate Efficiency (HREFF-3)
We can lower our floor to $30,000 in an attempt to create a bit more space for differences to be highlighted.
That does seem to help. At the least, we can distinguish them. (For 1969, the values are now 0.081 and 0.138.)
Like most utility functions, it is a bit hard to intuitively grasp. What does 0.138 mean anyway? How big of a difference is there really between 0.081 and 0.138. But even if we have a hard time mapping those scores back to reality, we can at least use them to help us rank order competing strategies.
Ulcer Index (Withdrawals)
We can take the same math underlying the Ulcer Index and apply it to withdrawals instead of portfolios. In theory, retirees should care way more about withdrawal draw downs — “I used to withdraw $52,000 a year but now I can only withdraw $33,000 a year” — than portfolio draw downs.
Because low numbers are better, the right hand side of the chart is the “bad” side.
This looks terrible for OmegaNot! In the two worst cases (the far right) it does noticeably worse. Yikes.
But if we look at the yearly numbers, it looks more promising. In the 1965–1969 years, when retirements were the worst, OmegaNot generally did better.
How do we explain the apparent differences? Our naive definition of Ulcer Index for Withdrawals has a fatal flaw: we are measuring the drawdown of withdrawals from this highest withdrawal ever seen. So going from $75,000 to $70,000 is counted as bad. In a way, this ends up punishing systems that give you high withdrawals — since you can drawdown further. Would you rather go from $70,000 to $50,000? Or from $55,000 to $50,000?
In order to make some sense out of this we’re going to need to tweak the calculations underlying the Ulcer index.
Ulcer Floors for Withdrawals
What are we really trying to measure here with our “Ulcer Index for Withdrawals”? We want to know how much stress due to withdrawals under our benchmark. In this case, we’ll treat $40,000 as our benchmark. We want to know how much stress is caused by low withdrawals, not medium withdrawals that happen to be lower than our previous, slightly higher, also medium withdrawals.
The change is a simple one. The original definition of the Ulcer Index is “the square root of the mean of the squared percentage drawdowns in value”. We instead want it to be the “square root of the mean of the squared percentage of shortfall below the benchmark”.
The original formula is:
SumSq = 0
MaxValue = 0
for T = 1 to NumOfPeriods do
if Value[T] > MaxValue then MaxValue = Value[T]
else SumSq = SumSq + sqr(100 * ((Value[T] / MaxValue) - 1))
UI = sqrt(SumSq / NumOfPeriods)We strip out all the references to MaxValue and just use the benchmark:
SumSq = 0
Benchmark = 40000
for T = 1 to NumOfPeriods do
if Value[T] < Benchmark then
SumSq = SumSq + sqr(100 * ((Value[T] / Benchmark) - 1))
UI = sqrt(SumSq / NumOfPeriods)
Since higher is worse, the right hand side of the quartile chart is the “bad” side. Our new, revamped version of the Ulcer Index for withdrawals, using a floor, looks much more like we expected.
OmegaNot improves the shortfalls from about the 75th percentile onward and if we look at 1969 we see an improvement from 0.202 to 0.169, a bit over 3%.
If we look in detail at the withdrawals for 1969 we can see what that 3% translates to. Here I’m only showing withdrawals that were under $40,000 — the only ones our new metric counts.
Recap
Once we stopped looking at averages and started looking at quantiles, the improvements from OmegaNot, at least in the worst 20–30% of cases, became clearer. It was clearest with the shortfall years, certainty-equivalent withdrawals, HREFF-3, and “Ulcer Floors for Withdrawals”.
So far we’ve just been looking at charts and, qualitatively, saying “yep, looks like OmegaNot did better on that one!”
We’ve also, at this point, become somewhat myopically focused on the worst 20–30% of cases. Everything in retirement is a tradeoff. How do we ensure that we’re not trading off too much of the upside for marginal improvements on the downside? Would you give up an upside of an extra $20,000 in order to improve the worst case scenario by $200? (Some people would.)
We ended the last installment saying there was no clear support for using OmegaNot. Now I think we can revise that. We see consistent evidence of improvements and no evidence of degradation in the lower 30% of scenarios. (That doesn’t mean we’ve established it never makes things worse in bad scenarios; only that our still somewhat aggregate metrics aren’t able to see it.) Sometimes the improvement may be small and sometimes we appear to perform worse in the top 30% of scenarios.
At this point, I think we can be cautiously optimistic about using OmegaNot, while remaining clear that it is not some kind of panacea: after all, for our 1969 cohort we still saw some withdrawals fall below $25,000.
