Friday, October 18, 2013

Fama vs Shiller on the 1987 stock market crash


Tomorrow marks the twenty-sixth anniversary of the 1987 stock market crash. On October 19, 1987 the Dow Jones Industrial Average fell 22.6%, the largest one-day decline in stock market history. The best explanation for the decline, and the least well-known one, was put forth by economist Robert Shiller. This post gives a quick rundown of Shiller's work on understanding crash phenomena, in particular the famous 1987 event.

Eugene Fama, who along with Shiller and Lars Hansen shared the Nobel Prize this week, had very different reaction to the event than Shiller. In an essay penned not long after the crash, Fama, a true believer in the efficient market hypothesis, did his best to square the event with theory. The crash, wrote Fama,
has the look of an adjustment to a change in fundamental values. In this view, the market moved with breathtaking quickness to its new equilibrium, and its performance during this period of hyperactive trading is to be applauded. [Perspectives on October 1987, or What Did We Learn From the Crash? 1988]
Fama's effort to justify the crash as a rational response to economic news falls flat. A 22.6% decline requires something cataclysmic, but no significant events preceded the crash. Sure, there was a skirmish in the Persian Gulf with an Iranian oil station, a new tax proposal in the House, and a sell signal from guru Robert Prechter, but none of these events were capable of moving markets more than a few points.

Robert Shiller, on the other hand, gathered data. The day after the crash, he sent out questionnaires to hundreds of investors. Among other questions, Shiller asked: "Which of the following best describes your theory about the decline: a theory about investor psychology, or a theory about fundamentals such as profits or interest rates?" 67.5% of individual investors and 64% of institutional investors said the crash was about market psychology. When Shiller asked what major news stories people in his survey were reacting to during the day of the crash, the most popular stories were those about past price declines themselves, not fundamental news. Noted Shiller:
It would thus be wrong to say, as many have done, that the market drop on October 19, 1987 ought to be interpreted as a statement of public opinion about some fundamental economic factor, e.g., that there is a lack of confidence in the White House or Congress. At best, any such opinions probably played a role in the crash mainly as they affected the vague intuitive assessments people under great stress made about the tendency of prices to continue or reverse, or about how other investors will react to the current situation.
Put differently, the crash was a purely psychological phenomena.  When it comes to explaining the 1987 stock market panic, Fama and Shiller couldn't have been further apart.

 -----

Let me take this post on a personal tangent and then I'll circle back to Shiller. I first got interested in the 1987 crash back in the late 1990s when I was a student. Fearing that equity markets were getting overextended, I started to mine 1980s price data for clues about what might happen. I discovered that the visual overlay of movements in market indexes in the late 90s was eerily similar to that of the 80s. In October 1999 I went short, sure that we were on the verge of repeating the 1987 crash. At first the markets moved a bit lower. But a week or two later prices found their footing. I didn't know it then, but the bull move that followed would be the last spurt higher before tech mania would be pricked in early 2000. Unable to stomach the losses, I covered my shorts and went back to my studies.

Though I lost money in the debacle, I did gain what I thought was an interesting idea. If enough traders like myself drew analogies to a historical crash, our combined trades -- executed on the same day -- might result in the self-realization of that crash, even though nothing had fundamentally changed about the economy. This idea jived with an observation that many market watchers had made about the 1987 crash: it was eerily similar to the 1929 crash. Wrote George Soros:
Technically, the crash of 1987 bears an uncanny resemblance to the crash of 1929. The shape and extent of the decline and even the day-to-day movements of stock prices track very closely. -The Alchemy of Finance
Both crashes were preceded by multi-year bull markets. They each occurred on a Monday near the end of October, the first crash hitting 55 days after its bull market peak, the second 54 days. In addition to similar timing, the breadth of their declines were almost the same. The 1929 crash resulted in a 23% fall over two days, the 1987 in a 22.6% fall. I append a chart below:


Could it be that the 1987 crash occurred because traders were using the same backward-looking strategy I had when I went short in 1999? The process might have worked something like this, I reasoned: the peaks and troughs in 1987 began to randomly align with those in 1929. Backward-looking traders began to notice this alignment. A feedback loop may have emerged in which scattered fears of a recurrence of 1929 resulted in trades that pushed prices down, in turn rendering the analogy between the two periods ever more clear. A final trigger, say an anniversary date, might have been sufficient to complete the loop, resulting in a realization of the 1929 crash in 1987.

Reading through accounts of the 1987 crash, I found ample evidence of traders basing their strategies on 1929 analog models. In a famous but hard-to-come-by documentary filmed prior to the crash, 1980s wunderkind Paul Tudor Jones explained how he was using a 1929 analog model developed by his research director Peter Borish to put on a large short position in October 1987. The documentary is here, for now at least.* The Friday before the crash, hedge fund giant George Soros received a copy of Tudor Jones's study and showed it to Stanley Druckenmiller, manager of Soros's famous Quantum Fund.** On the morning of the crash, the Wall Street Journal published a chart of stock price in 1987 superimposed on stock prices leading up to the crash of 1929. News of the analog was spreading across Wall Street, and by Monday, October 19, enough momentum may have built for the analog to self-realize itself.

Paul Tudor Jones circa 1987

The 1929-87 event taught me that investor's minds don't react passively to underlying fundamental phenomena. Investors create stories that, when acted upon by enough people, actually shape the fundamentals. In 1987, a psychological "worm-hole" linked to an event fifty-eight years prior seems to have emerged, leading to the greatest one-day drop in market history. It was a mistake, a mental glitch, or a wrinkle in time.

 -----

Back to Shiller. I later found out that all of this had been anticipated by Shiller long before I was even old enough to buy and sell stock. In his post-1987 survey, Shiller found that 35% of individual investors and 53.2% of institutional investors reported talking of events of 1929 on the few days before October 19, 1987. Memories of 1929 were therefore "integral" in creating the 1987 crash, wrote Shiller:
Investors had expectations before the 1987 crash that something like a 1929 crash was a possibility, and comparisons with 1929 were an integral part of the phenomenon. It would be wrong to think that the crash could be understood without reference to the expectations engendered by this historical comparison. In a sense many people were playing out an event again that they knew well.
Nor was this the end to Shiller's work on crashes. The memory-of-crashes effect would reappear two years later. On Friday, October 13, 1989, a mini-crash occurred, the Dow falling 6.9%. Once again Shiller sent out a questionnaire. The most likely reason for the mini-crash, wrote Shiller, was the fact that the coming Monday was to have been the second anniversary of the 1987 crash.*** The mental image of the two biggest crashes in history possibly happening that Monday would have been sufficient to amplify any random price decline into an all-out panic. Wrote Shiller:
It may be a silly notion, but silly thoughts may have come to the minds of people trying to decide whether to sell as prices plummeted in the last hour of trading. They did not then have all of the reassuring commentary that came later, and they had to act then or risk having to sell on the following Monday. - Fear of the Crash Caused the Crash, NYT, 1989
 -----

In sum, Shiller long ago provided the world with what is probably the best explanation for why the 1987 crash happened when it did, and why it fell so far. Because Fama was so closely wedded to the EMH, his only option was to stay mute on the causes. "What caused this shift in expectations? I do not know" he wrote. Fama gives us a good-enough framework for understanding 99% of market moves. But for the remaining 1%, we really do need Shiller.



* The documentary has an interesting history. See Ritholtz, the WSJ, and Business Insider, among others. Apparently Tudor Jones threatens to sue anyone who puts it up, so getting ones hands on it is challenging. While the documentary is the best place to learn about the 1929 analog model, it also appears in the first edition of Jack Schwager's Market Wizards. But do try to watch the video, it's quite fascinating in its own right.


** Said Druckenmiller: "That Friday after the close, I happened to speak to Soros. He said that he had a study done by Paul Tudor Jones that he wanted to show me. I went over to his office, and he pulled out this analysis that Paul had done about a month or two earlier. The study demonstrated the historical tendency for the stock market to accelerate on the downside whenever an upward-sloping parabolic curve had been broken – as had recently occurred. The analysis also illustrated the extremely close correlation in the price action between the 1987 stock market and the 1929 stock market, with the implicit conclusion that we were now at the brink of a collapse. I was sick to my stomach when I went home that evening. I realized that I had blown it and that the market was about to crash." - Market Wizards, Jack Schwager (1988)

*** The 1929-87 analog revisited markets once again in 1997. On Monday, October 29, 1997, the Dow went into a freefall, eventually tumbling 7% . See my explanation of the 1987-1997 analog here

PS: If market's plunge this coming Monday, you know why.  ;) 

22 comments:

  1. The crash also demonstrated that rationality is highly subjective and temporal. In Medieval times it made sense that the earth was at the center of the universe. In the 70-80s, technical analysis had a huge cult following. Major banks had whole departments doing technical analysis and R&D. Today, headlines of a coming crash due to the Hindenburg omen has a more muted effect (e.g. just read the daily Marketwatch headlines!).

    ReplyDelete
    Replies
    1. Hindenberg omen? Never heard of it till now -- I'll be sure to add that to my list.

      Delete
  2. J M Keyes 'beauty contest' explanation of changes in stock prices remains valid today and explains very well1987 as well as 1929. Fama has a few modifications in his EMH to be made.

    ReplyDelete
  3. Great post. You were in good company in 1999: Druckenmiller also took heavy losses from short positions in tech stocks at around the time, reversed course and went long, made up all the lost ground and then some, but held on too long and got burned again a few months later:

    http://rajivsethi.blogspot.se/2010/01/on-efficient-markets-and-cognitive.html

    The timing risk and unbounded loss from short positions is what makes bubbles possible.

    Regarding the 1987 Crash, Fama did see it as a "correction" but I think he also conceded (in the same paper you cited) that "the long, orderly upside of the swing preceding October 19" was probably a prolonged departure from fundamentals.

    ReplyDelete
    Replies
    1. Thanks Rajiv,

      Your link brings back a lot of memories. It was hard to stomach the October to March 2000 rally. I simply stopped watching markets.

      In his 1987 paper, Fama also invokes Mandelbrot. Fama says something like... rational forecasts of economic conditions depend on past conditions, specifically the longer a period of good times, the longer it is rational to forecast that it will continue. Stock prices rise above observed earnings. Information that times have simply changed from good to average produces a large price decline. So a large change needn't require important news.

      But that doesn't seem like a very EMHish thing for Fama to say. If I remember correctly, the EMH crowd broke from Mandelbrot long before.

      Delete
    2. @JP ... after the debacle of technical analysis (no major bank has a TA department, and hardly any "strategist"/CIO under 50 mentions "head and shoulders"), the two surviving strategies from that era are: trend following and mean reversion strategies. As you noted, trend following does have a sound psychological basis (anchoring) and perhaps also a statistical one (i.e. absent a model, and assuming the process is not white, what's the best estimate at t+dt given t?).. although the best estimates are usually wrong (i.e. this year's Nobels).

      Delete
    3. Is technical analysis really dead? On the sell-side, you could have a point, I wouldn't know. On the buy-side, I'd argue that TA is still alive and well -- except it's being conducted by algorithms crawling through price and volume data for patterns, not nerds hunched over charts.

      Delete
    4. Yah, I would say it's dead except for trend following ("managed futures") and mean reversion strategies. It's been decades since "wealth managers" (formerly full-service brokers; now selling alternative investments or managed accounts) tried to sell me on a product where "head and shoulders" was mentioned. E.g. if you look at JPM (www.jpmorganfunds.com) some of their specialty funds fall under this category. Just as a joke I googled H&S at JPM's site ... zero hits (but I now have a solution to my dandruff problem!). I would guess most computer trading are: trend following, statistical arbitrage/mean reversion strategies. Having said that I think most of us consciously or not use basic TA techniques as part of the fuzzy overall picture, but the day trading golden years showed most of us were not Soros or Tudor Jones, or even a Turtle on a bad day.

      Delete
    5. "If I remember correctly, the EMH crowd broke from Mandelbrot long before."
      Could you elaborate on that? I know Fama studied under Mandelbrot and dedicated part of his dissertation to the fat-tailed returns of stock markets. Gene Callahan contrasted Fama with Shiller recently, assuming that fat-tails were incompatible with the EMH, unaware of what Fama himself had to say about fat-tails.

      Delete
    6. I got that from reading Justin Fox's Myth of the Rational Market.

      http://www.byjustinfox.com/2010/10/benoit-mandelbrot-and-finance.html

      Delete
    7. Every major bank has a "TA Department," as you call it, and usually dedicated technicians on every desk across asset classes.

      - Buy Side Analyst/Trader

      Delete
    8. Every major bank has a "TA Department," as you call it, and usually dedicated technicians on every desk across asset classes.

      - Buy Side Analyst/Trader

      Delete
  4. JP,
    I think the formulation "normal times" is problematic, especially for complexity theorists. Agents interact using the same rules and incentives all the time. Sometimes stability emerges from that interaction, and sometimes instability. Complexity describes how these "phase shifts" happen, through non-linear dynamics such as feedback loops, herding, etc. Fama, I think, has no place for such behavior.

    The question is, if Fama's description of the world fits the data "most of the time", then is it useful? I think the answer is "no". By mis-characterizing the emergent behavior of investor interaction, Fama exposes risk managers to catastrophic events, thus rendering the economy more fragile. In effect, Fama's thesis incorrectly paints the world as one containing mostly risk and little uncertainty. Five years after the crisis, this is still the dominant view.

    ReplyDelete
    Replies
    1. "The question is, if Fama's description of the world fits the data "most of the time", then is it useful? I think the answer is "no". "

      I'd argue that Fama's description of the world is good enough for "most people most of the time". For the retail crowd, it's probably better to buy a low-cost index fund than try to pick a high cost manager who claims to be a winner. I don't think Fama's description of the world is good enough for the small minority of academics or fund managers who are very focused on markets.

      Delete
  5. JP,
    An important distinction, but I disagree on your premise. Most people should hedge against uncertainty. A whole industry arose to promote the idea that those hedges are unnecessary. Fifty years ago the average saver would not think to "play the market". Now, the average saver counts on 8% returns "over the long term". How much did Fama contribute to that transformation, and has it made us more stable as a society?

    ReplyDelete
    Replies
    1. That could be. However, bringing this back to the topic of the crash, if everyone took Fama seriously and avoided backward-looking technical strategies, then the 1987 event would never have happened.

      Delete
  6. Scott Sumner often likes to talk about the 1987 crash. He also asks why there's little talk of "anti-bubbles" among believers in bubbles. 1987 might be the clearest argument for the existence of "anti-bubbles". If we take Friedman's "plucking model" seriously, should we expect anti-bubbles to be more common than bubbles?

    ReplyDelete
  7. Fama & French more recently discussed the relevance of Mandelbrot & fat tails here.

    I used the term "anti-bubble" analogous to anti-matter, but searching Scott's blog I see that he used the term "negative bubble". If "irrational exuberance" results in prices being above where they "ought" to be, and so they must inevitably be in for a correction, a "negative bubble" is the opposite. A priori there seems no more reason to expect bubbles than "negative bubbles" but we hear much more about the former.

    ReplyDelete
    Replies
    1. The 1987 crash does seem like a good example of an anti-bubble. I don't think that anti-bubbles are more prevalent than bubbles. However, I agree that we do seem to hear about bubbles more in the press than anti-bubbles. Not sure why.

      Bubbles tend to burble along for long periods of time before accelerating, and only then do they suddenly pop. This gives journalists plenty of time to orient on them. Anti-bubbles tend to begin with a sharp acceleration down followed by a steady rise back to fundamentals. All journalistic attention is focused on the acceleration phase, and that attention dissipates very quickly once the boring uphill trek to normality begins. If bubbles began with sudden accelerations, stopped, and then slowly returned to fundamentals -- and if anti-bubbles slowly trended downward for long periods of time before the final death-spiral, then maybe we'd hear more about anti-bubbles than bubbles.

      Delete
  8. Free Commodity tips for today
    CALL: SELL COPPER (APR.) BELOW 413.50 TG-412.00/410.00/407.50 SL-416.00

    Free intraday tips on mobile

    ReplyDelete