Many early observers believed financial markets were completely random and therefore unpredictable. Anyone attempting to forecast markets was then and often still is to this day regarded as an ignorant fool who doesn’t understand basic principles. But an examination of these principles reveals some interesting flaws which could open doors to potentially profitable opportunities.
Fig 1 below shows a line chart of daily closing prices for the S&P 500 stock index for the last 30 years. This type of chart, which will be familiar to most traders and investors, is known as a time series.
Fig 2 opposite, uses exactly the same data displayed as a type of bar graph. It shows both up and down percentage price moves arranged from left to right according to the size and direction of the move. The height of the column indicates the number of moves for each size.
This distribution chart of price moves or histogram as it is more formally known is just a different way of looking at the same data. In general, it can be very useful for those trying to learn how to predict markets to get used to looking at familiar things in different ways.
Fig 1 Time series chart for S&P 500 stock index
Fig 2 Distribution histogram for S&P 500 stock index
Looking at the Fig 1 time series chart, it would seem that anyone who bought, maintained and held onto a reasonable portfolio of S&P 500 stocks 30 years ago, would have increased their original investment about 10 times. This looks easy enough to do – unfortunately we can all be wise and rich in hindsight – but the distribution graph on the right gives us some insight into why it is more difficult than it first seems.
Drawing your attention to the red vertical line on the Fig 2 histogram, you can see that it is very close to zero. The green bars appear to be distributed quite symmetrical around the vertical zero axis which simply means that the total of all the daily up moves for this 30 year period on the right of the zero axis are approximately the same as all the daily down moves on the left.
In other words, most of the market movement over the last 30 years has been up and down without actually going anywhere. The total up movement that we see in the Fig 1 chart accounts for only about 5% of the total movement. Our 30 year S&P 500 market is therefore working very hard for a very small gain. The other 95% of “useless” movement is one of the things that make it very difficult to predict markets.
We normally call this ‘going nowhere” movement volatility although I must point out that as with most things in the financial markets, there are many different ideas about what volatility is as well as various ways of measuring it. These different methods of measurement also often give significantly different results.
At the top of the graph in fig 2, we have an area marked “Thin Peaks”. This represents many smaller than average moves. There are far more of them than we would have expected to occur by chance. At the bottom of this chart are two other areas marked “Fat Tails”. It is difficult to see much on the fig 2 chart, but if we look at the next chart, which is a zoomed in view, they become quite obvious.
Fig 3 Zoomed in distribution histogram for S&P 500 stock index showing fat tail details.
These Fat Tails represent movements that are much larger than we would expect by chance and while they do not occur all that often, their daily movement of 5% or more will often raise an alarm. If 2 or 3 of them occur together over a short space of time, which can and does happen, it could invoke strong waves of panic or perhaps euphoria in the market, depending on the direction of the moves.
In 2001 a professional trader, Nassim Taleb, published a book entitled Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets. The book captured the public imagination and these days everyone talks about fat tails and black swans which are just another name for the same thing based on a related concept described by a British philosopher David Hume.
Taleb promoted the idea that people in general have a very poor understanding or awareness of the effects of randomness. According to him, even those formally trained in statistics are easily fooled into making poor decisions under conditions of uncertainty, often seeing things that aren’t really there and looking for explanations when there are none.
While Taleb dealt at length with the Fat Tail events that are commonly found in financial markets he barely mentioned Thin Peaks. As we will see in due course, these also relate to an important part of market behaviour and provide an often used but poorly understood mechanism for market prediction and trading strategies.
The next important thing that we should note on the Fig 2 distribution chart is the blue normal (random) distribution curve. It has been carefully constructed from random moves to have the same characteristics as the distribution for the S&P500 stock index. We can easily see, that the two distributions have significantly different shapes. If the S&P index was really just a collection of random moves then we would expect to see the blue normal distribution curve more closely matching the shape of the histogram distribution for the index. Below in Fig 4 and 5 is a simulated time series and distribution made up entirely from random moves. It has overall characteristics similar to the S&P500 index . As you can see, the match between the random distribution curve and the histogram are not perfect, but it is a much closer fit than the one for the real index shown in fig 2.
Fig 4 Simulated random time series S&P 500 stock index
Fig 5 Simulated random distribution for S&P 500 stock index
Fig 4 shows what looks like a nice trend, but it is only a random walk – an illusion. Attaining profits like this in a truly random market would simply be a matter of good luck! Can you spot any difference in the movements between the charts of fig 1 and fig 4?
The characteristics of all liquid financial markets tend to follow a similar pattern of thin peaks and fat tails. While these patterns may very a bit between different markets, the overall characteristics remain surprisingly constant, even for widely varying time frames ranging from minutes to months. This property is known as fractalization. If you don’t know what this means, you might want to look it up somewhere like on Wikipedia.
There is just one final point that I would like to add. In order for a distribution of price moves to be truly random, each individual move must be totally independent of any previous moves. When we toss a coin or throw a dice we say that the coin or dice has no memory of any previous result which is obviously the case but can we really say this about price moves in the financial markets?
In my opinion, it is pushing the bounds of reason to say that a significant percentage of traders or investors have no idea about previous prices when they buy or sell in the market.
Price movements in financial markets don’t obey the mathematical laws required for randomness. Therefore, it is incorrect to say that markets are unpredictable because of this. All that we can say is that markets behave in a somewhat random manner and because price moves are heavily encumbered with this random like behaviour that we normally call volatility, they can be very challenging to predict.
But why do markets behave this way? To uncover some more of these secrets we need to start looking at human behaviour which we will do in the next article.
This background information that we have been looking at is vitally important. Before we can hope to predict markets we must first understand how they work, so we will be returning to these basic ideas time and again. I encourage you to study them further until you are comfortably familiar with them and have them firmly imprinted into your brain.
It would appear that some of the financial market theorists that we have looked at, as well as many contemporary thinkers, traders and investors have been and continue to be fooled by randomness.
I hope to show how you can avoid this enigma and be able to consistently and successfully forecast any market. Mastering these skills however takes time and effort.