In order to solve any problem we must first understand it. This article identifies two key problems that make successful trading and investing difficult.
It is not easy to be a consistently profitable in the financial markets. Many try, but for various reasons, few succeed.
As discussed in previous articles, there are those who believe that the financial markets are unpredictable. If this is the case, then by definition, it must also be impossible to define and maintain an edge or statistical advantage and without an edge, any outcome, favorable or otherwise can only be a matter of luck.
I do not accept the claim that financial markets are totally unpredictable. I do believe the following…

Predicting markets is difficult.

The probabilities offered by various indicators and methods of prediction is usually very weak and so close to random (p=0.5 or 50%) that only through careful measurement can we determine any predictability at all.

Predictability varies significantly over different markets and time frames.

Some markets are harder to predict than others.

The predictability of any given market changes over time.
Why forecasting financial markets is so difficult
Fig 1 monthly retail sales over a 10 year period
Consider the above graph of Fig 1 which represents recorded monthly sales of a retail organisation over a 10 year period. We can quite easily see that there is a modest upwards trend together with a strong but consistent seasonal variation each year.
Given data like this it would be fairly easy to predict the monthly sales figures over the next 12 months with a reasonably high degree of confidence.
It would be more difficult trying to predict the sales figures 10 years ahead. We could attempt to do so but we would need to lower our level of confidence significantly. A lot of things can change in 10 years.
The general principle is that the shorter our forecast period, the more accurate our forecast is likely to be. This however, is not an endorsement of short term trading. There are many other issues that need to be considered when choosing a trading time frame.
Fig 2 monthly retail sales over a 10 year period with modest noise
The above graph of Fig 2 contains exactly the same underlying data as the one shown in Fig1 but this time we have added some noise, or randomness to it. We can still see the underlying pattern and trend, but any forecast made from the above is likely to be significantly less accurate then the first one.
Fig 3 monthly retail sales over a 10 year period with high level of noise
The graph shown in Fig 3 still contains the same basic pattern and trend as the one in Fig 1 but this time so much noise has been added that it is just about impossible to see any underlying pattern or trend. It is also almost impossible to see that this price series, just like the first one, is actually up trending.
Using some sophisticated mathematical tools we could extract the original predictable signal shown in Fig 1 from the noisy composite information in Fig 3 except for one additional problem. In real markets, that nice steady annual signal that we saw in Fig 1 is not nice and steady. It is constantly changing. It simply doesn’t stay still long enough for us to measure it with any degree of accuracy.
Cognitive Biases
The above explanation in itself explains why it is so difficult to consistently profit in financial markets. What makes it even more difficult, however, is the effects of our untrained instincts in the form of subconscious beliefs and patterns known as cognitive biases.
My own research suggests that not only are many experienced people unable to predict the direction of price movement with any degree of accuracy or consistency but some develop a consistent negative edge as they gain experience.
In a controlled experiment where the only variable was the participants ability to estimate market direction over a short period of time we found that on average people only got it correct about 39% of the time. By contrast, when choosing the direction of a trade simply by tossing a coin, they got it right about 50% of the time, which is what you would expect.
We could easily jump to the conclusion that a simple technique to take advantage of this anomaly would be to make a prediction and then reverse it, thereby obtaining a significant edge. Unfortunately this tactic doesn’t work. The subconscious mind appears to know that you are trying to trick it and it just flips the results so you are right back where you started.
The comment has often been made that a blindfolded monkey throwing darts at a stock list can do a better job of picking stocks than many professional experts. Unfortunately it seems that this is true.
This interesting phenomena does prove however that financial markets are predictable to some extent. If they weren’t, it would be impossible to consistently hold a negative edge as some people are able to do. No matter what we did we would always finish up with profits approximately equal to losses after making an adjustment for transaction costs.
One of the tricks of the trade is to learn how to pick markets, time frames and situations that are the easiest to read. A typical example of a high probability signal would be a strong steady trend with a low volatility.
If we wait patiently for easy opportunities and simply ignore the rest we will probably have quite a few less trades but overall our result are likely to be a lot better.