The rate of failure in quantitative finance is high, particularly so in financial machine learning. The few who succeed amass a large amount of assets and deliver consistently exceptional performance to their investors.
However, that is a rare outcome, for reasons explained in the book Advances in Financial Machine Learning by Marcos Lopez de Prado.
de Prado manages several multibillion-dollar funds for institutional investors using machine learning algorithms. Over the past 20 years, his work has
combined advanced mathematics with supercomputing technologies to deliver billions of dollars in net profits for investors and firms.
A proponent of research by collaboration, he has published with more than 30 leading academics, resulting in some of the most-read papers in finance.
Over the past two decades, de Prado has seen many faces come and go, firms started and shut down.
“In my experience, there is one critical mistake that underlies all those failures,” he writes.
Discretionary portfolio managers (PMs) make investment decisions that do not follow a particular theory or rationale (if there were one, they would be systematic PMs).
They consume raw news and analyses, but mostly rely on their judgment or intuition. They may rationalize those decisions based on some story, but there is always a story for every decision.
Because nobody fully understands the logic behind their bets, investment firms ask them to work independently from one another, in silos, to ensure diversification.
If you have ever attended a meeting of discretionary PMs, you probably noticed how long and aimless they can be. Each attendee seems obsessed
about one particular piece of anecdotal information and giant argumentative leaps are made without fact-based, empirical evidence.
This does not mean that discretionary PMs cannot be successful. On the contrary, a few of them are. The point is, they cannot naturally work as a team.
Bring 50 discretionary PMs together, and they will influence one another until eventually you are paying 50 salaries for the work of one. Thus it makes sense for them to work in silos so they interact as little as possible.
Wherever I have seen that formula applied to quantitative or ML projects, it has led to disaster. The boardroom’s mentality is, let us do with quants what has worked with discretionary PMs: let us hire 50 PhDs and demand that each of them produce an investment strategy within six months.
This approach always backfires, because each PhD will frantically search for investment opportunities and eventually settle for (1) a false positive that looks great in an overfit backtest or (2) standard factor investing, which is an overcrowded strategy with a low Sharpe ratio, but at least has academic support.
Both outcomes will disappoint the investment board, and the project will
be cancelled. Even if 5 of those PhDs identified a true discovery, the profits would not suffice to cover for the expenses of 50, so those 5 will relocate somewhere else, searching for a proper reward.