Simplicity has been difficult to implement in modern life because it is against the spirit of a certain brand of people who seek sophistication so they can justify their profession.

NASSIM NICHOLAS TALEB

Layers upon layers

There are several basic ways you can go about building a portfolio.

1.     You can buy stocks directly.

2.     Or, if you decide you don’t have the skill, time, or temperament to do it yourself, you can hire a portfolio manager to do it for you.

3.      Or, if you decide that you’d prefer to spread your money around to different managers, you might end up with something that looks bit like this:

Now things are starting to get a bit unwieldy. Depending on how many managers you’ve got, and what kinds of different strategies they are employing, you may or may not actually be well diversified. If they’re all doing the same basic thing, and owning the same basic stocks, you might be better off with a single manager.

4.     And so now there’s some coordination required. You might find that you need someone on top to oversee the managers. And now the portfolio looks like this:

They go by many names—“managers of managers,” “virtual chief investment officers,” and too many more to name. But they all share one thing in common: none directly manage client assets. They all farm the job out to external managers.

For the sake of simplicity, let’s call them “allocators.” This is because they don’t actually invest your money, but rather “allocate” it out to specialist managers who they hire to manage your funds on their behalf.

The model comes from the institutional investment world, where boards and investment committees are filled with professionals who volunteer to help an institution oversee money invested on behalf of its ultimate beneficiaries—a pension or endowment fund, for example.

These board members, while concerned with investment performance, are really more worried about getting sued by donors and/or beneficiaries.

And so they hire a professional consultant to guide their decisions. It’s mostly a “Cover Your Ass” exercise. There is a theatrical element to these quarterly meetings. The consultant will explain the minutia of a particular manager’s performance over the prior three-month period, and a boardroom full of executives will nod along. The consultant looks for reasons to replace a manager or put them “on watch,” so as to give board members assurance that someone is closely monitoring the underperforming segments of the portfolio. The exercise repeats for each portfolio manager, and then repeats again each quarter.

Does the exercise actually contribute to investment returns? In a 2014 paper titled “Picking Winners? Investment Consultants’ Recommendations of Fund Managers,” Jenkinson, Jones, and Martinez conclude that they “find no evidence that these recommendations add value, suggesting that the search for winners, encouraged and guided by investment consultants, is fruitless.”

So… there’s that.

These boards tend to be full of successful individuals who also often happen to be wealthy, and so the industry began to offer the same model to wealthy families. The basic pitch is that they will help you hire the “best” managers in each asset class. And while it’s dressed up in a veil of exclusivity, it’s simply a close sibling to the stockbroker who builds you a portfolio of mutual funds.

They will recommend a bunch of different managers, diversifying among different styles. Let’s have a growth manager, a value manager, a large-cap, a small-cap. Oh, and add a mid-cap, maybe an emerging markets specialist…

And on it goes. At the end of the day your portfolio has become so diversified that it becomes just another index fund—but with much higher fees.

Experience tells us that portfolios built by allocators perform no better than lower cost portfolios. In some cases, they can do far worse. Why, then, are they so popular in the “high net worth” segment of the wealth management industry?

Wealthy people tend to have their preferences dictated by a system meant to milk them.

NASSIM NICHOLAS TALEB

One obvious problem that should jump out at you is that the allocator model sure has a lot of layers of people between you and your money.

All of those people are getting paid.

That’s a lot of juice being squeezed from your assets.

Is this the perfect business model?

Let’s go back to basics on the High Net Worth Industrial Complex. Imagine you own a profit-maximizing wealth management firm. You will have three financial goals:

·       Charge as much as possible

·       Retain the client for as long as possible

·       Keep your own costs as low as possible

The allocator model facilitates all three at the same time. 

The allocators will make every effort to slice and dice your portfolio into tiny specialty sub-portfolios—based on strategy, company size, and geographic allocation. If you want to, you can oversee an empire of an infinite number of specialist managers, each of which in turn owns tens or even hundreds of individual securities. 

The beauty of this arrangement, for the allocator, is that there is minimal accountability. The allocator will take credit for getting you into well-performing funds, but the blame for underperformance always rests with the crappy manager, not with the allocator who recommended them in the first place.

One reason some private clients and family offices prefer using allocators is because there’s a perceived safety in numbers. Owning an ultra-diversified portfolio feeds the perception that choosing money managers is a lower-risk endeavor than buying stocks directly. And while it’s true that it doesn’t demand the same detailed analysis as investing directly in a company’s shares, it does demand an entirely different type of analysis, which is perhaps more difficult.

Can you identify “the best” money managers?

Evaluating managers is tricky, even for professionals who do it for a living. Through my experience, I’ve come to believe it’s more difficult than evaluating stocks. Why? Because while public companies are required to report all relevant and material information to investors, the most critical information about money managers is generally not public. Compounding the matter, there is no central “exchange” where you can see all of the managers and compare them against one another.

Even the allocators—the ones who will tell you they can choose “the best” managers for you—only deal with a tiny subset of the universe of managers. This is because many of the best managers choose not to play the allocator game, and so end up completely off the radar to the majority of investors.

It really requires a sixth sense to figure out if a manager has the magic you’re looking for. And you have to meet hundreds of them over time to hone your ability to discriminate. If you are doing it yourself, or have a small staff, or—God forbid—someone working part time on this, the odds are stacked heavily against you. Inevitably you are going to end up investing with the managers who get in front of you first, or the ones your friends are already invested with. If you think these are likely to be “the best,” you’re fooling yourself.

It’s not as simple as it sounds

There are a few factors that make choosing managers quite tricky.

The first is identifying the drivers of past returns, and separating luck from skill.

If a firm has a really impressive track record, you need to dig into the portfolio to understand what has driven that return. It is possible that one decision can drive a fund’s outperformance over an extended period of time. I recall meeting with a manager who had a stellar record throughout the 1990s and into the early 2000s. This is an extraordinarily rare feat, as the managers who did well during the 1990s tech bubble are generally not the same ones who did well in the years after the crash, when value stocks led the market recovery.

When I asked the manager about their performance drivers, I didn’t receive a satisfactory answer. And so I asked to take a close look at the portfolio. And it turned out that among their traditional value holdings, the manager owned a single stock—Qualcomm—that drove the entire 1990s outperformance. The manager, whose office was in California, caught on to the potential of San Diego–based Qualcomm’s wireless patents early on, bought in, and let the position ride to become a huge part of the portfolio. Not only did they do a great job in buying Qualcomm, they did a great job in selling it—liquidating near the top and reallocating into their traditional value portfolio.

Let’s make no mistake—this is a great achievement by the manager, and they deserve full credit for buying that stock and managing that position extraordinarily well. But what are the odds that they will have another Qualcomm in their portfolio in the future? Removing Qualcomm from their results suddenly made a spectacular manager appear quite average.

I’d much rather see performance driven by consistent application of a firm’s stated strategy, even if it’s not as impressive as the numbers generated by this Qualcomm-savant manager.

Next, beware the superstar manager. In my experience, most stars eventually burn out. Passive investment advocates will tell you that it’s just as likely that an individual manager was lucky as it was that they were good. But it’s also true that even if they were particularly skilled, things can change. And let’s face it, in order to become a “star” you have to be exceptional, and that means taking big bets on assets and industries that inevitably mean-revert. I recall a long-term-successful manager telling me, “My goal is to be second quartile each year… the top-ranked managers need to take too many risks to get there.” (To clarify the investment-speak, the second quartile manager would rank in the 25 to 50 percent best out of the universe of managers.)

Success also brings its own challenges. It can be easier to manage a small portfolio than a large one, and so as a manager accumulates assets their strategy needs to adapt. It becomes harder to match past performance. As allocators herd client funds into the “best” performers, it becomes less likely that they will be able to match the performance that got them on the allocators’ radars in the first place. 

Over the years I’ve also heard stories about managers who lost their edge due to divorce or drug addiction. Both of these situations are more likely after experiencing a period of financial success.

There are countless ways that an individual’s personal life will impact their investment results. How capable are you of making these judgments? How plugged in are you to know what’s actually happening in their lives?

Another factor in the fall of many successful managers is hubris. Portfolio managers need to find a balance between having the confidence to believe they are correct while everyone else is wrong, and the humility to know when their analysis is faulty and they need to course correct. An extended period of success or acclaim can throw off that balance, overloading the confidence side of the scale and resulting in a portfolio full of excessive risk and one-sided bets.

And then there are the specialized strategies pursued by hedge funds and niche managers. It’s best to think of any strategy that relies heavily on manager skill and constant trading as a business in itself. That business is trading assets—stocks, commodities, currencies, and so on. Consider them as you would consider a direct investment in any other business. What processes are in place to ensure a consistent outcome? Who are the key people and what if something happens to them? And, most importantly—do you understand what they are doing to earn you a return on your money?

Very recently, a high profile Canadian hedge fund failed, wiping out the entirety of investor funds and leading to the tragic suicide of its founder. The details are grim and disturbing, and it illustrates a stark example of the failure of the allocator model. 

After you have considered all of that, let me ask you a question: Is researching an investment manager any simpler than researching a large public company? It sounds like there are a lot of variables at play, and they’re not clearly reported to you each quarter like the results of that blue chip stock you own. You definitely need to do a bit of detective work to maintain full confidence. Do you even want to go there?

Among the firms presenting themselves as allocators, I’ve never seen one consistently able to choose top performers. They promise to build you a portfolio of the “best” managers in each asset class, but given all of the variables I’ve just described, I wonder if you’re really just paying someone to over-diversify your portfolio for you.

The failure of the allocator model

Failure to identify outperforming managers in itself is not my biggest problem with the allocators, although I think they should be a lot more honest about their inability to do so. No, the greatest sin of the allocator is the extra cost involved in this management structure. You see, an allocator will have you paying two levels of fees—to the underlying managers, and to themselves at the top. I can virtually guarantee that they address this by telling you they negotiate lower fees with the managers you are allocating money to, and so the net cost to you is lower—not actually two full layers of fees. But that claim fails under any serious scrutiny, because if you are allocating to the “best” managers, what are the odds that those are the same managers who would be willing to cut their fees to collect assets from an allocator? Why would the “best” see the need to discount to attract assets?

Extra layers between you and your money are great for the High Net Worth Industrial Complex. Are allocators promising to add value to your wealth plan in other ways that might justify their fees? Or are they just squeezing as much juice as they can from your portfolio?

The allocators also benefit from less accountability. “No pressure to consistently outperform,” an allocator once confessed to me over a beer. “Clients are a lot more forgiving when you’re not the one actually managing the money. It’s easier to just move money around and get paid for it.”

Generally, for the investor, these arrangements work out just as you might expect they would. Some of your managers will do well, some won’t. Occasionally, your allocator will recommend replacing one manager with another one, or adding a new strategy (all in the pursuit of “uncorrelated returns”). There is an illusion of incrementally moving towards a “perfect” portfolio made up of the “best” managers, but you will never actually get there. 

So they’ve devised a great model that involves collecting fees on your money with no accountability for adding value. Amazingly, this industry has flourished in recent decades—a testament to the strength of a well-crafted sales pitch.