This is a guest post by Dylan Tredrea, Head of Publishing at Zeptolab. Hit him up on LinkedIn or twitter and let him know how you liked the post!

Recently Apple decided to delay the privacy requirements of iOS 14 until ‘early next year’ presumably after the mobile game industry uproar became too much. Many developers, for understandable but misguided reasons, cheered this decision. While there are short term benefits of the delay, the period of transition from deterministic to probabilistic attribution will wreak havoc on studios’ ability to build and maintain sustainable businesses. Extending that period of instability makes the hardest and most vital part of the business- making new games- more difficult for a longer period of time. 

To be clear, this isn’t a judgement of Apple’s decision to implement these privacy measures. Apple’s approach to running a platform is by design, quite unique, and it certainly seems they share a lot of the same core values many game developers have about quality, creativity, and variety. There are also many aspects of the decision that go far beyond the game industry. There aren’t many trillion dollar companies eager to fight for user privacy, after all. Either way, as it seems this change is happening one way or another, it would be better for the game industry for this change to happen as quickly as possible- even at the expense of increased short term chaos.

Mobile games is soundly not a growth industry anymore. The sector sits decidedly at the top of the s-curve. The challenges of this fact are exacerbated in games where hits are ephemeral and growing a sustainable business requires the constant generation of new, successful products. Consistently releasing hits is always challenging, but in crowded, mature markets the challenge is particularly immense. 

Mobile game developers have developed tools and methods to mitigate the risks of a saturated market but they rely on a relatively stable user acquisition ecosystem. As the mobile ad market is in the process of a ‘burn down and burn up,’ the reliability of these tools and methods and how much risk they are really mitigating is sure to be deeply impacted. As a result, Apple’s decision to delay could very well do more damage than they intend and many developers currently realize. 

The industry’s reliance on paid user acquisition may ebb and flow, but it’s probably here to stay. Smartphones brought billions of new gamers into the market, but digitally speaking that was lifetimes ago. New technologies may provide spurts of growth, but nothing analogous to that tremendous growth and opportunity is happening anytime soon. As a result it’s easy to sympathize with what seems to be Apple’s intentions: a concern that the user acquisition ecosystem has become unbalanced, much like a poorly designed game, and dominant strategies (hypercasual & whale reliant games) have stifled diversity, creativity, and overall user enjoyment. 

Even if their intentions are the best, however, this is an incomplete, limited framing of the game industry and in particular of the game industry at the top of the s-curve. Limiting the ability for incumbents to dominate is helpful, but developers need to ensure- as much as possible- that the incredibly capital intensive work of game development is only being done on products with real audience potential.

And therein lies the rub of Apple’s decision to delay. When considered against the details of how game developers are currently managing the biggest and scariest risk- audience- the risks & costs of Apple’s decision to delay are clear. 

The industry has adapted rather well to a world of red oceans, all things considered, and that can be seen in how game pitches are becoming increasingly rare. These days most mobile game projects don’t kick off with copious market research, compex financial models, and slick presentations. Instead small teams, often completely at their own volition, start with a marketing test. They make just enough of a game- in a few weeks or even a few days- to be accepted by the app store. They create a few marketing videos and spend a few hundred bucks on Facebook to buy some users on iOS in the US and if they are able to show a very low CPI the team gets more resources and time to, well, actually make the game. 

This CPI marketing test process was pioneered by hypercasual games, but it’s becoming more and more common as leaders admit market research, financial models, and the judgement of leaders themselves are not realistically adding value. Though there is great value for the team in doing this ‘old school’ work to some degree, driving green light decisions solely on a stack of paper and the charisma of the creative director has been clearly demonstrated to be limited, if not dangerous. Especially at this stage of the industry lifecycle. There’s no way to show clear potential on paper without making something similar to what already exists... which is the easiest way to waste gobs of money at the top of the s-curve. 

So up until now, all well and good. Growth is hard to come by but the industry is filled with incredibly creative & clever people and ‘games found a way.’ Just as it was becoming virtually impossible to find hit games ‘on paper’ the CPI market test process saved the day. Well, to some degree. A great CPI test result is far from a silver bullet, but at least it significantly minimizes the risk of spending months making something that turns out DOA because customers can’t be acquired at anything close to a profitable cost. 

The industries increasing reliance on market testing cannot be understated nor can the testings reliance on the status quo remaining so. The keel of this industry keel is a test on Facebook, on iOS, in the US. That ad network, that platform, and that territory are chosen for good reason. Facebook’s efficacy, even with tiny budgets, along with their size and dominance in the US market paired with the fact that 70% of revenue comes from iOS users means the test is well structured to be as representative as possible of future potential. 

So much so, a game’s ‘Facebook, iOS, US CPI’ quickly became the industry standard for the early product lifecycle. ‘We have $X CPI, on Admob, on Android in Greece’ isn’t something anyone can really do anything with. There is no constant to extrapolate what the FB, iOS, US CPI will be with any actionable degree of precision or accuracy. Virtually every game in the early stages is presented and reviewed with ‘Facebook, iOS, US CPI’ in mind. No one, for good reason, will evaluate a project on anything else. Rather cleverly, the CPI test process flips the efficacy, stability, saturation, and scale of deterministic user acquisition on itself into a (bit of a) fastlane to success.

But now… it’s all gone up in smoke. The strength of that methodology only holds when what the Facebook/iOS ecosystem are doing today is representative of the future.  As Facebook takes a ‘controlled burn’ approach to their ad network the only sure thing is tomorrow will be very different from today and next week is likely to be wholly different from tomorrow as well. For Facebook this is likely a wise decision, but this means that until that transition is complete, the risk that the reliability of CPI tests has been greatly hampered is huge and even worse virtually unmeasurable. 

Exacerbating the situation is the fact that identifying, quantifying, and acting on how the transition is impacting the reliability of CPI tests is virtually impossible. CPI tests are already noisy indicators. The industry already well understands the steep funnel from great CPI test results to successful releases as retention, monetization, and scale can all easily fail. As much as developers desperately try to reverse engineer failures it’s rare that anyone is able to say with any professional honestly it was because of X and X alone. It’s an entertainment industry. Most things just don’t work out. As a result, as the industry goes through this transition, if the reliability of CPI tests is indeed impacted there’s no practical way to quantify or act on it. Studio strategy should react completely differently if the reliability is impacted by -10, -30, or -70%, but no one will know this number until it’s far too late. Even at larger studios who run enough CPI tests to find 10 great results, what does it mean if all of them fail to scale? (I.e., CPI starts low but increases too quickly as more users are bought).

Even today, with all the externalities and confounding factors of making entertainment in red oceans, how can anyone honestly say the failure is solely due to unseen, under the hood changes in an ad network? Especially as it’s unlikely for the shift to be clear cut. The ‘burn down’ and ‘burn up’ processes of the transition from deterministic to probabilistic will each likely have their own phases and ups & downs as Facebook has to re-learn how to manage attribution. 

As difficult it is to directly measure though, the logic is depressingly clear. The more a market is saturated and the higher up the s-curve the more the industry relies on early marketing tests to ensure expensive investments in full games are done wisely. Any change to the user acquisition ecosystem- especially one where a huge, public company wisely decides to ‘burn down’ and ‘burn up’ the core of their trillion dollar business- is sure to result in dramatic shifts in the reliability of those tests. As CPI tests become less reliable the resources sunk into doomed products increases and as a result so do job and studio losses. 

The depressing cherry on top of the cake of aquish is in such a dynamic environment not only are learnings incredibly difficult to collect they are also impossible to exploit. If a studio is able to definitively ID, somehow, before others a key facet of how the CPI testing process can be more reliable at some point of the transition, the probability that key facet will continue to be true in the next stage(s) of that transition is unknowable. And given the huge cost of acting on that intel, that uncertainty essentially makes the value of that intel (even if the developer knew it was 100% fact) moot as well. 

Every month until the UA ecosystem has stabilized will undoubtedly kill more and more studios. So while Apple has certainly given some breathing room to many of us about the revenue we’re generating today, the industry should know well enough that building a sustainable business in games is never about today. It’s all about running an operation that is optimized for what comes next.

The industry’s reliance on CPI testing is so strong it’s all but certain that no matter what happens there will be no choice but to continue to utilize the tool and hope for the best. Until some other option comes along (and is used to such a degree that it becomes the new standard), even testing in an unmeasurably more chaotic and unreliable UA ecosystem is still probably better than pretending the old fancy excel models provided shareholders and executives any real value. 

As a result studios large and small can only hope this transition happens as quickly as possible. Without a stable user acquisition ecosystem the sky high risks of game development at the top of the s-curve rocket into the stratosphere. Unless of course, putting on our tin foil hats, this disruption is very much the point and all of this is leading up to Apple releasing tools/capabilities for testing new games directly on the app store. Who knows if that’s a wise choice for Apple or a fit for their unique way of running a platform, but the one sure thing is the longer the industry goes with a less reliable method of testing new games the more studios will die off as increasing resources are poured into games with no real potential.  

More content like this: