IB Boost Blog

It's now been 18 months since ChatGPT sent cold-chills through career-minded folk as the long-touted "just over the horizon" AI capabilities finally matured to the level of being able to provide generalised value and people every started to count back the years from their expected retirement age.

AI is everywhere in both the social conscience and modern working life and no doubt in many companies there is an AI-centric model of operation that is coming, but not yet here. What will it take to get there?

The most dramatic development in 2024 has not been further increases in the boundary of where LLM are - this may change later in the year with the rumoured OpenAI ChatGPT 5 (or at least 4.5) being released - but the most notable development this year has been a rapid levelling of the playing field and reduction of the lead OpenAI have with their models. On some benchmarks it has been surpassed by Meta's Llama 3 and by Anthropic's Clause 3.

Whether we can draw any conclusions from this development is unclear. It could mean we are approaching exponentially diminishing gains in terms of the benefit from longer, more costly training sessions. It could mean that to provide any improvement we're being time-bounded by duration of the training, meaning we're essentially seeing an artifact of the Wait calculation as much as it applies to model training. It could just mean OpenAI take all the air out of the room with their far superior entry into the popular psyche.

Yet, despite these advances in generalised capabilities, and uncertainties on whether these "Gain of Function" type behaviours are emergent capabilities that will scale exponential with model size and training depth, penetration into the white collar market has so far been speculative and fleeting.

Of course, the financial quarter waits for no executive and so there have been the predictable "right-sizing" exercises by corporations in that phase of belt-tightening but this has been much more in advance of an eventual destination rather than due to current capabilities. Still, the signs are certainly there that AI will become ubiquitous in all aspects of corporate life: embedded in your operating system, in your spreadsheets, slides and word processing documents, in your browser and, most relevantly to us, in the heart of all new future software development, and indeed most creative, pipelines.

In many cases the job functions that are most at risk from AI are those that have already been at risk from automation and outsourcing for decades. Scripted call-centre phone systems, chatbots, documentation, translation, and auto-completions, have all had more algorithmically-based solutions in place for many years, and not many people will shed tears over one "robot" who was eminently terrible at understanding one saying "Yes" over a phone line being replaced by another "robot" that could moonlight as a graphic novelist, given the chance.

Nevertheless, whilst AI adoption has increased in the periphery and some corporates have used that as casus belli against their flesh-bearing workforces there remain gaping holes betweeen isolated pockets of how AI (and LLMs in particular) is used in a workforce. This isn't unexpected, the nature of the hype cycle, and as OpenAI's Sam Altman himself likes to say, people overestimate what's likely in the shorter-term and under-estimate what's likely in the longer-term.

However what's noticeable and impacting a lot of adoption is the re-serverfication of workloads due to the processing- and memory-hungry nature of the AI models. Surely, for developers using GitHub Copilot with its integrated APIs and GitHub hosted AI processing, this is not a concern, but the majority of enterprise developers - yet alone non-developers - are simply not permitted to use such tools, and for quite understandable reasons given systematic, repeated, and evasive actions relating to copyright and intellectual property rights matters relating to the training.

Beyond that, for the same reason cloud adoption has lagged at least a decade from the hype in most companies, companies simply do not trust other companies with their crown jewels. For every "it was only possible because of the cloud" story, there are as many about the cost savings and improvements of moving away from the cloud.

Beyond the lock-in avoidance, the lack of control and true sovereignty over data stored on someone else's servers (and the commensurate cost for leasing)has been an issue and this applies even more greatly with AI due to the high resource needs.

In the past, any developer could run literally dozens of web servers or applications on their local machine or a server in the corner and it could satisfy many use cases up until larger scale. This also meant cloud adoption could be a gradual, phased approach at a certain point where scalability or reliability become material factors. This is more or less not possible for most corporate end-user computing devices as AI models - even with all the tricks like quantization of models and reduced parameter sets - are simply to large to run in a cost effective way on almost any laptop and even a large desktop footprint needs a GPU that probably costs as much as the whole rest of the PC to provide adequate performance.

Hardware is often refreshed in cycles taking years, often based around large, structural changes to workforce management in conjunction with major operating system releases (for those who work in technology that remember the first days one was allowed to use Windows XP or Windows 7 in a corporate setting this was the team mood). Budgets are often drawn up without reference to such paradigm shifts so I think it's unlikely that those bearing standard issue ThinkPads will be allocated full-tower machines with an RTX 4090 at double the price on the speculative need they might be about to do great things with local models.

For those in Legal and in charge of IT Security, this is a weight off their shoulders, in that requiring an API gateway to a well-bounded service to use these capabilities gives them a natural chokepoint to keep their lane clean. But in terms of getting most out of AI for the workforce it means the pre-conditions are having a well-integrated, centralised workload capability (whether run in-house or outsourced to OpenAI and the like).

There are great advancements being made in the realm of open-source and miniaturised models, and fantastic, fast-moving developer ecosystems around them, but, so far, the apparent emergent capabilities of larger models have meant that these end up being very niche, unreliable, or woefully slow (or all three). And the spectre of legal ramifications and untested copyright and licencing decisions provides a further barrier - it's far less risky to use OpenAI's APIs and try to sue them for any future deemed liability than it is for your force of developers to download models created by unknown authors from unknown training material.

CTOs may be looking forward to this world of remote processing but for many technologists the inability to work locally and experimentally may slow down development of the type of truly innovative use cases that will transform the workplace forever.

One other factor to consider is that existing platform vendors are not using AI as a facilitator but more as a moat. That is, AI is so far mostly being used - at least by incumbents - to keep users more IN their platforms. Bridging these moats is often a second-order effect. At IB Boost we focus on trying to automate disparate things, so whilst we're beavering away on the future, it remains to be seen how adoption will be outside of the forced intake due to monopolistic operating system add-ons.