A Case for Process

There is a tip I learned from a company mandated cybersecurity training. The subject of the lesson was how to reduce the chance you will fall victim to email phishing. The narrator claimed that an email written to evoke a sense of urgency is one of the many things to watch out for. The idea is that if the author is successful in making you think you need to take immediate action, you will do so without thinking. You may even do something that, under normal circumstances, you would not have.

I have used that technique in many contexts since then. So much so that my suspicion is reflexively heightened when I hear messaging that sounds frantic or heavy handed, especially when it is not obvious where the sense of urgency is coming from. That is the response I have to the discussion around large language models (LLM). Claims that all jobs will be lost or a monopoly on the technology is a matter of national security or even warnings reminiscent of Terminator robots and humans as fodder for The Matrix just seem…unserious.

As someone who remembers a time when personal computers were not ubiquitous, I am certain I have never experienced anything like the marketing juggernaut that is the push for “generative AI”. In my experience, new technology is introduced to the market in a manner that gives off an air of exclusivity. This time, it was just given away…essentially for free. I can think of a few other contexts where this has happened and it has not worked out well for the consumer, but I digress.

At this point, it has been around three and a half years since the introduction of a chatbot to the marketplace and the headlines are continuously rolling in. Though it is tempting to follow the many rabbit holes available, I want to focus on two major issues to highlight my point. First is the use case. A colleague once tried to equate an LLM to a car, claiming that a lack of knowledge around how combustion engines work does not prevent people from driving cars and therefore the same logic should apply to large language models. Here is why that simply does not work here. A vehicle is a fully realized product, developed for a single purpose as a conveyance and designed by individuals who understood exactly what the consumer would be using the product for. An LLM is a tool that was introduced to the market as a multi-purpose solution to an undefined problem.

Second is the transition to token-based billing. Essentially, large language models were rolled out under a subsidized subscription model that did not account for product costs. Companies were encouraged to adopt a technology under one pricing model and are now forced to reckon with the implications of another. Now, we are all subject to changes in market conditions, but there was evidence that the economics of the subscription model did not work way before the token-based model was introduced. With the implication that prices were going to have to increase greatly to justify the investment. One only needed to look for it.

I have no insider knowledge about how corporate leaders came to the conclusions they reached about their respective implementations of LLMs. However, based on the predictable issues they are having, it does not appear they opted for a process-oriented approach to their decisions, leaving the door open for the mounting costs I fear are just beginning to come to light.

All of this is happening in the context of for-profit enterprises, but the very public lesson is agnostic about that fact. Further, the industry has its sights set on the nonprofit sector, so the topic is relevant if, for no other reason, the hope is to avoid the missteps that are becoming clearer every day. When the question of whether or how resources are going to be allocated arises, it helps to have a process for making that decision. One that evolves in stages and can provide early signals about whether it makes sense to proceed or not.

A rigorous decision-making process requires the establishment of a set of defined objectives, a plan to achieve them, and an approach for measuring outcomes. With respect to the consideration of an LLM-based implementation, none of that is achievable without understanding what the technology is, how it functions, and what its limitations are. I am by no means an expert on any of those things, but I made it my business to learn as much as I could. I found computer scientists, software engineers, and investigative journalists who offered informed analyses of these tools and the environment that birthed them. Armed with that knowledge, I was highly skeptical about the use case for the kind of work that I do. The stories of “hallucinations” found in consulting reports and legal briefs support my initial conclusions.

Properly measuring outcomes with respect to the use of LLMs is where my doubts were always greatest. I kept hearing people talk about the efficiency gains obtained by pushing high-volume tasks to LLMs. And I kept wondering, how do you know it works? If you have turned a high-volume task over to an LLM with no feasible means to validate the output in a meaningful way, then you just have nicely packaged nonsense. I performed a test for myself by opting for a low-volume task for which I could easily develop an answer key. I was given access to a tool that was supposedly designed to summarize documents. I selected a single document, a piece of academic research, read it from beginning to end, and extracted the information I needed. I then developed a series of questions that I could ask the LLM to glean the same information. After a few hours of engaging with it, I understood it to be a futile exercise. The output was inconsistent, incomplete, unreliable, and unusable for my purposes.

The problem with surrendering key business functions to a technology you do not own and cannot feasibly control the costs of should be obvious, but I want to offer a few thoughts on associated costs that may not initially come to mind. When deciding to offload work to a technology, consideration should be given to the substance of that work. In essence, you could be shifting core functions to an outside vendor over which you exercise absolutely no control. Additionally, serious consideration should be given to the effects on talent. From the perspective of skill development alone, it is short-sighted to encourage the kind of cognitive off-loading associated with the use of LLMs for young professionals without thinking critically about how it will affect them in the long-term. Honestly, I am a little concerned about the effects on more seasoned professionals as well, but I will table that conversation for now.

All of this brings me back to where I started. Speed and sound decision-making are simply not compatible. Are there certain contexts where that is necessary? Certainly. Operating rooms. Battlefields. Avalanche. You get my point. Except for these and similar cases where the decision-maker is trained to react under pressure, decision absent deliberation should be avoided. There is no emergency associated with the adoption of new technology. If you hear something different, I suggest you proceed with caution. They just might be trying to get you to act without thinking.