SmartBots: Expert systems, reborn
How RAG is giving expert systems another shot at success
Do you recall a time when AI meant expert systems?
(You might be concerned about revealing your vintage by admitting to knowing about them, so I’ll understand if you say you don’t).
Expert Systems were designed to mimic the decisions of experts in particular domains. They consisted of a “knowledge base” — a set of facts that experts knew, and an “inference engine”- logical rules- which coupled together was supposed to reproduce expert level decisions.
They had some success in medicine and law, but ultimately ran into three major sorts of problems. First, expert knowledge is often not articulable by the experts themselves. As Michael Polanyi famously said, “we know more than we can tell”. Second, expert reasoning is not always reducible to standard logical operations; we now know that associative reasoning and pattern recognition play a large role in the way experts think. Third, (novice) users often asked questions in a language that differed from the way experts thought and talked, so the expert system often faced a formidable translation problem.
By the end of the last millennium, expert systems were no longer in the mainstream of computer science. So why think about them again?
From RAG’s to riches
RAG (Retrieval Augmented Generation) has emerged as a powerful and inexpensive technique to customize Large Language Models (LLMs). Basically, we can take documents and integrate them with the responses from a foundation model (e.g. gpt4 or claude3.5) to user queries. The results become a combination of the knowledge used to train the foundation model and that in the documents used for customization. It beats retraining or finetuning the models itself in terms of cost and flexibility. Think about a consulting company hiring a fresh MBA, and either giving them a few pages to read or sending them to a training program before meeting a new client. The first is cheaper and becomes more effective relative to the second the smarter the MBA is. That’s what is happening with advances in LLM’s.
RAG is also a technology that has become widely available now through “custom GPT’s” (Open AI), “projects” (Claude), “agents” (MS Co-pilot) and ”gems” (Gemini).
We could set up a RAG model to use an underlying LLM to query what is in the documents we upload, or alternatively to apply the knowledge in the documents in a user friendly and flexible manner. An example of the first kind would be a law firm that uses RAG to help pore through their archives of case records. An example of the second would be to build a legal assistant that can help give basic legal advice. The second is basically an expert system.
SmartBot architectures
Unlike generic AI chatbots, one that is enhanced with a layer of domain specific expertise and context specific information can be smarter in terms of solving the user’s specific problems. That’s why I call them “SmartBots”. You can find a suite of them here.
A SmartBot is defined by its knowledge layer(s), not the foundation model (e.g., OpenAI’s gpt4 or Anthropic’s Claude3.5). I have taken the same expertise and context layers and created versions of the same SmartBot on multiple platforms (e.g. Gemini, NotebookLM, Anthropic, MS Co-Pilot, proprietary platforms etc). Open AI has the easiest to use tech platform to do this currently, though for some applications where I need to collect user data or need multi-LLM architectures, I have had to rely on other platforms (e.g. see noodle factory) or even build my own (e.g. Bionix, co-designed with Nghi Truong).
How to build the expertise layer?
I start with… expertise. In my experience, uploading a bunch of papers or books (assuming the copyright regime allows you to do so) does not work well because books and papers, unless they are of the “how to do X” genre, do not necessarily represent directly actionable knowledge. A RAG model with a book as input is optimized to answer questions about the book (querying), not necessarily for how to make decisions based on knowledge in the book (applying).
Instead, I have found it useful to build classic expert systems — by converting knowledge into a structured format, such as declarative statements, flowcharts, decision trees, and examples. The expertise layer thus represents a proprietary set of ideas — just like an academic paper- which duly cites prior art but represents a novel output achieved through synthesis, integration and re-organization of knowledge contained in prior work. Like an academic paper, some of these expert layers are co-authored.
To take an example, I wrote a book that suggests a new way of thinking about organization design. It is written primarily for researchers, as indeed all my academic papers are. To convert these into actionable knowledge, I have to think about the typical set of problems that organization designers face and set up frameworks to solve each of them, identifying information needed and underlying trade-offs, and create a logic for deriving a decision. The results must include explanation and action plans. None of this arises automatically from reading the book. It is knowledge work, that has to be done. A SmartBot like Eunomicon does a lot of this work for the user.
Expertise layers can also be enhanced by the addition of automated machine learning that can extract useful predictions from data without having to rely on knowledge that experts may find hard to articulate. We can also bootstrap the process by using an LLM to convert a paper or a book into an expert layer, but so far human involvement seems to be necessary in this process, based on performance. How do we measure that? We conduct A/B tests to compare the performance of the SmartBot against 1) a foundation model and 2) human expert answers. Finally, we iterate… a lot.
SmartBots thus represent a modern re-incarnation of expert systems, leveraging LLM’s capacity for associative reasoning, and their undisputable competence at natural language based inputs and outputs. That leads to an important practical advantage: they can explain their decisions in terms their builders and users can understand.
