The Semantic Router

There's a rumour circulating that GPT-4 might be an 8-way mixture model. Whilst I can't confirm its veracity, it's a fascinating opportunity to delve into the world of Semantic Routing.

So, what's the deal with a Mixture of Experts (MoE) model? Essentially, it revolves around a router that directs questions to the appropriate expert. If GPT-4 does adopt the MoE approach, it would consist of eight specialist models each trained in a specific domain, like mathematics or storytelling. When a question is posed, the router analyses it and seamlessly forwards it to the most suitable expert.

The concept of MoE is quite prevalent, with Langchain's high-level implementation of an LLMRouterChain, and notable low-level integrated examples like Google's Switch Transformers.

Now, let's take things a step further and introduce the notion of a 'Semantic Router'. Here's the idea: the router initially maps the question to relevant classes within the organisation's upper ontology—a structured representation of key concepts that your business is focused on. Utilising these classes, the router then retrieves the corresponding 'Semantic Data Product' from the organisation's Semantic Layer. If we elevate this approach and assume that each Data Product is, essentially, a ‘Data Agent’ equipped with its own embedded AI model, then the entire system becomes a sort of Mixture of Experts.

If our aim is to integrate ALL the information within our organisations, we'll need to distribute these ‘Semantic Data Agents’ over the network. Picture this: each agent receives a context graph from the router and, in return, provides a results graph as an answer. The router seamlessly integrates these results into its Working Memory Graph, fuelling the subsequent steps within its Graph of Thought.