Turing Machine 2.0
Large Language Models (LLMs) are trained with a fixed set of parameters, creating a vast yet unchanging knowledge base after training. Their memory capacity, defined by architecture and context window size, is also fixed.
Unlike Turing machines, which can theoretically have unlimited memory via an expandable tape, LLMs lack this flexibility. This limits them from computations needing unbounded memory, distinguishing them from Turing-complete systems. Instead, LLMs function as powerful pattern recognisers, approximating functions based on large but finite datasets.
The context window, however, provides a way to temporarily extend an LLM’s memory. This is why Retrieval-Augmented Generation (RAG) has become so popular: it dynamically feeds relevant information into the model, though it remains read-only. As we explore more “agentic” uses of LLMs, though, we start considering the need for read-write memory. In this setup, the LLM functions as the “read/write head” of a Turing machine, reading from memory into its context window and writing key information back.
The question is: if the LLM is the "read/write head," what serves as the "tape"? A simple solution is plain text, as used in tools like ChatGPT. This works to a degree, but plain text alone is imprecise, lacks mechanisms for compression, generalisation, and interlinking of information, and may not integrate well with the structured "memory" already present in organisational documents and databases.
A fully neural architecture with slowly evolving 'background' neurons and faster-changing, inference-time ones—capable of integrating new information incrementally—could indeed be the end goal. However, a more immediate and pragmatic solution for organisations today is to build a Semantic Layer.
Anchored in an "Ontological Core" that defines the organisation’s key concepts, this Semantic Layer interfaces with LLMs, allowing them to read from an ontology that links back to the data in underlying systems. With human oversight, LLMs can also write missing classes into the ontology and add missing facts back into these systems, creating a dynamic Virtual Knowledge Graph for the organisation.
In this setup, the Virtual Knowledge Graph effectively becomes the Turing Machine’s “tape”—a dynamic, interconnected memory that the LLM can read from and write back to. By linking facts with URLs and using ontology for generalisation and compression, this graph provides an ideal read/write memory structure for the LLM. Combined in a neural-symbolic loop, this LLM+Graph system could even be Turing complete.
This may sound like a far-future concept, but the future is arriving fast. I’m not saying it will be easy, but any organisation transitioning to Data Products can begin this journey today by simply adopting DPROD as their Data Product specification. Often, the first step in a journey turns out to be the most important one!
⭕ Ontologies + LLMs: https://www.knowledge-graph-guys.com/blog/llms-ontologies
