Prompting with Knowledge Graphs:
Why the Language Model Only Needs to Translate
A regular pentagon with vertices A through E. What is the shortest path from A to D? It sounds like elementary school geometry, yet a large language model usually answers: directly along the edge A–D. Unfortunately, that edge doesn’t exist.
Dr. Sebastian Goeser, Managing Director of Stuttgart-based Textverstehen GmbH, has demonstrated how knowledge graphs enable AI responses to be precise and verifiable—even with smaller models. He used this simple test to introduce his topic. His point: the model lacks a true representation of the pentagon. Knowledge graphs fill precisely this gap.
The LLM as Translator, Not Knowledge Source
At its core, a knowledge graph is a vast collection of statements following the pattern: subject, property, object. During prompting, relevant statements from the graph are appended to the query; according to Sebastian, they act like a track along which the answer travels. The language model then has one primary task: translating structured facts into natural language. The side effect is significant. Because the knowledge resides in the graph rather than the model, there is no need for massive models from overseas.
In the demo, everything ran on a Mistral model. And if the graph yields no match—such as when asked for a straw wall lamp that simply doesn’t exist in the catalog— the result is an honest “no” rather than a fabricated answer.
When the Effort Pays Off
Not every use case requires a knowledge graph; many can be effectively handled using classic RAG. Things get interesting in highly structured domains involving heterogeneous data from various formats, teams, and sources—and regarding a point that is often overlooked: data changes constantly. Updating this in a standard model or RAG repository is a laborious process; a knowledge graph, by contrast, is a database and can therefore be modified at any time.
Snow shovels: cumulative vs. distributive
For the demo, Sebastian built a graph based on a product catalog: 1,000 items from a Tyrolean online shop—featuring lots of wood and tools—resulting in around 14,700 statements. He used this to demonstrate various prompting styles. The difference became tangible with the question, “What kind of snow shovels do you have?” The cumulative style aggregates statements about all matching products to provide an overview. The distributive style examines each product individually, clearly distinguishing between, say, an aluminum car shovel and a practical two-piece set. The rule of thumb: use the cumulative style for an overview, and the distributive style when focusing on the attributes of individual products—or when the graph yields so many results that the model would otherwise be overwhelmed.
How does the AI actually know this?
Perhaps the most important feature is reification: every statement within the graph is itself an object that can be traced back to the specific text snippet from which it originated. Take, for instance, the automatically extracted statement: “Insect hotel is related to environmental protection.” Sebastian’s dry comment on this: “Who would want to dispute that?” This traceability is invaluable for generating explainable answers.
He also demonstrated live how such a graph is created: inputting a small XML document, applying AI-driven mapping, and performing grammatical analysis and cross-referencing with Wikidata. The result was 69 statements derived from a single short document. If a new statement is added that contradicts existing ones, it is flagged as a conflict—and, in cases of doubt, the client is consulted. Absolute truth? “Well, who really knows?”
And what about MCP?
This question came from the chat. Sebastian’s answer: MCP connects various data sources but does nothing for semantic integration. You are dependent on how the LLM accesses the connected resources, meaning you have less control over the response. “To put it simply, it’s only half the battle.” Incidentally, the actual use case extends far beyond snow shovels: it involves machinery manufacturers with complex products whose information is scattered across databases, PDFs, and websites—plus a project for object recognition in images.