In short: Knowledge engineering is the discipline of making what an organization knows explicit enough that a machine can use it — defining concepts, encoding the relationships between them, and capturing expertise that currently lives in people's heads. It is not a new field, and the practices data teams are inventing from scratch already exist. This piece covers what the term means, why it's resurfacing now, and where a data team should actually start.
There is a slide in circulation at almost every enterprise right now. At the center sits the company brain: every document, ticket, table and Slack thread fused into something an AI agent can reason over. The pitch is that the company already holds the knowledge and only needs to connect it.
Most of those slides don't survive contact with production, and the reason isn't the model. It's that knowledge engineering has methods, standards and roughly three thousand years of practice behind it, and almost nobody building a company brain is consulting any of it.
Knowledge engineering has two meanings, and the gap between them is the problem
Search the term and you'll mostly find the classic definition: a subfield of AI concerned with building expert systems, formalized from the 1970s onward around rule bases and inference engines. That version is real and it never went away. As Ido has written previously, Edward Feigenbaum's original framing — extracting knowledge from human experts and encoding it into a knowledge base — still describes the work accurately.
The second meaning is the one data teams have backed into. Somebody has to define what a metric means, reconcile two systems that disagree, and write down why a pipeline works the way it does. That's the same discipline, unnamed, distributed across people who were hired to do something else. We've argued before that the knowledge engineer never disappeared so much as got absorbed into data engineering — and if you want the role-by-role comparison against data engineering, that piece is the place to start.
Jessica Talisman comes at the same gap from the other side, and she's less forgiving about it.
Jessica Talisman has a name for that, and she knows it sounds harsh.
Epistemic squatting is what happens when data practices meet knowledge work
"There's this assumption that people can come into the world of knowledge and that we can just lift and shift practices from data over into knowledge practices. It doesn't really work that way."
She calls it epistemic squatting. Moving into a discipline you haven't studied and rearranging the furniture.
She has standing to say it. Talisman started in the late 1990s building one of the largest visual history catalogs in the world, for Steven Spielberg's Shoah Foundation, back when the source material was VHS tape. She holds a master's in information and library science and spent years as an academic librarian before crossing to the corporate side in 2018. Then Amazon. Then Adobe. She now consults independently and helps lead the Knowledge Graph Academy.
Her objection isn't that data teams are unqualified. It's that they're treating a field with a lineage running from scrolls through the printing press into every digital system since as though it were greenfield.
"Context" is the most misused word in data right now
Ask her to name it and the answer arrives in one syllable: context.
The complaint is about precision, not fashion. "Context" now covers anything adjacent to knowledge — a sentence, a whole document, data tables, metadata, a taxonomy. She's watched people define knowledge as context plus data, a formulation she considers close to meaningless. Her own definition is narrower and more useful:
"What they're really talking about is the relationships between things. That's what context actually is."
Objects can sit in space indefinitely. How they relate is where context gets codified, and that codification is where it starts becoming knowledge. Offered the choice between the two words, she takes knowledge without pausing.
This matters more than a terminology dispute normally would, because vague words produce vague architectures. A team that cannot say what context is will buy a product claiming to supply it and have no way to test whether it did — which is why we've argued that the context layer won't build itself.
Your database holds a sliver of what your company knows
The expensive version of this error is an assumption about location.
Most organizations keep financial records, customer records and similar structured data in databases, tables and lakes. Talisman's position is that this represents a fraction of what the organization actually knows. There is Confluence. There is SharePoint. There are S3 buckets holding whatever got dropped in them, which she describes, generously, as a yard sale. There are Jira tickets nobody has classified.
Her summary of the current state is the line from the episode most likely to end up on somebody's slide:
"Knowledge right now exists as just sort of a 52-card pickup across your organization."
The analytics view — the thing currently marketed as the semantic layer — is one sliver. Warehouse and lake metadata is a second. Feeding a model the narrow slice while calling it the company's knowledge is, in her assessment, not the soundest deployment available.
She traces the mistake to a hangover from the big data era, a bias that everything mattering in an organization lives in a database and therefore knowledge must live there too. It's the same root cause behind AI systems that return confident wrong answers — not a model problem, a knowledge problem.
The test she offers is personal. On your own quest to understand a topic, you don't open a database. You find papers. You read. You build understanding, and only much later define it in terms a machine can use. Nobody applies to their own learning the method they're applying to their company's.
Tacit knowledge and untagged knowledge are not the same problem
The sloppiness runs straight into the phrase everyone reaches for when describing the gap.
Tacit knowledge means knowledge that has never been printed, codified or captured. It sits in someone's head. That is the entire definition, and it carries a consequence most roadmaps haven't absorbed:
"As soon as it enters a Slack thread, it's no longer tacit knowledge, because it's recorded."
What people usually mean by it is knowledge nobody has tagged, labeled or classified. Real problem. Different problem.
Tacit knowledge | Untagged knowledge | |
Where it lives | In a person's head | In systems, unclassified |
What it needs | Elicitation — observing an expert work | Governance and controlled vocabulary |
Who does the work | Domain expert plus interviewer | Data and knowledge teams |
Failure if confused | You build a catalog nobody can populate | You interview people about things already written down |
Alt text: Comparison chart showing tacit knowledge requires elicitation from experts while untagged knowledge requires governance and controlled vocabulary.
On a roadmap the two look identical. In practice they consume different budgets, different people and different amounts of calendar time.
This is the part that should give pause to anyone shipping a knowledge product, ourselves included. Upriver is built on the premise that the understanding trapped in people's heads and scattered across systems is worth capturing permanently — that's what our agent's living map of a data environment is for. Talisman's point is a discipline check on that premise. Know which of the two problems you're solving, because the marketing language collapses them and the work does not.
The topic comes before the table
So where does a data engineer with a large lake and no obvious entry point start? Four words.
"Think beyond your database. So that's the biggest advice."
That means inverting the usual direction of travel. Decide which topics you are building knowledge around. Understand them. Then go back into the systems and find where those topics actually live — in tables, but equally in Confluence pages, in tickets, and in how the rest of the world defines the same concepts. Start from the tables and ask what could be built from them, and you get, as Ido put it during the recording, a set of random relationships between points.
There's a library science instinct worth borrowing here too. Commercial AI products arrive pre-trained, largely on the open web, carrying opinions about what your terms mean. So it pays to define things internally and to disambiguate how those definitions sit against everyone else's.
The mechanics predate the current cycle. Talisman builds and teaches taxonomies using SKOS, the Simple Knowledge Organization System — a W3C standard and RDF data model structured around broader and narrower concepts, where each concept carries alternative labels, definitions, scope notes and a unique identifier. Unglamorous, and foundational. Asked whether an AI can build a taxonomy on its own, she answers in one word: no.
The newest idea in AI is one of the oldest
There's a reason this material is resurfacing, and it isn't nostalgia.
Scaling laws demonstrated that more data does not make AI more reliable. That result pushed attention back toward taxonomies, ontologies, thesauri and knowledge graphs as a route to truthfulness.
These are symbolic AI. GOFAI, good old-fashioned AI. Combine symbolic with generative and you have neurosymbolic AI, which Talisman is quick to note is not a new discipline — thirty-plus years old as a field, with research reaching back to the 1940s. Her read on the current enthusiasm is dry: plenty of people are discovering it now and think they've struck gold.
The older tradition also carries a practice worth reviving. Augmentation before automation. You keep a human in the loop at the points where you aren't yet certain the implementation is right — a reliability measure, but also something more useful. Those breakpoints are precisely where you can record how a subject matter expert actually works. If the knowledge is experiential, brains to fingers on a keyboard and hard to articulate, watching it happen is the only way to capture it. Going zero to sixty skips the checkpoint and the record in one move.
The same theme ran through our conversation with Tiankai Feng on why data quality is a people problem. The hard part keeps turning out to be human.
The cleanup bill is already in the mail
Meanwhile, a mess accumulates.
For three years organizations handed AI tooling to employees with no guardrails, no plan and no strategy. The byproduct is an enormous volume of generated documents. Duplicates. Triplicates. Nothing versioned. Prompts never saved. Content pasted into Confluence pages nobody has read.
Talisman expects a correction inside two years, driven less by conscience than by EU AI Act requirements and the penalties attached. It returns to the fundamental she keeps circling: you cannot organize, label or serve up something you can't find. Ido's response in the moment was to laugh and say we're digging ourselves a bigger hole, but it'll be fine. Which is roughly where the industry sits.
Start slower than you want to
Talisman's book, out in September, comes from a decade of building knowledge programs inside organizations, aimed at the question she hears most: where do I start, and how?
Two answers stand out. Too many organizations jump straight to taxonomies or knowledge graphs before they've cleaned, reconciled or defined their data — and that preparatory stage is the same conceptual, logical and physical modeling discipline databases have always demanded. And knowledge graphs are not an architecture. They get discussed as a universal thing you adopt, when the shape varies enormously with your stack, your size and how many sources you pull from.
Before either, she starts clients on functional requirements: what do we have, in what shape, in what format does it exist. A slower opening move than any roadmap wants. That is the point. Knowledge is not a fast thing, which is precisely why "move fast and break things" generates so much of it that nobody can use.
Closing takeaways
- Audit before you architect. What exists, in what shape, in what format — not tool selection.
- Separate untagged from uncaptured. One needs governance, the other elicitation. Budget them apart.
- Inventory beyond the warehouse. A knowledge map that stops at the database covers a sliver.
- Choose topics, then find data. Define the concepts before hunting for tables to connect.
- Keep humans at the uncertain points. That's how expertise nobody wrote down gets recorded.
- Treat the document sprawl as a liability. Three years of ungoverned AI output is a compliance exposure before it's a knowledge problem.
Frequently asked questions
What is knowledge engineering?
Knowledge engineering is the discipline of capturing what an organization or expert knows and encoding it in a form a machine can reason over. In practice that means defining concepts, formalizing the relationships between them, and eliciting expertise that has never been written down. It originated as a subfield of AI focused on expert systems, and it has resurfaced because scaling more data into large language models did not make them more reliable.
Is knowledge engineering the same as data engineering?
No. Data engineering is concerned with whether data moves correctly — pipelines, freshness, schema, reliability. Knowledge engineering is concerned with what the data means: definitions, relationships, and the reasoning that connects them. The two overlap heavily in practice because, in most organizations, no one holds the second job formally, so data engineers absorb it. The full role comparison is here.
What is the difference between tacit knowledge and untagged knowledge?
Tacit knowledge has never been recorded anywhere — it lives in someone's head. Untagged knowledge has been recorded but never labeled or classified, so nothing can find it. As Talisman puts it, the moment something enters a Slack thread it stops being tacit. The distinction matters because tacit knowledge requires elicitation from a person, while untagged knowledge requires governance and a controlled vocabulary.
Can AI build a taxonomy or ontology on its own?
No. Asked directly on the episode, Talisman's answer is one word. On her account the definitional work — deciding what a concept means, how it relates to others, and how it disambiguates against outside definitions — is where knowledge begins, and it requires people who understand the domain.
Why is knowledge engineering relevant again now?
Scaling laws demonstrated that more data does not make AI more reliable. That pushed attention back toward taxonomies, ontologies and knowledge graphs — symbolic AI — as a route to systems that can be trusted. Combined with generative models, the approach is called neurosymbolic AI, a field over thirty years old with research reaching back to the 1940s.
Where should a data team start with knowledge engineering?
With functional requirements, not tools: what do we have, in what shape, in what format does it exist. Then choose the topics you're building knowledge around and find where they live across every system, not just the warehouse. Jumping straight to a knowledge graph before the underlying data is reconciled and defined is the most common failure mode.
Your taxonomy can wait. Your tickets can't.
Upriver's agent maps your entire data environment and encodes what your team knows — so the knowledge stops walking out the door every time someone leaves.
Listen to the full episode
YouTube: https://youtu.be/ZlBAjOU111c?si=xN2h46C7eRsWwKAh
Spotify: https://open.spotify.com/episode/4D9KomhymgRZWUusJCvdLK?si=VR_GlXctQHOlBm8gC4qNPA
Follow Data Splash for more conversations on data engineering in the AI era. New episodes every other week. Spotify · YouTube
