In-company RAG
Start building you In-company RAG system and learn how to train your model. Learn the basic setup and common pitfalls when it comes to in-company AI.
Read more
First Steps in Building a Company-Specific AI
The past year has shown remarkable momentum in artificial intelligence. At the same time, we are still very much at the beginning of understanding what AI can really do inside companies. There is a large elephant in the room: you cannot simply upload company data to the cloud and hope for the best. Despite all the promises around safety and privacy, many organisations are simply not willing to take that risk.
As a result, most organisations experiment with generic AI tools. These tools are useful for learning, inspiration, and exploration. They help people understand what AI can do. But they fall short when it comes to capturing the unique processes, documents, terminology, and logic that defines how a specific company works. In my opinion that is where the true value of in-company AI lies: specialized systems that truly understand the data, language and logic of a company and its context. Safe, up-to-date, transparent and accurate. The next phase in AI within companies therefore requires a shift: moving from general-purpose experimentation to building company-specific AI systems that are grounded in your own data and context.
What is RAG?
A RAG system combines two core capabilities. First, Retrieval: fetching relevant information from your own sources, such as documents, procedures, policies, product data, contracts, or planning files that you have explicitly selected. Second, Generation: a Large Language Model uses this retrieved context to produce accurate, grounded, and verifiable answers.
When I say LLM, I should add that I prefer to use the smallest model that gets the job done. In simple terms, RAG ensures that the answers you receive are based only on your own information. Traditional LLMs rely on statistical patterns learned during training and on whatever data happened to be in their original corpus (the included information). RAG makes output more reliable, more consistent, and better aligned with how your organization works.[1] It also makes it possible to manage information, versions, and quality through defined KPIs. I will return to this later, but the key point is that RAG increases transparency and controllability. If done correctly.
And if structured properly, it can all be done in a safe and secure way—if necessary, even fully local or air‑gapped.
In a typical RAG setup, when a user asks a question, a Smart Retriever is responsible for selecting the most relevant pieces of information from the knowledge base. Rather than relying on simple keyword matching, it uses embeddings and metadata to capture intent and context. The Smart Retriever searches content that has been indexed in advance (documents, databases, transcripts) and retrieves small, structured pieces of information known as chunks. These chunks are fragments of larger documents that are stored individually so the system can retrieve only what is relevant, instead of entire files.
[1] Oche, A.J., A. G. Folashade, T. Ghosal, A. Biswas, (2025), A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions, arXiv:2507.18910v1 [cs.CL] 25 Jul 2025. {https://arxiv.org/abs/2507.18910}.
Download our Guide
The retrieval part of RAG seems straight forward really. But why is it so difficult to develop a system that can be used in a company environment? This is due to the fact the building a proper knowledge base requires quite some insight into the different techniques, it needs to be prepared, checked, and maintained. You must organize the knowledge base or corpus as it is called. Later, we will focus on the corpus and how distinct types of data require different ingestion and retrieval techniques.
But first let us focus on building the knowledge base: the Ingestion Pipeline. Simply uploading your files to an app does not cut it unfortunately. It is this process and the way it is set up that determines the quality of the answers. It is this part of the process, the ingestion pipeline, which is not very visible when using the commercial apps out there, that is the crucial part. The way data is stored and determines the quality of the output.
Some closing remarks on RAG
RAG reduces risk and increases trust by forcing models to answer using verified sources. It lowers hallucination risk, improves auditability, and enables better control over versioned documents. In regulated and high-stakes environments such as manufacturing, logistics, and pharmaceuticals, this matters enormously.
Equally important is the fact that RAG enables secure, on-premises deployment. Unlike fully cloud-based AI, RAG can run on company servers, on edge devices, or in air-gapped environments. This dramatically reduces concerns around intellectual property, sensitive data exposure, compliance, and latency. RAG is also the foundation for more advanced capabilities. Mature RAG systems enable knowledge graphs, AI agents, automated reasoning, scenario analysis, and end-to-end decision support. Without RAG, these systems remain unreliable. In short, RAG transforms AI from a generic assistant into a precise, contextual, and trustworthy enterprise system. When implemented carefully, it becomes the backbone of company-specific, vertical AI. When implemented crudely, it becomes little more than a search tool with better phrasing.
Create a safe environment | the sandbox
We’ve seen firsthand that creating an environment where small, scrappy teams can innovate without being slowed down by endless reviews is key to unlocking AI’s potential.
How to deploy In-Company AI
Building a Company-Specific Vertical AI: A Practical Guide for C-Level and Supply Chain Leaders. Why Companies Are Exploring Vertical AI.
What we Learned building our Vertical
Going from an abstract vision of “AI that works for supply chain” to a functional, compliant, production-ready AI stack is harder than it sounds
When numbers start talking
Our founder Bas Groothedde meets our In-Company AI Agent AVA. Great stuff.
Giving your AI context in the supply chain
Context-prompting, an important tool to give an llm context. There is strong theoretical and empirical evidence that providing AI with better context improves its performance.
The Evolution of AI Agents
The landscape of AI agents has advanced from basic chatbots to sophisticated entities managing complex tasks across industries.
This is World interview featuring Professor Yann LeCun
LeCun brings clarity to the current discourse on AI with a number of thoughtful, remarks and putting it into perpective. A must-watch for anyone navigating the future of artificial intelligence.
AI is moving so fast, which is why your strategy should be long-term.
Danny Hillis, a pioneer in AI He understood something decades ago that many executives are only now coming to terms with. Technology does not evolve in stable, predictable increments.
The future of AI is not just language models. It’s causal reasoning.
Judea Pearl, is another true pioneering in artificial intelligence. But he is a little bit different. And not on the bandwagon if LLMs. Teach machines to understand the question why.
Intelligence is not what we think it is
In our series about AI, we came across the work of Professor Herbert “Herb” A. Simon, one of the founding figures in artificial intelligence.
Why the human factor remains essential in the Digital Era
No matter how advanced technology becomes, human insight remains essential in managing real-world complexity.
In the age of AI, large corporations—not just startups—can move fast
At verticallm.ai, we’ve seen firsthand that creating an environment where small, scrappy teams can innovate without being slowed down by endless reviews is key to unlocking AI’s potential. Too often, even agile teams in large organizations get stuck in molasses. Privacy, marketing, legal, and financial reviews all pile on—each valid on its own, but collectively a major brake on speed. When an engineer needs approval from five vice presidents to ship a prototype, how can you expect innovation?
And yet, the paradox: we now have the tools to move faster than ever.
AI-assisted coding. Rapid prototyping. LLMs and vertical agents that can be tested in days. But while the tech moves fast, corporate systems haven’t kept up.
So what’s the answer?
A sandbox.
A controlled environment where innovation can happen quickly—without putting your company, your brand, or your customers at risk. Testing Agents, RAG, Tools in a controlled environment.
Why Sandboxes Work
Think of a sandbox like a lab. You don’t build the final product there—you explore, you test, you learn. You make mistakes early, where it’s safe. A sandbox is not about building a polished enterprise-grade solution. It’s about answering questions like: can we use RAG to surface the right knowledge to the right user, in real time, can agents improve the workflow of our supply chain analysts, can this tool be customized to fit our specific vertical—without months of development?
In our experience at verticallm.ai, a good sandbox gives you: (1) Speed, build and test ideas within days or weeks, not quarters; (2) Safety, keep experiments isolated from production systems and sensitive data; (3) Structure, let teams innovate with just enough guardrails—not red tape.
What You Need to Get Started
Here’s a concrete blueprint we’ve used—refined with our partners inside large companies
Set a Clear Scope (Don’t Build the World)
Start with a narrow business question or use case. Examples:
- Can we summarize customer complaints using internal data and a RAG model?
- Can we use an agent to automate 20% of our weekly KPI reporting?
- Can AI surface lead time risks from our supply chain dashboards?
Small questions lead to fast, clear results.
Define the Boundaries of Your Sandbox
You need technical and organizational constraints:
- Data: Use anonymized, non-sensitive, or synthetic data.
- Access: Restrict access to a small team of builders and testers.
- Environment: Run tests in a secure, isolated cloud instance.
- Governance: Define what needs approval—and what doesn’t. Make it explicit.
Tip: A sandbox doesn’t need to connect to your enterprise systems to be valuable. Simulate where needed.
Choose Your Stack Wisely
Your sandbox is not your product stack. Use tools that are:
- Lightweight (no 6-month implementations)
- Composable (easy to swap components)
- Interpretable (you need to explain what the model is doing)
For RAG and agent experimentation, we recommend:
- LLM access via APIs (e.g., OpenAI, Anthropic, or open models via HuggingFace)
- Vector databases like Pinecone, Weaviate, or Chroma
- LangChain or LlamaIndex for orchestration
- Streamlit or Gradio for building quick front-ends
We also offer our own agent framework at verticallm.ai, optimized for supply chain verticals—reach out if you’d like early access.
Put a Team in Place—and Shield Them
A sandbox only works if the team can actually move. That means:
- One product owner: Someone who knows the business pain point.
- One AI engineer or builder: Doesn’t need to be an LLM expert—just someone comfortable with APIs and prototyping.
- No committee reviews. Weekly check-ins with a single exec sponsor is enough.
This is not a transformation team. It’s a strike team.
Common mistakes
A common pitfall is trying to solve everything at once. Everybody should focus on one sharp use case and try to get results fast. Another showstopper is usually getting blocked by data access or low quality data. A fix is to use public, dummy, or anonymized data to start. Remember, it is not about the actual value yet. The value is in getting started and experience. Another pitfall is overbuilding. really not to focus to much on CI/CD, security audits, and enterprise architecture (for now). Finally we saw that organizing the cases in small teams, not too many stakeholders by keeping the core team small and update others weekly.
Ground Rules for a Corporate AI Sandbox
Start Narrow—Prove Value Quickly
Pick one clear, focused use case (e.g., summarize customer emails, automate KPI checks).
No boiling the ocean. If your scope takes more than 3 weeks, it’s too big.
Use Dummy or Anonymized Data
Only use non-sensitive, public, or sanitized data.
If you must use internal data, strip out PII and secure access.
Rule of thumb: If legal needs to review it, it doesn’t belong in your first test.
Build Ugly, Fast
This is not a production system. It’s a learning lab.
Use Streamlit, Jupyter, or even CLI demos. No need for slick UI.
Focus on answering: “Does this work?” Not: “Is this beautiful?”
Demo Every Week
Show progress, no matter how small.
Share failures and dead ends—that’s what the sandbox is for.
Short demo sessions (15–30 min) with one sponsor or stakeholder max.
No Permission Loops
Agree upfront: what can the team do without asking?
For example: “You don’t need approval to deploy inside the sandbox” or “You can test on dummy data without data governance review.”
Keep approval surfaces small and explicit.
Document Learnings as You Go
Use a shared doc, wiki, or Slack channel.
What worked, what didn’t, what you’d do differently.
This becomes your internal playbook for future sandboxes.
Keep the Team Tight
Max 3–4 core contributors.
Everyone should be able to ship something—not just discuss.
If more than 6 people are on a call, you’re probably out of sandbox mode.
Everything Lives Inside the Sandbox
No integration with production systems.
No direct access to customer-facing tools.
Think of it like a lab-in-a-box. Experiments don’t leak.
In-company RAG
Start building you In-company RAG system and learn how to train your model. Learn the basic setup and common pitfalls when it comes to in-company AI.
Create a safe environment | the sandbox
We’ve seen firsthand that creating an environment where small, scrappy teams can innovate without being slowed down by endless reviews is key to unlocking AI’s potential.
How to deploy In-Company AI
Building a Company-Specific Vertical AI: A Practical Guide for C-Level and Supply Chain Leaders. Why Companies Are Exploring Vertical AI.
What we Learned building our Vertical
Going from an abstract vision of “AI that works for supply chain” to a functional, compliant, production-ready AI stack is harder than it sounds
When numbers start talking
Our founder Bas Groothedde meets our In-Company AI Agent AVA. Great stuff.
Giving your AI context in the supply chain
Context-prompting, an important tool to give an llm context. There is strong theoretical and empirical evidence that providing AI with better context improves its performance.
The Evolution of AI Agents
The landscape of AI agents has advanced from basic chatbots to sophisticated entities managing complex tasks across industries.
This is World interview featuring Professor Yann LeCun
LeCun brings clarity to the current discourse on AI with a number of thoughtful, remarks and putting it into perpective. A must-watch for anyone navigating the future of artificial intelligence.
AI is moving so fast, which is why your strategy should be long-term.
Danny Hillis, a pioneer in AI He understood something decades ago that many executives are only now coming to terms with. Technology does not evolve in stable, predictable increments.
The future of AI is not just language models. It’s causal reasoning.
Judea Pearl, is another true pioneering in artificial intelligence. But he is a little bit different. And not on the bandwagon if LLMs. Teach machines to understand the question why.
Intelligence is not what we think it is
In our series about AI, we came across the work of Professor Herbert “Herb” A. Simon, one of the founding figures in artificial intelligence.
Why the human factor remains essential in the Digital Era
No matter how advanced technology becomes, human insight remains essential in managing real-world complexity.
Building a Company-Specific Vertical AI: A Practical Guide for C-Level and Supply Chain Leaders
Introduction: Why Companies Are Exploring Vertical AI
Artificial Intelligence (AI) is transforming industries, and supply chains are no exception. Many companies have already started experimenting with AI-driven forecasting, automation, and analytics. However, generic AI models—such as ChatGPT or other large language models (LLMs)—often fall short of the specificity needed to address complex supply chain challenges. This is where Vertical AI comes in: a company-specific AI model tailored to your industry, your data, and your business processes.
This article is designed for executives who are considering investing in a Vertical LLM (Large Language Model). We will break down the concept, discuss its benefits and challenges, and provide realistic expectations regarding costs, implementation, and adoption.
This illustration shows how we are able to transform your business questions into actionable insights through structured data and AI. And Structure and structured context is the trick.
By combining company-specific data, industry insights, and supply chain expertise, our trained AI platform provides tailored decision support. The process is intentionally guided through our surveys (that cover strategy, resilience, maturity, technology, and sustainability). This helps you through this guided process, it helps the AI because it can be trained on structured and input that can be processed in a similar way, and finally by structuring this process we can control and validate the output.
Step 1. Your Business Question
The process starts with your business question—whether it relates to supply chain strategy, operational resilience, or other critical areas. This question is the foundation of the analysis.
Step 2. Business Context via Surveys
To ensure that the AI platform is fully aware of the company’s specific context, we guide the user through a structured set of surveys. These cover topics such as strategy, resilience, maturity, technology, and sustainability. This step ensures the AI receives structured, relevant information tailored to your company’s unique situation.
Step 3. Industry Data Integration
We use Agents to train our AI vertical and our AGI-driven News Desk continuously scans for news, trends, reports, and industry-specific developments 24/7. This ensures the AI is always up to date with the latest market information relevant to your business environment, providing real-time context and competitive insights.
Step 4. Supply Chain Expertise
A second team of agents helps us to capture the essential knowledge of supply chains. Our AGI Supply Chain Research Team contributes deep research on supply chain concepts, emerging trends, methodologies, and industry terminology. This expertise is essential to train the AI in understanding the nuances and complexity of supply chain management, ensuring that outputs are accurate and grounded in best practice.
Step 5. Numerical Data Integration
The process is further enhanced by integrating structured numerical data—such as orders, volumes, locations, production figures, costs, and schedules—through secure, SQL-based queries within a closed environment. This ensures a safe and confidential analysis of hard data, enabling precise trend detection, benchmarking, and actionable insights.
Step 6. Feedback and Continuous Learning
The AI delivers tailored outputs and insights back to the user, providing structured feedback such as customer value profiles, resilience assessments, maturity evaluations, and technology readiness. This process is continuously refined through our agent-based architecture, which ensures daily improvements as results are validated against hard numerical data.
What is a vertical AI model
Unlike general-purpose AI models, a Vertical LLM is designed to understand and generate insights within a specific domain. For a supply chain company, this means an AI that deeply understands logistics, demand planning, procurement, and supplier risk management—not just general business concepts.
A Vertical AI must grasp the terminology, concepts, and specific aspects of the industry context in which the company operates. Since supply chains are driven by numbers, it must be integrated with numerical data. Without this foundation, even the most sophisticated AI remains generic—relying on averages that can lead to risky decisions. Instead of providing textbook responses to generic queries like “best inventory practices,” a Vertical AI model can access your historical supply chain data and patterns, is trained on the unique terminology and constraints of your industry, understands key regulations affecting your operations, and integrates seamlessly with existing ERP and planning systems.
The benefits
The value of a Company-Specific Vertical AI
A Vertical AI, trained on your company’s data and tailored to your operations, delivers more relevant and actionable insights than any general AI can. It enhances decision support by providing accurate, context-specific recommendations that align with your company’s realities. Data privacy and security are also key advantages. Many executives hesitate to use cloud-based AI tools due to concerns over data confidentiality. A Vertical AI can be deployed in a private cloud (VPC) or on-premises, ensuring that sensitive information remains under your control. Another significant benefit is reduced manual effort and faster problem-solving. Imagine an AI that automates repetitive reporting, suggests optimal shipping routes, or flags supplier risks in real time—all without human intervention. This empowers your team to focus on strategic tasks and value-adding activities. A company with a custom-trained AI model can respond to disruptions, optimize inventory, and manage costs more effectively and rapidly.
Setting expectations
Cost, Accuracy, and Adoption Challenges
Building a Vertical AI is an investment. While some foundational models can be deployed relatively quickly, advanced implementations—particularly those requiring deep customization—can be resource-intensive. Regardless of the model’s complexity, the first step should always be deploying in a secure, controlled sandbox environment. This approach allows for experimentation, learning, and refinement in a safe setting. Our view is that Vertical LLMs will become increasingly accessible in the near future, but every implementation still requires careful training, tailoring, and operational integration.
It is important to recognize that even a well-trained Vertical AI will not be flawless. Supply chains are inherently dynamic and unpredictable, and AI models rely on historical data that may not always fully capture future uncertainties. Expect AI to enhance efficiency and reduce errors rather than eliminate them altogether. The best outcomes are achieved when AI is combined with human oversight and expert judgment. Start with a sandbox model, establish appropriate guardrails, and embrace early mistakes as part of the learning process.
One of the most significant challenges is change management. Employees may resist AI tools due to concerns about job displacement or a lack of trust in automated decision-making. Overcoming this requires training and education to demonstrate how AI assists rather than replaces human roles. It also requires clear governance regarding when AI provides recommendations versus when human intervention is necessary. Begin by piloting AI with a small team before expanding across the organization.
How to Get Started: A Step-by-Step Approach
The first step is to create a sandbox environment—a secure space where teams can safely experiment with the AI’s capabilities. This environment should encourage creativity and allow users to test, learn, and refine their approaches without impacting critical operations. Next, ensure secure deployment. Most companies prefer on-premise or VPC-based installations for data protection reasons. While cloud-based AI tools are viable, they must comply with company-specific data protection and privacy policies.
Once the environment is established, define your use case. Identify the specific problem the AI should address—such as automated demand forecasting, supplier risk monitoring, optimized production scheduling, or intelligent order fulfillment and logistics planning. Then choose the right technological approach. You may opt for retrieval-augmented generation (RAG), where the AI dynamically retrieves knowledge from your internal systems, or for a fine-tuned AI model that uses historical company data to make specialized predictions.
After selecting the approach, pilot the AI in one department—such as procurement or demand planning. Measure the impact, refine the model as necessary, and then expand AI capabilities across the broader organization.
Is a Vertical AI Right for You?
Investing in a company-specific AI model is a strategic move that can drive efficiency, reduce costs, and strengthen decision-making. However, success requires careful planning, investment, and strong buy-in from both leadership and operational teams. For executives considering this path, the key is to start small, focus on business value, and prioritize adoption strategies alongside technical development. When done right, a Vertical AI becomes a long-term asset that boosts supply chain resilience and sharpens competitive advantage.
In-company RAG
Start building you In-company RAG system and learn how to train your model. Learn the basic setup and common pitfalls when it comes to in-company AI.
Create a safe environment | the sandbox
We’ve seen firsthand that creating an environment where small, scrappy teams can innovate without being slowed down by endless reviews is key to unlocking AI’s potential.
How to deploy In-Company AI
Building a Company-Specific Vertical AI: A Practical Guide for C-Level and Supply Chain Leaders. Why Companies Are Exploring Vertical AI.
What we Learned building our Vertical
Going from an abstract vision of “AI that works for supply chain” to a functional, compliant, production-ready AI stack is harder than it sounds
When numbers start talking
Our founder Bas Groothedde meets our In-Company AI Agent AVA. Great stuff.
Giving your AI context in the supply chain
Context-prompting, an important tool to give an llm context. There is strong theoretical and empirical evidence that providing AI with better context improves its performance.
The Evolution of AI Agents
The landscape of AI agents has advanced from basic chatbots to sophisticated entities managing complex tasks across industries.
This is World interview featuring Professor Yann LeCun
LeCun brings clarity to the current discourse on AI with a number of thoughtful, remarks and putting it into perpective. A must-watch for anyone navigating the future of artificial intelligence.
AI is moving so fast, which is why your strategy should be long-term.
Danny Hillis, a pioneer in AI He understood something decades ago that many executives are only now coming to terms with. Technology does not evolve in stable, predictable increments.
The future of AI is not just language models. It’s causal reasoning.
Judea Pearl, is another true pioneering in artificial intelligence. But he is a little bit different. And not on the bandwagon if LLMs. Teach machines to understand the question why.
Intelligence is not what we think it is
In our series about AI, we came across the work of Professor Herbert “Herb” A. Simon, one of the founding figures in artificial intelligence.
Why the human factor remains essential in the Digital Era
No matter how advanced technology becomes, human insight remains essential in managing real-world complexity.
Some lessons from our Journey
Going from an abstract vision of “AI that works for supply chain” to a functional, compliant, production-ready AI stack is harder than it sounds. At verticallm.ai, we embarked on this journey with one goal: build an AI that truly understands supply chain data, processes, and decisions—not just a generic model that spits out averages.
First, we learned that Modular design is everything. The temptation to build a monolithic system that does it all is strong—but it’s a trap. Modular design lets you integrate with existing ERP and planning systems, swap components, and scale individual pieces as your needs evolve. For supply chains, where every company’s data and processes are different, this flexibility will be very important.
Second, Observability and Transparency isn’t just a nice-to-have. When your AI is making decisions that impact inventory levels, supplier risk, or customer orders, you need to know how it got there. We invested heavily in monitoring, logging, and traceability—so that we can explain AI decisions, tune performance, and catch unexpected behaviors before they become business risks.
Third, we discovered that Low-code integration matters more than we thought. Not every company has an army of data engineers waiting to deploy a new model. A vertical AI needs to plug into existing systems with minimal friction. That means investing in connectors, API endpoints, and configuration layers that make the AI accessible to business users—not just the IT team.
Finally, scaling is a marathon, not a sprint. We had to balance fast iteration in a sandbox with governance, compliance, and performance testing before going live. Each step required hard choices: how much data to train on, how to manage regulatory requirements, how to deploy in on-premise or VPC environments for maximum security. Every decision impacted the overall resilience and reliability of the stack.
It’s been an eye-opening experience. If you’re thinking about building your own stack, or adapting AI to your industry, this post gives you a front-row seat.
Now, with our own vertical AI stack in place, we’re helping our first development partners build theirs—without the hard lessons we had to learn the hard way. We’ve gone from vision to reality, and we’re excited to help others do the same. If you’re exploring how to build or adapt AI for your industry—whether it’s supply chain, finance, or any other domain—let’s talk. We’d love to share what we’ve learned and help you build an AI stack that works for you.
What is a vertical AI model
Unlike general-purpose AI models, a Vertical LLM is designed to understand and generate insights within a specific domain. For a supply chain company, this means an AI that deeply understands logistics, demand planning, procurement, and supplier risk management—not just general business concepts.
A Vertical AI must grasp the terminology, concepts, and specific aspects of the industry context in which the company operates. Since supply chains are driven by numbers, it must be integrated with numerical data. Without this foundation, even the most sophisticated AI remains generic—relying on averages that can lead to risky decisions. Instead of providing textbook responses to generic queries like “best inventory practices,” a Vertical AI model can access your historical supply chain data and patterns, is trained on the unique terminology and constraints of your industry, understands key regulations affecting your operations, and integrates seamlessly with existing ERP and planning systems.
In-company RAG
Start building you In-company RAG system and learn how to train your model. Learn the basic setup and common pitfalls when it comes to in-company AI.
Create a safe environment | the sandbox
We’ve seen firsthand that creating an environment where small, scrappy teams can innovate without being slowed down by endless reviews is key to unlocking AI’s potential.
How to deploy In-Company AI
Building a Company-Specific Vertical AI: A Practical Guide for C-Level and Supply Chain Leaders. Why Companies Are Exploring Vertical AI.
What we Learned building our Vertical
Going from an abstract vision of “AI that works for supply chain” to a functional, compliant, production-ready AI stack is harder than it sounds
When numbers start talking
Our founder Bas Groothedde meets our In-Company AI Agent AVA. Great stuff.
Giving your AI context in the supply chain
Context-prompting, an important tool to give an llm context. There is strong theoretical and empirical evidence that providing AI with better context improves its performance.
The Evolution of AI Agents
The landscape of AI agents has advanced from basic chatbots to sophisticated entities managing complex tasks across industries.
This is World interview featuring Professor Yann LeCun
LeCun brings clarity to the current discourse on AI with a number of thoughtful, remarks and putting it into perpective. A must-watch for anyone navigating the future of artificial intelligence.
AI is moving so fast, which is why your strategy should be long-term.
Danny Hillis, a pioneer in AI He understood something decades ago that many executives are only now coming to terms with. Technology does not evolve in stable, predictable increments.
The future of AI is not just language models. It’s causal reasoning.
Judea Pearl, is another true pioneering in artificial intelligence. But he is a little bit different. And not on the bandwagon if LLMs. Teach machines to understand the question why.
Intelligence is not what we think it is
In our series about AI, we came across the work of Professor Herbert “Herb” A. Simon, one of the founding figures in artificial intelligence.
Why the human factor remains essential in the Digital Era
No matter how advanced technology becomes, human insight remains essential in managing real-world complexity.
Our founder Bas Groothedde meets our In-Company AI Agent AVA. Great stuff.
Some examples in Supply Chain
Some words of introduction on Prompting and giving context
In our previous post, we illustrated how you can help an LLM by giving it context—what we refer to as context-prompting. Some people asked us, rightfully so, whether this actually works. Are the answers any better? That’s a good question. The simple answer is: yes, they are. However, while giving context is essential, having a proper framework in place to validate every answer is also a must.
Let’s try to answer this question first from a theoretical perspective before we dive into some practical examples. There is strong theoretical and empirical evidence that providing AI with better context significantly improves its performance. This is grounded in several areas of research, including natural language processing (NLP), cognitive science, information retrieval, and reinforcement learning. Here’s why structured context matters and how it aligns with both theoretical principles and experimental findings:
1. The Information Theory Perspective
Claude Shannon’s Information Theory (1948) explains that the more structured and relevant information we provide, the less uncertainty there is in an output. AI systems—especially Large Language Models (LLMs) like GPT—operate on this principle: when an AI model is given vague or insufficient input, it must “guess” more based on probabilistic inference, leading to lower accuracy or more generic, hallucinated, or misaligned responses. When we provide an AI with well-structured, high-quality context, it can reduce uncertainty and generate more precise, relevant, and insightful outputs. In practical terms, this is why prompt engineering and context injection are essential when interacting with AI models: they reduce ambiguity and allow the model to focus on relevant information.
2. Empirical Evidence from AI Research
Several recent studies confirm that providing AI with more structured, domain-specific context improves performance. Key findings include: Few-Shot & Chain-of-Thought Prompting (Brown et al., 2020; Wei et al., 2022). The GPT-3 paper (Language Models are Few-Shot Learners, Brown et al., 2020) demonstrated that LLMs perform significantly better when provided with contextual examples rather than just raw questions. The Chain-of-Thought (CoT) prompting paper (Wei et al., 2022) found that breaking down problems step-by-step with structured context leads to better reasoning and accuracy in complex tasks like math and logical reasoning.
When we expect AI to generate valuable output, we must first structure the inputs properly. If an AI is fed disorganized, context-poor data, its output will be far weaker than if it’s given structured domain knowledge and clear problem framing.
3. Providing Industry and Company-Specific Data
One of the biggest breakthroughs in AI accuracy has been Retrieval-Augmented Generation (RAG)—where an AI retrieves relevant documents, structured data, or domain-specific knowledge before generating an answer. Recent research found that AI models perform significantly better when they retrieve structured context before generating answers rather than relying purely on pre-trained knowledge.[1]
AI’s effectiveness depends on its ability to access high-quality context, either through structured knowledge bases, domain-specific fine-tuning, or real-time information retrieval. It’s as simple as that. This is why, in our opinion, investing in structuring data for AI models is the right strategy. Instead of relying on generic AI outputs, organizations that provide structured knowledge graphs, curated datasets, and well-framed business questions will see vastly superior results.[2] It really is one of the three important ingredients that any company needs to have in place to fully benefit from AI (next to a Datagraph and AI-intelligence).
Back to Supply Chain: How This Applies to Business AI
In line with our article about context and the guide on AI in Supply Chain (download here), we gave the AI a context-rich prompt before starting our line of questioning, in a so-called chain of thought. This is what we provided first:
Back to Supply Chain: How This Applies to Business AI
In line with our article about context and the guide on AI in Supply Chain (download here), we gave the AI a context-rich prompt before starting our line of questioning, in a so-called chain of thought. This is what we provided first:
Prompt to give context:
We operate in the automotive sector with a company size of 50,000+ employees and annual revenues exceeding €50 billion. Our business model is a combination of manufacturing and B2B/B2C distribution, producing vehicles, components, and aftermarket services for both commercial and individual consumers.
We are subject to key regulations and compliance needs such as EU vehicle safety and emissions standards (Euro 7), supply chain due diligence regulations, sustainability directives (EU Green Deal), and international trade laws (WTO agreements, Brexit-related regulations).
Additionally, we comply with ISO 9001 for quality management and ISO 14001 for environmental management.
We serve markets in Europe as our primary region, with strong operations in Germany, France, Italy, Spain, and the UK. Beyond Europe, we have a global supply chain with key suppliers in Asia (China, Japan, South Korea), North America (US, Mexico, Canada), and emerging markets in South America and Africa.
This requires us to manage a highly complex and interdependent supply chain, balancing just-in-time (JIT) production, multi-tier supplier risk management, geopolitical uncertainties (US-China trade relations, EU sanctions), and sustainability challenges such as reducing Scope 3 emissions in our logistics network.
This realistic and structured example ensures that the AI model can retrieve and generate relevant, context-aware insights—whether for AI-driven supply chain optimization, risk management, or regulatory compliance analysis. But let’s focus on resilience.
Once given this context, we can start asking questions to the LLM of choice, and here comes the impressive part..
Example real-life data Supply Chain Resilience
This is already good stuff but we have to look beyond the actual output. The important lesson in my opinion is that the system is capable of generating this kind of output based on very limited information. It is still a test.
Let’s take it one step further. What if we give AI additional information about the type of product (product group level) linked to the supplier base? This happened.
A first, pretty good risk analysis for our global automotive company with a broad manufacturing and supplier footprint across Europe, North America, and Asia. So after uploading this additional information. What impressed me is that these models are capable of linking all these elements of information together and come up with a solid answer. We are not there yet, but we are very close having an assistent that we can rely on.
Conclusions
Now, this AI had clear constraints, objectives, and a defined problem space, allowing it to generate a more precise, strategic, and actionable answer. This is why companies that structure their data and questions effectively will extract more value from AI. The theoretical and empirical evidence overwhelmingly shows that AI delivers better results when provided with structured context. AI is only as good as the knowledge and structure we provide it. Treating AI as a black box leads to inconsistent results, but providing domain-specific, structured context makes it vastly more effective. The real competitive edge in AI adoption isn’t just about using the latest models, but about structuring the right data, knowledge, and business questions. And it goes beyond prompt engineering—it’s about knowledge engineering. AI needs structured information, just like humans do.Want to Know More?
In this guide we present some examples that give you insight in the value of giving context, a strategy called In-Context-Learning or Context Prompting. Start simple and take it from there, with adding data, context and training your AI.
Impact of AI Agents in Modern Business
The landscape of AI agents has advanced from basic chatbots to sophisticated entities managing complex tasks across industries. By 2024, these agents were at the heart of strategic shifts in enterprise operations, symbolizing a new era of efficiency in automation. This article examines the evolution of AI agents, their current roles, and their strategic implications for enterprises, offering insights crucial for future adaptability and innovation.
Advancements in Autonomous Workflows
Autonomous workflows have transformed AI agents, enabling them to tackle complex, multi-step tasks without human input. This shift, from traditional automation to more independent AI systems, allows real-time decision-making and dynamic responses across industries. In fields like customer service and legal technology, these workflows boost efficiencies, reduce costs, and enhance user experiences.
Some Examples
In customer service, AI agents streamline operations significantly. Companies like Klarna use AI systems to handle two-thirds of customer inquiries, boosting cost-efficiency and reducing resolution times. Automated agents cut operational costs by 30% to 80% compared to traditional human agents. Gartner predicts that by 2030, half of service requests will be autonomously resolved through AI systems, underscoring a shift towards self-service automation and its impact on customer interactions. [Source: Destination CRM].
In customer service, AI agents streamline operations significantly. Companies like Klarna use AI systems to handle two-thirds of customer inquiries, boosting cost-efficiency and reducing
resolution times. Automated agents cut operational costs by 30% to 80% compared to traditional human agents. Gartner predicts that by 2030, half of service requests will be autonomously resolved through AI systems, underscoring a shift towards self-service automation and its impact on customer interactions [Source: Destination CRM].
In legal technology, firms like Harvey leverage autonomous workflows to refine legal processes. These systems manage multi-step tasks—such as contract drafting and reviewing—adhering to compliance standards to enhance accuracy and efficiency. Automating routine legal tasks
not only boosts productivity but also frees legal professionals to engage in strategic work [Source: InTouch CX].
Danny Hillis Legacy: AI is moving so fast, which is why your strategy should be long-term.
Danny Hillis, a pioneer in AI, parallel computing and creator of The Connection Machine, understood something decades ago that many executives are only now coming to terms with. Technology does not evolve in stable, predictable increments—it moves in waves of rapid obsolescence and reinvention.
The past teaches us that every major technological revolution—the printing press, the steam engine, the internet—was first underestimated and then misunderstood by those who clung to outdated ways of thinking. AI is no different. AI is evolving faster than most companies can keep up with. Any AI investment made today will be outdated in a year. So what should executives do? My take on this is that treating it as a one-time transformation is not the way to go.
Brace yourself for constant change. That is at least how i’m preparing myself. Adopt an iterative, experimental mindset, embracing “strategic decay”—the idea that any AI capability has a limited shelf life, and businesses must plan for continuous renewal. I think that the worst decision is to wait for stability in a field that will never be stable.
The companies that thrive in the AI era will be those that experiment relentlessly, test rapidly, and build learning systems that adapt in real time. You don’t need a perfect AI strategy today—you need a framework that allows you to stay ahead despite inevitable obsolescence. You don’t need to have a perfect AI strategy today. You need a learning system that keeps you ahead despite inevitable obsolescence and a framework for testing output. If it fails, fail quickly.
If we look at what a few of the pioneers in the field of AI and parallel computing gave us. I came across the work of Danny Hillis, an pioneer in AI and parallel computing, but what makes him particularly interesting ni this context is his work on long-term thinking. He helped build the 10,000-Year Clock, designed to keep time for ten millennia, forcing people to think beyond short-term cycles.[3] His ideas are the perfect counterbalance to today’s AI hype cycle, where businesses assume that whatever is state-of-the-art now will be relevant in five years. Hillis would tell us: Plan for a future where everything changes—because it will.
Most AI strategies are fragile because they assume today’s AI is the final form. But just as no one runs a business on 1990s software, no one will rely on today’s AI in five years. Instead of chasing trends, leaders must design for continuous renewal:
- No permanent AI solutions—foster a culture of experimentation.
- AI isn’t the end goal—treat it as an ongoing adaptation.
- Beyond tools, invest in AI integration for long-term agility.
A key reminder: AI is an evolving tool, not a fixed investment. Your strategy must account for continuous obsolescence—standing still isn’t an option.
The Connection Machine [1] developed by Danny Hillis in the 1980s, was one of the first attempts at massively parallel computing, mimicking the brain’s structure. Instead of a single processor, it used millions of interconnected units in a 20-dimensional hypercube.[2]
Parallel with Today’s AI
One of the most striking parallels between the Connection Machine and today’s AI revolution is the shift from linear, rule-based computing to massively parallel processing. Modern deep learning models, like transformer architectures (which power GPT models and other LLMs), rely on parallelism—processing massive datasets across thousands of GPUs to identify patterns and relationships.
Hillis believed that the future of AI wouldn’t be about one central brain, but about interconnected intelligence—many small systems working together. This idea is more relevant now than ever. LLMs like GPT-4 and Gemini operate by integrating vast networks of data and learning in parallel. The kind of intelligence Hillis foresaw.
The Connection Machine was revolutionary, but it was quickly overtaken by newer technologies. This is a powerful lesson for businesses today. AI is not a static investment—it is constantly evolving. The AI models we use today will be outdated in 12–24 months, just as the Connection Machine was quickly surpassed by more efficient computing paradigms. Hillis understood that technology has a built-in expiration date, which is why he later focused on long-term thinking. His work with the Long Now Foundation emphasizes that businesses shouldn’t think in 1–2 year cycles, but in decades.
Lessons
What I learned from the work of Hillis and his colleagues is that you should not treat AI as a fixed investment. Instead, build a culture of ongoing AI adoption. Don’t assume today’s models will last. The companies that thrive will be those that can adapt to new AI generations quickly. What you are doing now with AI will be silly in a years time, but the question is where do you stand in a year? Ever tried, ever failed, no matter, try again, fail again, fail better[4]
[1] Hillis, W. Daniel (1989a). “Richard Feynman and the Connection Machine”. Physics Today. 42 (2): 78. Bibcode:1989PhT….42b..78H. doi:10.1063/1.881196. Retrieved 30 June 2021. Danny Hillis and Sheryl Handler founded Thinking Machines Corporation (TMC) in Waltham, Massachusetts, in 1983, moving in 1984 to Cambridge, MA. At TMC, Hillis assembled a team to develop what would become the CM-1 Connection Machine, a design for a massively parallel hypercube-based arrangement of thousands of microprocessors, springing from his PhD thesis work at MIT in Electrical Engineering and Computer Science (1985).
[3] Connection Machine : https://en.wikipedia.org/wiki/Connection_Machine#cite_note-WDHmit86-3 https://longnow.org/seminars/02014/jan/21/long-now-now/
https://www.youtube.com/watch?v=k8mX_prIllI
[4] Samuel Beckett.
In-company RAG
Start building you In-company RAG system and learn how to train your model. Learn the basic setup and common pitfalls when it comes to in-company AI.
Create a safe environment | the sandbox
We’ve seen firsthand that creating an environment where small, scrappy teams can innovate without being slowed down by endless reviews is key to unlocking AI’s potential.
How to deploy In-Company AI
Building a Company-Specific Vertical AI: A Practical Guide for C-Level and Supply Chain Leaders. Why Companies Are Exploring Vertical AI.
What we Learned building our Vertical
Going from an abstract vision of “AI that works for supply chain” to a functional, compliant, production-ready AI stack is harder than it sounds
When numbers start talking
Our founder Bas Groothedde meets our In-Company AI Agent AVA. Great stuff.
Giving your AI context in the supply chain
Context-prompting, an important tool to give an llm context. There is strong theoretical and empirical evidence that providing AI with better context improves its performance.
The Evolution of AI Agents
The landscape of AI agents has advanced from basic chatbots to sophisticated entities managing complex tasks across industries.
This is World interview featuring Professor Yann LeCun
LeCun brings clarity to the current discourse on AI with a number of thoughtful, remarks and putting it into perpective. A must-watch for anyone navigating the future of artificial intelligence.
AI is moving so fast, which is why your strategy should be long-term.
Danny Hillis, a pioneer in AI He understood something decades ago that many executives are only now coming to terms with. Technology does not evolve in stable, predictable increments.
The future of AI is not just language models. It’s causal reasoning.
Judea Pearl, is another true pioneering in artificial intelligence. But he is a little bit different. And not on the bandwagon if LLMs. Teach machines to understand the question why.
Intelligence is not what we think it is
In our series about AI, we came across the work of Professor Herbert “Herb” A. Simon, one of the founding figures in artificial intelligence.
Why the human factor remains essential in the Digital Era
No matter how advanced technology becomes, human insight remains essential in managing real-world complexity.
The legacy of Judea Pearl. The future of AI is not just language models. It’s causal reasoning.
To Build Truly Intelligent Machines, Teach Them Cause and Effect
In my own journey to better understand what AI—and especially language models—are capable of, I’ve found myself looking to the past. What can we learn from the pioneers in this field before we look forward? Turing is an obvious figure to study. Danny Hillis, though perhaps less widely known, also stands out—especially given the current pace of change.
Judea Pearl is another true pioneer in artificial intelligence. But he’s a bit different. He’s not on the LLM bandwagon. In fact, Pearl argues that AI has been stuck in a decades-long rut. His prescription for progress? Teach machines to understand the question why.
Pearl brings forward a powerful argument: artificial intelligence must move beyond pattern recognition and correlation to truly master causal reasoning. He criticizes modern AI—particularly deep learning—as being limited to what he calls “glorified curve fitting.”
He believes that true intelligence requires the ability to ask and answer “why” questions—enabling machines to reason about interventions and counterfactuals (what would happen if circumstances were different). This is the foundation of causal inference, a critical yet missing piece in most of today’s AI systems.
Modern AI—especially systems based on machine learning—is highly effective at detecting patterns in data. These systems learn statistical associations: when X happens, Y often follows. But they struggle with a more profound question: Does X cause Y?
This distinction becomes crucial when moving from passive prediction to active decision-making.
Imagine a model that notices people who carry umbrellas are more likely to get wet. A traditional AI might conclude: carrying an umbrella causes you to get wet. Without understanding that rain causes both, the model could suggest avoiding umbrellas to stay dry—a dangerous misstep. This is precisely where causal inference becomes essential.
“All the impressive achievements of deep learning amount to just curve fitting. We can’t trust a system that doesn’t understand cause and effect.” Judea Pearl.
Pearl’s Legacy : the Ladder of Causation
Judea Pearl, a Turing Award winner, reshaped the conversation around artificial intelligence. He introduced the ladder of causation, which defines three levels of reasoning. The first is association, or seeing—recognizing statistical patterns, such as the observation that people who exercise tend to be healthier. The second is intervention, or doing—asking what happens if we take a specific action, like wondering whether starting to exercise will make us healthier. The third is counterfactuals, or imagining—considering what would have happened under different circumstances, such as asking whether we would have gotten sick if we had exercised last year. Pearl’s work challenges us to move beyond curve-fitting. He shows that causal reasoning is not only possible—but necessary—for building truly intelligent systems. His ideas push AI toward understanding, not just recognizing.
The most fruitful way to think of intelligence is as the ability to answer ‘What if?’ questions.
Most AI systems today operate only at the first level of Pearl’s ladder: recognizing associations. Pearl’s work enables us to move to the second and third levels—intervention and counterfactual reasoning—which are essential for building truly intelligent systems. With this capability, AI could diagnose problems by identifying why a failure occurred, plan actions by determining what should be done to prevent it, adapt to new situations by assessing whether a model will still work in a different context, and provide explanations by justifying its decisions through causal reasoning. In sectors like healthcare, public policy, and economics, this isn’t just useful—it’s essential.
As Pearl himself puts it provocatively:
An AI system that cannot reason causally is not intelligent.
References
The Book of Why (Pearl & Mackenzie, 2018)
Causality: Models, Reasoning, and Inference (Pearl, 2009)
Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., & Bengio, Y. (2021). Toward causal representation learning. Proceedings of the IEEE, 109(5), 612-634.
Peters, J., Janzing, D., & Schölkopf, B. (2017). Elements of causal inference: Foundations and learning algorithms. The MIT Press.
LeCun, Y. (2022). A path towards autonomous machine intelligence. Retrieved from https://openreview.net/pdf?id=BZ5a1r-kVsf
Hartnett, K., (2018), To Build Truly Intelligent Machines, Teach Them Cause and Effect, Quanta magazine,https://www.quantamagazine.org/to-build-truly-intelligent-machines-teach-them-cause-and-effect-20180515/
In-company RAG
Start building you In-company RAG system and learn how to train your model. Learn the basic setup and common pitfalls when it comes to in-company AI.
Create a safe environment | the sandbox
We’ve seen firsthand that creating an environment where small, scrappy teams can innovate without being slowed down by endless reviews is key to unlocking AI’s potential.
How to deploy In-Company AI
Building a Company-Specific Vertical AI: A Practical Guide for C-Level and Supply Chain Leaders. Why Companies Are Exploring Vertical AI.
What we Learned building our Vertical
Going from an abstract vision of “AI that works for supply chain” to a functional, compliant, production-ready AI stack is harder than it sounds
When numbers start talking
Our founder Bas Groothedde meets our In-Company AI Agent AVA. Great stuff.
Giving your AI context in the supply chain
Context-prompting, an important tool to give an llm context. There is strong theoretical and empirical evidence that providing AI with better context improves its performance.
The Evolution of AI Agents
The landscape of AI agents has advanced from basic chatbots to sophisticated entities managing complex tasks across industries.
This is World interview featuring Professor Yann LeCun
LeCun brings clarity to the current discourse on AI with a number of thoughtful, remarks and putting it into perpective. A must-watch for anyone navigating the future of artificial intelligence.
AI is moving so fast, which is why your strategy should be long-term.
Danny Hillis, a pioneer in AI He understood something decades ago that many executives are only now coming to terms with. Technology does not evolve in stable, predictable increments.
The future of AI is not just language models. It’s causal reasoning.
Judea Pearl, is another true pioneering in artificial intelligence. But he is a little bit different. And not on the bandwagon if LLMs. Teach machines to understand the question why.
Intelligence is not what we think it is
In our series about AI, we came across the work of Professor Herbert “Herb” A. Simon, one of the founding figures in artificial intelligence.
Why the human factor remains essential in the Digital Era
No matter how advanced technology becomes, human insight remains essential in managing real-world complexity.
