Many existing SAP customers automatically associate the use of AI with a complete move to the cloud. However, AI with SAP on-premises is already possible today. Whether SAP R/3 or S/4HANA: Analytical and operational AI scenarios can already be integrated into existing system landscapes in a targeted manner.
Therefore, the key factor is not whether your entire SAP landscape is already running in the cloud. What matters much more is which specific use case adds value and how it can be meaningfully integrated into your existing architecture.
Here at XEPTUM, we’ve implemented exactly these kinds of focused AI scenarios over the past few months. This article shows you what options are available, where the limitations lie, and why you don’t have to wait until you’ve fully migrated to the cloud to get started.
How can you get started with AI using SAP on-premise?
The appeal of the grand AI vision is understandable, but you don’t have to start big. A step-by-step approach often makes more sense: a specific pain point, a clearly defined scenario, and a measurable quick win. That’s exactly why a complete move to the cloud isn’t necessary.
Generally speaking, it is worth distinguishing between two types of AI:
- Analytical AI “Talk to Your Data”: Analyses, forecasts, and anomaly detection.
- Operational AI: Agents that act directly within business processes, such as entering documents or maintaining master data.
Both approaches can be integrated with existing SAP systems, but in different ways.
How can analytical AI be used with SAP on-premise?
Today, established data platforms offer powerful ML and LLM capabilities, whether within the SAP ecosystem, such as with SAP Business Data Cloud, SAP Databricks, and SAP AI Foundation, or on non-SAP platforms like Databricks, Microsoft Fabric, or Snowflake.
For existing SAP customers, one key point is that the connected tools are initially indifferent to whether the source system runs on-premise. Using appropriate data connectors, an R/3 or S/4HANA system can be integrated without having to fundamentally change the existing system landscape.
Analytical AI can thus be a relatively quick first step for many companies.
How can AI agents operate within SAP on-premise processes?
Operational AI is even more exciting and for many companies, it’s the real game-changer. Here, AI not only analyzes data but also actively supports specific tasks and processes.
In addition to SAP Joule, there are numerous agent, copilot, and orchestration platforms that can serve as entry points for end users. The spectrum ranges from Microsoft Copilot, OpenAI ChatGPT, and Anthropic Claude to custom solutions and our own HeinzAI.
To learn how Business AI can be integrated into existing SAP environments and what role SAP technologies and agent-based AI play in this process, visit Business AI & Autonomous Enterprise.
The key advantage of existing SAP systems is that they already feature numerous traditional interfaces, such as OData, IDoc, and RFC. These interfaces have long been used to perform operational tasks, such as processing incoming invoices, entering quotes and orders, or updating master data.
It is precisely these interfaces that can become a tool or skill for an AI agent. To achieve this, the interface fields are enriched with semantic information. Simply put: Technical fields are given human-readable descriptions, while framework conditions, posting specifications, and business context are also provided.
This allows the agent to understand which functions are available to it, what information it needs, and under what conditions it is permitted to perform an action.
AI doesn’t require a complete move to the cloud: Even existing on-premises SAP systems can be connected to AI via existing interfaces – for quick wins, automated processes, and a step-by-step path to the Autonomous Enterprise.
What role does MCP play in integrating AI agents into SAP?
The standardized protocol for such a connection is the Model Context Protocol (MCP), often described as “USB-C for AI.”
An MCP server serves as a translation layer between a traditional SAP interface and an AI agent. Instead of fundamentally restructuring the existing SAP landscape, the agent can use existing functions in a controlled manner as tools.
Depending on its complexity, a focused MCP server can be implemented with manageable development effort. However, this requires not only technical expertise but also a deep understanding of the underlying business process: What is the agent allowed to do? What rules apply? What context does it need?
This is exactly where we at XEPTUM bring both worlds together: technical expertise in SAP and AI, and the necessary understanding of business processes.
This makes targeted AI quick wins for specific pain points a realistic possibility even in existing on-premises SAP environments.
Which AI scenarios are already working in practice?
Scenarios that we at XEPTUM have already implemented demonstrate that this approach works not only in theory:
- Order and Quote Entry via AI Agent
- Internal time tracking via natural language
Rather than building a comprehensive AI architecture from the ground up, the focus here is on a clearly defined process. This allows for an early assessment of the actual value added by AI and whether scaling up later is worthwhile.
What AI use case is there in your SAP landscape?
Together, we’ll identify specific use cases and explore how AI can be integrated into your existing SAP landscape in a cost-effective and practical way.
What are the limitations of AI with SAP on-premise?
As enthusiastic as we are about the possibilities, it’s also important to take a realistic view. Each operational agent tool may require its own MCP server or corresponding integration logic. If an agent needs to access many different tools and processes, integration, governance, and operations can become correspondingly more complex.
That isn’t a deal-breaker. However, it does illustrate why a focused approach makes sense: Instead of opening up as many processes as possible to AI at once, you should start where the benefits, technical effort, and risk are well balanced.
How does this fit into SAP’s AI strategy?
XEPTUM is an SAP Silver Partner and has been helping companies further develop their SAP environments for more than 25 years.
Learn more about our SAP partnership
SAP is also consistently driving forward the integration of AI agents into business processes. With Joule, SAP already offers an AI and agent solution that is deeply embedded in the SAP ecosystem and features numerous preconfigured skills. So far, the focus has been on cloud scenarios.
For existing SAP customers, however, this doesn’t mean you have to wait until you’ve fully migrated to the cloud to start using AI. In addition to your own MCP servers, the SAP ecosystem also offers ways to connect existing systems to AI agents.
Using the SAP Integration Suite, existing interfaces can be made available to agents as MCP tools without having to run a separate MCP server for each scenario. With Joule Studio and the Skill Builder, you can also develop your own skills that, for example, call existing OData interfaces with the necessary context. The various approaches can also be combined with one another.
We’ve explored in more detail how SAP defines the Autonomous Enterprise and the roles that humans and AI agents play in it in our article “AI Agents in the Workplace: People Remain in the Driver’s Seat in the Autonomous Enterprise”.
How can you get started with SAP AI the right way?
You don’t have to wait for a full migration to the cloud to leverage AI profitably. The key is to start with the right use case.
Together, we’ll identify the biggest pain points in your existing SAP processes, evaluate suitable AI scenarios, and develop a pragmatic approach that aligns with your current system landscape and your long-term SAP strategy.
Want to know which AI scenario can be implemented in your SAP landscape in the short term? Schedule a no-obligation consultation, together, we’ll assess your current situation.
