Most small businesses connecting AI to their systems are focused on what it can do. Fewer are thinking about what it can do wrong. Once an agent acts – posting a transaction, changing a record, sending an email – there's no taking it back.
Our previous blog covered external AI cyber threats. This part two covers the AI risks businesses create for themselves.
Risk #1: Sending more data to the model than it needs
SMEs connecting AI to live business data tend to make the same mistake – they trust the model too much. A useful way to think about it is to treat the model like an external person you know nothing about, and act accordingly.
Before connecting any AI tool to your business systems, define:
-
What data it can access
-
What it can write or action
-
What requires a human to review first
What goes into the model matters just as much as what comes out. A customer might include sensitive personal or financial information in a sales order email. If an AI is processing that email, information it doesn't need could be passed outside the business too.
The Australian Cyber Security Centre (ACSC) recommends checking where your data is processed, what is stored and for how long, and whether it's used to train AI models.
Only call on the LLM to do the heavy lifting
Running a local model such as Qwen, Llama or Mistral is one way to keep sensitive data within your own environment. But the hardware cost and technical complexity can put this out of reach for many SMEs.
The more practical approach is often to be surgical about what you send to an external LLM. Keep confidential information within your systems where possible and only send what the model needs to do the job.
With an ERP like SAP Business One, structured data gives you much more control over what information AI receives. Instead of passing a full report to the model, you can retrieve only the information required for the task.
It also means fewer unnecessary tokens.
Risk #2: Prompt injection – the risk most SMEs haven't heard of
Prompt injection is when a malicious instruction is hidden inside content your AI is reading – an email, document or web page. The model can treat that instruction as a command.
In June 2025, researchers demonstrated how a hidden instruction inside an ordinary email could exploit Microsoft 365 Copilot. When Copilot processed the email as part of a user's request, the hidden prompt could instruct it to access sensitive information and send that data to an attacker. The user didn't even need to open the email or click a link.
It's the elephant in the room for agentic AI. You can build the setup, test it and have everything working as intended – but if someone successfully injects a prompt and the AI has access to everything, the potential damage grows with it.
That's one reason prompt injection sits at the top of the list of risks for LLM applications – and why limiting what an AI can access matters from day one.
Connect AI to SAP Business One through the front door
How you connect AI matters just as much as what you connect it to.
For an ERP like SAP Business One, how AI connects to your ERP should be part of the setup from the start. AI integrations can use the SAP Business One Service Layer, allowing access to be managed through SAP Business One's existing authorisations rather than giving the integration unrestricted access to the underlying database.
This allows the AI to operate as a defined ERP user, with permissions controlling what it can see and do. Connecting directly to the underlying database can bypass those role-based controls.
Think of it as giving someone a key to a specific room rather than the whole building.
Don't over-trust the model – keep a human in the loop
AI can automate the work. That doesn't mean you can stop checking it.
The idea that you can set up a task, walk away and never look at it again isn't realistic. Even when nobody needs to intervene, someone still needs to make sure it's doing what it's supposed to do.
Only let it write drafts – not live transactions
If AI can create or change records in your ERP, let it create a draft rather than posting a live transaction. Someone can then review an invoice, purchase order or other transaction before it's committed to the system.
Log what your AI actually does
Don't just check whether the final result looks right. Log what the AI is doing in the background so you have visibility into how it got there.
Review regularly – don't set and forget
The higher the cost of failure, the closer the oversight should be. Financial and operational automations need more scrutiny than low-risk tasks.
The goal isn't to intervene every time. It's to know the AI is still doing what you intended.
AI still needs guardrails
AI can save time and make businesses more productive. But connecting it to customer records, financial data and live transactions introduces risks that don't exist when you're simply using an LLM to draft an email.
The answer isn't to avoid AI. It's to control what it can access, what leaves your systems and what it can do without approval.
Key Business Solutions helps businesses set up SAP Business One, including working out where AI can add value and how to introduce it in a measured, monitored and secure way.
Ready to explore where SAP Business One and AI could fit into your business? Start a conversation with our team today.
