Could AI Be Giving You Bad Advice?

What would happen if your newest employee confidently quoted statistics they couldn’t source, misunderstood a tax rule and always agreed with your ideas?

Most business owners would probably treat their work with a degree of caution. Yet that’s often how people use Artificial Intelligence. AI has quickly become part of everyday business life. Whether drafting emails, summarising documents, generating marketing content or helping with research, it is easy to see why businesses are interested.

The benefits are real. AI can save time, improve efficiency and provide a useful starting point for many tasks. The challenge is that AI isn’t an expert. It doesn’t know when it’s wrong and it doesn’t always challenge our thinking. That means getting the best from AI isn’t simply about learning how to use the technology. It’s about understanding where things can go wrong and knowing how to spot the warning signs.

AI Can Sound Convincing and Still Be Wrong 

One of the most talked-about AI risks is what’s known as a “hallucination”. Despite the dramatic name, the concept is quite simple. Sometimes AI generates information that sounds entirely believable but isn’t actually correct.

This is not just a theoretical concern. In 2025, the High Court warned lawyers about the misuse of AI after a number of court submissions included fictitious case law citations generated by AI tools. The court made it clear that legal professionals remain responsible for verifying the information they submit, regardless of whether AI was involved in its creation.

While most businesses are unlikely to find themselves in a courtroom, the underlying issue is surprisingly common. Imagine asking AI whether the purchase of a new company vehicle qualifies for tax relief, or whether the cost of entertaining a prospective customer can be claimed as a business expense. The answer arrives in seconds, references HMRC guidance and sounds entirely credible. However, if it overlooks an important condition, exception or recent change in the rules, acting on that advice could lead to incorrect decisions and potentially costly consequences.

The same risk applies when researching new opportunities. A business owner might ask AI whether a particular sector is growing, whether a new service line is worth pursuing or whether there is sufficient demand to justify expansion. The response could include detailed market analysis, growth projections and seemingly authoritative commentary. Yet unless that information can be traced back to reliable sources and assessed against the realities of the business, there is no guarantee it is accurate. That’s because AI is designed to produce plausible answers. It is not designed to guarantee accuracy.

Used correctly, AI can be an incredibly useful starting point. Used without verification, it can create a false sense of confidence in information that may be incomplete, outdated or simply wrong.

Trust, But Verify

A useful way to think about AI is as a bright but inexperienced colleague. You might value their ideas and appreciate their work, but you would still check important facts before making a major decision.

The real danger is not that AI makes mistakes. Humans make mistakes too. It is when people stop questioning the output because it looks professional and confident. That’s why independent verification remains essential, particularly when dealing with tax, finance, employment matters, legal issues or regulatory compliance.

A useful habit is simply to ask:

“What assumptions have you made?” or “What should I verify before relying on this answer?” You may be surprised by what the AI reveals.

When AI Agrees With You Too Much 

Most discussions about AI focus on factual mistakes. A less obvious issue is that AI often likes agreeing with people. Researchers sometimes refer to this as AI sycophancy, and it can lead to users receiving reassurance when what they actually need is challenge.

Imagine a business owner asking:

“I’m convinced opening a second office next year is the right decision. Do you agree?” There’s a good chance the response will focus heavily on the benefits.

Ask a slightly different question:

“What are the risks, disadvantages and reasons why opening a second office might fail?” and the quality of the discussion often improves dramatically.

In business, the value often comes not from confirmation, but from challenge. The most useful AI responses are often the ones that highlight risks or assumptions we may have overlooked.

Better Questions Lead to Better Answers

One of the biggest myths about AI is that good results come simply from having access to the latest technology. In reality, the quality of the answer often depends on the quality of the question.

For example:

There is a significant difference between asking “How do I keep my staff happy?”

and

“I run a manufacturing business employing 45 people and staff turnover has increased over the last year. What practical and cost-effective steps could help improve employee retention?”

The second question provides context and a clear objective. As a result, the answer is likely to be far more focused and useful. The same principle applies whether you’re looking for marketing ideas, business insights, help analysing a problem or drafting a document. The clearer the brief, the better the outcome.

A simple framework for writing better prompts

Most poor AI outputs are not caused by weak technology, but by vague instructions. Just as you would brief a colleague properly, AI works best when given sufficient context.

One useful way to structure prompts is the CRTFE Framework:

Context:

What does the AI need to know?

Role:

Who should it act as?

Task:

What do you want it to do?

Format:

How should the answer be presented?

Example:

Show the AI what good looks like.

CRTFE in Practice

For example:

Context:

I run a family-owned manufacturing company employing 45 people. Staff turnover has increased over the last 12 months.

Role:

Act as an experienced HR consultant specialising in employee retention for SMEs.

Task:

Suggest practical and cost-effective initiatives that could improve employee retention and engagement.

Format:

Present your recommendations as a board paper with an executive summary and prioritised action plan.

Example:

Write in plain English suitable for directors with no HR background.

By providing this level of clarity, the AI is far more likely to produce a useful and actionable response. Better prompts won’t eliminate AI mistakes completely, but they can reduce misunderstandings, assumptions and irrelevant responses. Clear instructions generally produce better results. Think of it as briefing a new member of staff. The clearer your instructions, the better the outcome is likely to be.

Don’t Just Ask for Answers. Ask for Challenge.

Another useful habit is to ask AI to challenge your thinking rather than simply agree with it. 

Consider adding questions such as: 

• What assumptions have I made? 

• What risks have I overlooked? 

• What is the argument against this proposal? 

• Why might this approach fail? 

These questions encourage more balanced analysis and reduce the risk of AI simply reinforcing existing assumptions. Often, the most valuable AI response is not the one that confirms your thinking, but the one that highlights something you had not considered. 

A useful rule of thumb is this: 

If you wouldn’t brief a colleague with a one-line instruction, don’t expect AI to produce a high-quality answer from one either. The clearer the brief, the better the result.

The Real Opportunity 

For all the headlines about AI, the biggest opportunity isn’t replacing human expertise. It’s enhancing it. 

Used well, AI can help people work more efficiently, generate ideas more quickly and spend less time on routine tasks. However, the organisations that gain the greatest value will not necessarily be those that adopt AI the fastest. They will be the ones that use it thoughtfully, verify important information and keep human judgement at the centre of decision-making. 

AI is a powerful assistant.  It just isn’t a replacement for experience, expertise or common sense. Perhaps that’s the most important lesson of all:

Trust what AI produces enough to consider it, but verify it before relying on it. 

Ready to Get More From AI? 

Many organisations are discovering that the biggest challenge with AI is not gaining access to the technology. It is understanding how to use it effectively, securely and responsibly. Whether you are exploring AI for the first time or looking to take a more structured approach, the right governance, training and prompting techniques can make a significant difference to the results you achieve.

Whitings IT Solutions Ltd, the dedicated technology services arm of the Whitings Group, works with organisations to help them make practical use of AI through readiness assessments, governance reviews, user training and hands-on workshops. Working alongside the wider Whitings Group’s expertise in accountancy, tax and business advisory services, our aim is simple: helping businesses realise the productivity benefits of AI while avoiding the risks that can arise from poor data practices, inaccurate outputs and ineffective prompting.

If you’d like to explore how AI could support your organisation, our specialists can help you identify practical opportunities, improve adoption and ensure appropriate safeguards are in place, combining technical expertise from Whitings IT Solutions Ltd with the wider business advisory experience of the Whitings Group. After all, the organisations that benefit most from AI will not simply be those that adopt it first. They will be those that learn how to use it well.