Why most in-house AI tools fail and what to do instead

image copyright © The wub - licensed under the Creative Commons Attribution-Share Alike 4.0 International license.

Introduction

In 2009 Thomas Thwaits attempted to build a toaster from scratch. Properly from scratch. Mining ore, smelting metals, extracting plastic and building components. What he ended up with was something that worked (for about 5 seconds) but is not something you’d want in your kitchen.

The aim of the project was to shine a light on the intricate web of global cooperation needed for everyday objects we take for granted.

It also illustrates another couple of very useful points:

• Most things are deceptively complex once you scratch beneath the surface.
• Just because you can build something, doesn’t mean you should.

These points are just as true for software as they are for toasters.

The rise (and fall) of AI software projects

The emergence of tools like ChatGPT and Claude mean that powerful AI functionality is available to all, and this has given rise to an explosion of in-house initiatives to build AI-enabled tools within companies.

Despite the potential on offer, many AI projects will hit the rocks and fail. Famously, MIT, in their State of AI in Business 2025 report, recently put the percentage as high as 95%.

Despite $30–40 billion in enterprise investment into GenAI, this report uncovers a surprising result in that 95% of organizations are getting zero return.

In our world – AI-powered verbatim coding software – we are also seeing this trend playing out. Currently, about once a week we speak to people who have grown frustrated with internal AI tools developed to analyze verbatim survey responses.

So, what’s going on here?

Reasons for failure

Here are our thoughts on why this is happening, and why companies struggle to “roll out their own” verbatim coding software:

• Poor understanding of workflow
  With all the hype around generative AI, it is a trap to think that simply wrapping a tool around an LLM will be enough for real researchers and coders. To achieve the required levels of accuracy, precision and relevance, you also need human oversight. To achieve that, you need a sophisticated user interface that allows people to refine and shape the AI output in any way they need to.
  
  For example, people need to relabel codes, merge codes, create nets, recode/code verbatims and so on. This is an order of magnitude harder than simply spitting out results from an LLM but is often an afterthought in in-house projects.

• Underestimating Requirements
  Unless you work hands-on with verbatim data and coding, it’s hard to understand all the detailed requirements and features needed to get the job done properly and efficiently. Data loading, quality checking, sorting, filtering, translating, delegating, exporting, analyzing, the list goes on.
  
  In-house developers, often working in silos, usually don’t find out about all this until it’s too late.

• Underestimating Maintenance
  Building software isn’t just about writing code. It’s about maintaining a living system that constantly evolves. People often overlook the fact that software is a bit like physical machinery – it needs ongoing maintenance to keep running. Unless it is properly looked after, “software rot” creeps in, it degrades and stops working.
  
  For AI coding software, models need to be retrained, databases need to be maintained, UI frameworks need to be updated, patches need to be applied and so on. The AI landscape is also changing very fast, keeping up with the latest developments quickly becomes a full-time job in itself.

Implications for businesses

The issues above mean that in-house development usually ends up being far more costly than originally anticipated. If the project fails, then all that money is wasted.

If the software lives on, then there are still some hidden costs to consider:

• Opportunity cost
  Every hour spent building a tool is an hour that could be spent building something that truly sets you apart. For example, you wouldn’t set about trying to build your own spreadsheet application or fire alarm system because this doesn’t help differentiate your business.

• Talent drain
  Recruiting, training, and retaining AI and product engineers is hard.
  It is then a waste of all that effort putting the team to work reinventing something that already exists.

• Frustrated users
  Quite often, in-house developed tools are sub-optimal, ineffective and not fit for purpose. Users may struggle on regardless, but ultimately this leads to frustration and negatively affects staff and morale.

Yes, but we’re different, we need something “bespoke”

Many companies believe that an internal build will produce software more “tailored” to their needs. But the reality is, mature SaaS products have been shaped by hundreds of client deployments, real-world feedback, and continuous improvement over years.

That accumulated learning — what we call product intelligence — is nearly impossible to replicate internally.

Why codeit is the smarter choice

• At codeit, we’ve been working with real-world researchers and coders for nearly 10 years. We’ve been through that learning curve, so the software is mature and packed with features that real users need.

• codeit, puts the user firmly in control and makes it easy for people to review, refine and edit any aspect of the AI output. Even better, any changes are fed back into the AI so it can learn and get better over time.

• Part of our job at codeit is to keep the software truly cutting edge. That means regular updates, new features and staying up to date with the latest AI technology.

Conclusion

Look again at the toaster above - it reminds us that just because you can build something from scratch, it doesn’t mean you should. The same goes for verbatim coding software. Building in-house tools is tempting, but the hidden complexity, ongoing maintenance, and risks usually lead to failure.

Mature SaaS products, on the other hand, are battle-tested. They’ve been refined over years of real-world use, shaped by feedback, and continuously improved — advantages nearly impossible to replicate internally.

The takeaway? Focus on what truly sets your business apart, not reinventing the wheel. Choose the right tools, empower your team, and leave the toaster-building to the hobbyists.

If you want to see how it is done then try codeit for free.