An internal AI assistant lets staff ask questions in plain language and get answers drawn from your own policies, SOPs and past work, with a link to the source document. To build one, you collect and clean your documents, index them for search, connect a language model that answers only from what it finds, and control who can see what.
Who actually benefits from an internal assistant?
Teams that keep asking the same people the same questions. If your operations lead answers "Where's the refund policy?" five times a week, you have a use case.
Typical good fits:
- New hires who need to learn processes without interrupting senior staff.
- Support and sales teams looking up product details, pricing rules or past proposals.
- Operations staff checking SOPs, supplier terms or compliance steps.
- Project teams searching old project notes to avoid repeating mistakes.
Poor fits: very small teams where knowledge lives in two people's heads and nothing is written down. There is nothing to index yet. Write the SOPs first.
An internal assistant is often the lowest-risk first AI project, because mistakes stay inside the business and staff can sanity-check answers. To see how it compares with customer-facing projects, read our overview of AI automation priorities for growing businesses.
Step 1: Which questions should the assistant answer?
Start by collecting 50 to 100 real questions your team asked in the last month. This list becomes your scope, your test set and your success measure.
Pull them from Slack or Teams threads, support tickets, onboarding notes and your own inbox. Group them by topic. You will usually find that a handful of topics, such as HR policies, product specs or client processes, account for most of the volume. Build for those first.
Write down, for each question, the document that holds the right answer. If no document exists, that is a gap to fill before launch, not something to expect the AI to work out.
Step 2: How do you prepare your documents?
Audit, de-duplicate and date your sources before indexing anything. An assistant fed outdated documents will confidently give outdated answers.
A practical clean-up pass:
- List every source: shared drives, Notion or Confluence, Google Docs, PDFs, help-desk articles.
- Archive anything superseded. Three versions of the leave policy means three possible answers.
- Give every live document an owner and a "last reviewed" date.
- Convert scanned images of text into real text where possible.
- Split very long documents into sensible sections with clear headings.
This step is unglamorous and usually takes longer than the technical build. It is also where most of the answer quality comes from.
Step 3: Which build approach suits you?
Pick based on how sensitive your data is, how many systems you need to connect, and how much control you want. There are three main routes.
| Approach | Typical cost | Strengths | Limits |
|---|---|---|---|
| Built-in AI in your existing workspace tool | Per-user subscription | Fast to switch on, respects existing permissions | Only searches that tool; limited customisation |
| Custom GPT or similar no-code assistant | Low monthly fee | Quick prototype with uploaded files | Manual file updates, weak access control, harder to audit |
| Custom retrieval-based assistant (RAG) | $10k–40k / ₹2–8 lakh / AUD 15k–60k (typical range) | Connects many sources, enforces permissions, cites sources, fits your workflow | Needs a build and ongoing maintenance |
RAG stands for retrieval-augmented generation: the system first retrieves relevant passages from your documents, then asks the model to answer using only those passages. Our explainer on how RAG lets AI answer questions from your own business data covers the mechanics without jargon.
If you're unsure whether a no-code assistant is enough, read our comparison of a custom GPT versus a custom AI app. A common path is to prototype with the cheap option, prove people use it, then build properly.
Step 4: How do you index and search the content?
Documents are split into chunks, converted into searchable vectors, and stored with metadata such as owner, date and access level. Good chunking is the difference between precise answers and vague ones.
What to get right:
- Chunk by meaning, not by character count. Keep a heading with its paragraphs, and a table with its title.
- Store metadata. Department, document type, review date and permission group let you filter results.
- Use hybrid search. Combine keyword search (great for product codes and names) with semantic search (great for "how do I…" questions).
- Re-index automatically. A nightly sync, or an update triggered when a document changes, keeps answers current.
Step 5: How do you stop it making things up?
Force the assistant to answer only from retrieved passages, always show its sources, and let it say "I don't know". These three rules prevent most of the trouble.
Write instructions that tell the model to:
- Quote or link the source document for every answer.
- Say plainly when the documents don't contain an answer, and suggest who to ask.
- Never guess at numbers, dates, prices or policy details.
Then test it. Run your 50–100 real questions and grade each answer as correct, partly correct, wrong or "correctly declined". Fix the failures, usually by improving the source document or the chunking, and re-run. Keep this test set and re-run it whenever you change models or add sources.
Step 6: How do you handle permissions and privacy?
The assistant must never show someone a document they couldn't open themselves. Build permission checks into retrieval, not just into the chat interface.
- Mirror existing access groups from your drive or wiki where possible.
- Exclude highly sensitive sources, such as payroll or medical records, unless there is a strong reason to include them.
- Log questions and answers, and decide who can see those logs.
- Use AI providers whose business terms exclude your data from model training.
- Check obligations under the privacy laws that apply to you, such as GDPR, Australia's Privacy Act 1988 or India's DPDP Act 2023.
Our guide to role-based access and permissions in business software explains how to design access groups that stay manageable as you grow.
Step 7: Where should the assistant live?
Put it where people already ask questions. Adoption collapses if staff must open yet another tab.
Common homes are a Slack or Microsoft Teams bot, a panel inside your intranet or internal dashboard, or a search bar on your help centre for staff. Add a thumbs-up and thumbs-down on each answer; that feedback tells you which documents need work.
Roll out to one team first. Give them two weeks, read the feedback, fix the obvious gaps, then widen access.
What does it cost to run after launch?
Expect three running costs: AI model usage, hosting, and maintenance of roughly 15–25% of the build cost per year. Model usage is usually modest for internal tools, because staff ask far fewer questions than customers do.
The hidden cost is content ownership. Someone must keep documents current. Assign owners by topic and put a quarterly review in their calendars, otherwise quality decays slowly and trust goes with it.
A simple timeline for a focused first version:
| Phase | Typical duration | Output |
|---|---|---|
| Question collection and source audit | 1–2 weeks | Scope, test set, cleaned documents |
| Build and indexing | 2–4 weeks | Working assistant with citations and permissions |
| Pilot with one team | 2 weeks | Feedback, fixes, accuracy report |
| Wider rollout | Ongoing | Additional sources and teams |
Durations vary with the number of sources and integrations involved.
Frequently asked questions
How is this different from just searching our shared drive?
Search returns a list of files; an assistant returns an answer with the source attached. It also understands questions phrased the way people talk, not just matching keywords. For staff, that usually means seconds instead of minutes of digging.
Can it learn from conversations in Slack or email?
It can index them, but be careful. Chat threads contain opinions, drafts and outdated decisions alongside facts. Many teams index only curated channels or summaries, and keep formal policies as the authoritative source.
Which AI model should we use?
Any of the major commercial models from providers such as OpenAI, Anthropic or Google will handle internal Q&A well. Choose based on data-handling terms, cost and how easily it integrates with your stack. Design the system so you can switch models later.
How accurate will it be?
That depends mostly on your documents and your test process, not the model. Measure accuracy against your real question set before launch and set a bar your team agrees is acceptable. Treat "correctly declined" answers as a success.
Can a small business afford this?
A no-code assistant or your workspace's built-in AI can cost very little to try. A custom build makes sense when you have many sources, sensitive permissions or heavy daily use. Start small and upgrade when usage proves the value.
Turn your documents into answers
GRWW builds internal tools and AI assistants that sit inside the systems your team already uses; our services page covers how we approach them. If you'd like help scoping a first version, request a fixed-price proposal and share the questions your team asks most.