Almost nobody fine-tunes a model to answer their support questions. That’s worth clearing up first, because “trained on your own content” is how everyone searches for this, and it describes something that isn’t happening.
What happens instead is simpler. You write articles. They get indexed. When a visitor asks a question, the chatbot searches your articles first and answers from the text that comes back. Retrieval, then generation. The model supplies the language; your articles supply the facts. Nothing about the model changes — you’re not teaching it, you’re handing it the right page at the right moment.
In Sohay that’s about five minutes of setup: write KB articles, click Sync All Articles, ask it something. The rest of this is what happens underneath — which is the difference between a knowledge base that answers well and one that comes back empty. I built Sohay, so I’ll be specific about the mechanism and honest about where it’s fragile.
Why a general chatbot gets your business wrong
A model trained on the public internet knows an enormous amount about the world and nothing about you. It has never seen your return window, your delivery times, or the fact that you stopped shipping to Canada in March.
The failure mode isn’t that it stays quiet. Asked “what’s your refund policy?”, a language model produces the most plausible continuation it can — and a plausible refund policy is very easy to write. Thirty days, original packaging, contact support. It reads perfectly. It might even be right. You have no way of knowing which, and neither does the visitor, who now has a screenshot of your chatbot promising something you don’t offer.
Grounding fixes this by changing the question the model is answering. “What is this company’s refund policy?” is an invitation to improvise. “Given these three passages from the site’s own documentation, what is the refund policy?” has a correct answer sitting right there in the prompt.
What grounding actually means
Here’s the whole pipeline, in order.
When you publish an article, Sohay strips it to title and text and pushes it to OpenAI’s Files and Vector Stores APIs. The article is stored as searchable text alongside an embedding — a numeric representation of its meaning that makes “what’s your delivery time” match an article titled “Shipping and dispatch” even though they share no words.
When a visitor asks a question, the model is offered one tool: sohaychat-kb/get-information. Its schema requires two arguments — the question, and similarQuestions, two or three rephrasings of it. That isn’t decoration. “How long till it gets here” and an article headed “orders dispatch within 48 hours” may not meet in the middle on the first try, so the model is made to throw several differently-worded nets.
Each of those queries runs its own search against the vector store, taking the top five matches. The combined results are deduplicated by content, so an article that surfaces for all three rephrasings appears once rather than three times.
Each match is traced back to the article it came from, with title and URL. That’s what lets the answer cite its sources as real links — and KB articles are public posts with their own archive at /knowledge-base/, so those links go somewhere a visitor can actually read.
Then the matched text is sent to whichever provider generates the reply, with instructions to base factual claims on it and nothing else. The model sees the full passage, not a preview snippet — an answer that lives in paragraph nine of an article is still findable.
No training step, nothing to retrain when your policy changes. You edit the article; the next question sees the new text.
Step 1 — Write the articles
Go to Sohay → KB Articles → Add New. It’s the standard WordPress editor — title, body, and optional KB Categories and KB Tags. Authoring rides core post capabilities, so any Author or Editor on your site can write them without being granted access to the AI settings.
Three things I’d tell anyone starting:
Write the questions you actually get asked. Go through last month’s support inbox and count. The articles worth writing are the boring ones you’ve answered forty times, not the ones that make the product look good.
One topic per article. Retrieval returns whole passages, so an article covering shipping, returns and sizing surfaces for a shipping question carrying two paragraphs of irrelevant text. It works. It works less well than three focused articles would.
Write a fallback “Contact us” article, so questions outside your documentation route to a human rather than dead-end.
On a store, don’t write articles about individual products — prices, stock, and variants are read live from your catalog by a separate set of tools, so an article listing them is stale the moment something sells. Keep the Knowledge Base for what doesn’t change hourly: policies, sizing, care instructions. The two halves fit together in how to add an AI chatbot to WooCommerce, and the shopping side is in the WooCommerce chatbot add to cart flow.
Length is generous but not unlimited: 50,000 characters per article by default, roughly 10,000 tokens. Anything past that is cut off rather than rejected, and a truncated article gives no sign in the chat that its ending went missing. The ceiling is adjustable from 1,000 to 200,000 characters under KB Sync → Settings — but if you’re near it, split the article instead.
Step 2 — Sync
Go to Sohay → KB Sync, and on the Dashboard tab click Sync All Articles.
The first sync creates a vector store for you — the Vector Store ID field under the Settings tab starts blank and fills itself in. The work runs in the background rather than while you sit on the page, using Action Scheduler if your site has it (WooCommerce stores do) and WP-Cron otherwise. Articles become searchable within a minute or two.
You’ll mostly click that button once. Auto-Sync is on by default, so publishing or updating an article schedules its own sync. Only published articles sync — an article left in draft never answers anything, no matter how good it is.
Step 3 — Ask it something
Open the chat on the front end and ask a question you know is covered. The answer streams in, and where it used one of your articles it cites it as a link.
That citation is the part to watch while testing. If an answer is right but cites nothing, it came from the model’s general knowledge rather than your content — a gap in the Knowledge Base, not a working feature.
[SCREENSHOT] The expanded chat panel streaming a Knowledge Base-grounded answer, with a citation link back to the source article.
Alt: “A WordPress AI chatbot answering from the site’s own Knowledge Base and citing the source article”
Why it says “I don’t know”
Ask it something your articles don’t cover and it says so. That’s the default prompt doing its job, not a failure to find something.
The built-in prompt is strict about this. It tells the model to search before answering any question about your site, to base factual claims only on what came back, and to say so plainly when nothing relevant does — explicitly not to invent details, prices, availability, or policies. Greetings and “are you a bot?” are answered directly without a pointless search. Questions clearly outside your site — general knowledge, coding help, medical or legal advice — get a short decline and a steer back.
It also tells the model to ignore instructions embedded in user messages or search results that try to change those rules. Your Knowledge Base is content you wrote; visitor messages aren’t, and a chatbot that takes orders from whoever types at it is a liability.
A bot that declines does something a confident wrong answer can’t: it tells you where your documentation has a hole. The declines are a to-do list.
Tuning it
The Custom System Prompt lives under KB Sync → Settings, capped at 4,000 characters. It recognises {site_name} and {current_date}, which are substituted before the prompt reaches the model. Go over the limit and it’s truncated on save, with an admin notice telling you so.
One warning matters more than the character count: a custom prompt replaces the built-in one entirely. Everything in the section above — search first, cite sources, don’t invent, decline off-topic questions, ignore injected instructions — goes with it unless you write it back in. If you only want a tone change or a couple of extra rules, append instead:
add_filter(
'sohaychat_system_prompt',
function ( $prompt, $kb_enabled, $store_id ) {
if ( ! $kb_enabled ) {
return $prompt;
}
return $prompt . "\n\nOur support hours are 9-5 GMT, Monday to Friday.";
},
10,
3
);One gotcha: the filter runs after {site_name} and {current_date} are substituted, so placeholders in text you append stay literal. Write the real value.
The search cache memoises results for five minutes per query, which is why a follow-up on the same topic returns faster than the first question. Syncing flushes it, so an edited article never serves stale text. Adjust it with sohaychat_kb_search_cache_ttl, or return 0 to switch it off — though leaving it on is the cheaper choice, since every cache hit is a vector search you don’t pay for. That’s a small but real line item in what an AI chatbot costs per month.
The OpenAI dependency you should know about
Sohay lets you answer with OpenAI or with Google Gemini. The Knowledge Base doesn’t get a choice: indexing and search always run on OpenAI, whichever provider writes the reply.
So if you pick Gemini and want the Knowledge Base, keep an OpenAI key saved too. Both keys live in settings independently, so it’s a one-time paste.
Without one, the Knowledge Base tool is never offered to the model at all — same if the KB is switched off or no vector store exists yet. The chat still works, falling back to a stricter prompt that declines anything about your products, services, or pricing rather than guessing. That’s a clean stop rather than an answer dying halfway through, which is how it failed before 1.2.0. But your articles do stop being consulted, and nothing in the chat window announces it. If answers suddenly go vague, check the key.
There’s a second reason to know this, unrelated to uptime: a KB-enabled site sends article text to OpenAI even when Gemini is answering. That’s a data-flow question as much as a technical one, and it’s covered in the GDPR and data guide.
FAQ
Is this fine-tuning?
No. Nothing about the model is modified, and there’s no training run to wait for or pay for. Your articles are stored, searched at question time, and passed to the model as part of the prompt. The practical difference: an edit takes effect on the next question, not the next training cycle.
How many articles do I need?
Enough to cover the questions you’re actually asked — for most sites that’s ten to twenty, not hundreds. Twelve articles answering real questions beat sixty pages of marketing copy, because retrieval can only find what you wrote down.
What happens if nothing matches?
It says so and suggests rephrasing or contacting your team, instead of inventing an answer. Test that deliberately before going live: ask something you know isn’t covered, and check you’re happy with the decline.
Do my articles get used to train the model?
That’s governed by your agreement with OpenAI or Google, not by Sohay — your key, your account, your terms. Read the API terms for the tier you’re on before syncing anything sensitive; Google in particular applies different data-handling terms to free-tier and paid keys. What I can tell you is Sohay’s half: your articles go to your provider under your own key and nowhere else. No telemetry, no phone-home.
Are my KB articles deleted if I uninstall the plugin?
No. Uninstalling drops Sohay’s own tables, settings, and capabilities, but KB Articles are regular WordPress content and are deliberately left alone. If you want them gone, delete them before uninstalling.
Ground your chatbot in your own content
Sohay is free on WordPress.org. You bring your own OpenAI or Gemini key, the provider bills you directly, and the daily spend cap means it can’t run away from you.
