What a Simple AI Chatbot Is Actually Made Of
The demo takes a minute to build. The version your customers actually talk to has a dozen working parts behind the chat box. Here is the whole list, in the order a message travels through them.
Open any AI tool, type a question, get an answer. It feels like one thing. So when a business owner hears that an AI chatbot for their company needs real planning, the reaction is often reasonable: the demo looked simple, so why is this a project?
This is the first post in a series called Under the Hood. Each one opens up a piece of the machinery behind business AI systems, in terms an owner can follow. We start with the whole map. Later posts take each part in depth.
What Happens When Someone Types a Message?
Follow one message from a visitor on your site. Here is the path, written as a request and response diagram:
Visitor's browser (chat widget in React)
| POST /api/chat { message, session }
v
Server function on the host (Vercel)
1. Spam checks: honeypot, time token,
Turnstile token, rate limit
2. Load the business facts and the
last few messages (database)
3. Build the prompt, call the model API
4. Check the answer, save it, write a log
5. If it is a real inquiry: email the
owner (Resend), queue a follow-up
|
v
Reply streams back to the browserThat is one message. Every box in it is a separate product or a separate piece of code, and every arrow is a place where something can fail. The rest of this post walks the boxes in order.
The Front End Is the Part You Can See
The chat window is a small web application. On a modern build it is usually written in React, inside a framework such as Next.js, so it loads fast, works on a phone, and can be used with a keyboard or a screen reader. It has to handle ordinary human behavior: someone closes the tab mid-answer, loses signal in a parking lot, pastes in a wall of text, or taps send three times.
It also has to show the answer as it is produced. Models generate text a piece at a time, and a good widget streams those pieces to the screen as they arrive, rather than leaving the visitor staring at a spinner.
A Server Function Holds the Secrets
The browser never talks to the AI model directly. If it did, the key that proves your account is allowed to use the model would sit in the page code, where anyone can copy it and spend your money. Instead the widget calls a function that runs on a server. The key lives there, stored as an environment variable: a named setting kept outside the code and outside the browser.
On a platform like Vercel, that function is a serverless function. It starts when a request arrives, does its work, and shuts down. You do not rent a server that sits idle at night, and you also inherit a few rules: time limits on how long a function may run, and a requirement that anything worth remembering gets saved somewhere else. Our next post covers that world. Preview versus production explains how a change gets from a laptop to a live site safely.
The Model Is One Call Among Many
The call to the model API is a few lines of code. Everything around it is the work. Four details matter:
- The prompt is code. The instructions that tell the model who it is, what the business offers, what it must never say, and when to hand off to a person are written, reviewed and kept in version control like any other source file.
- The model remembers nothing. Each call starts blank. The server has to resend whatever the model needs to know, within the model's context window, which is the limit on how much text it can consider at once.
- Output needs a shape. If the system must pull a name, phone number and job type out of a conversation, the model is asked to return structured data in a fixed format, and the server checks it before using it.
- Tools make it useful. Through tool use, the model can ask your code to look up open appointment times or save a contact. The model asks, your code decides whether to allow it, and your code does the work.
A Database Remembers What the Model Cannot
Conversations, contacts, business facts and settings live in a database. A common choice is Postgres, often through a hosted service such as Supabase, which adds sign-in and per-user access rules on top. The database also holds the knowledge the assistant draws on, such as your service area, hours policy and answers to common questions. When that material grows past what fits in a prompt, the system searches it and sends only the relevant pieces, a technique called retrieval. We take that apart in a later post, and the next-but-one post covers the database itself.
Email and Spam Defense Sit on Both Ends
When a conversation turns into a real inquiry, someone has to be told. That usually means an email, sent through a service built for it, such as Resend. Getting that email into an inbox instead of a spam folder depends on three DNS records named SPF, DKIM and DMARC, which prove to receiving mail servers that your domain authorized the message. A system that sends mail without them can look finished and still lose messages.
On the way in, every public form and chat box attracts bots. The stack on this site's own forms is a good picture of the minimum: a hidden honeypot field that humans never fill in, a signed time token that rejects submissions sent faster than a person could type, a Cloudflare Turnstile check verified on the server, server-side validation, and a rate limit per IP address. Any one of them alone is easy to get around. Together they remove most automated abuse before it reaches the model, which matters because every junk message that reaches the model costs money.
DNS Is the Phone Book Everything Depends On
Your domain name is a set of records that say where the website lives, where email goes, and which services may send mail on your behalf. A small mistake in one record can take email offline. A table of the kind of records involved looks like this:
Type Name Purpose
A @ where the site is served
CNAME www points www at the same place
MX @ where incoming email is delivered
TXT @ SPF: who may send mail as you
TXT selector._domainkey DKIM: signing key for mail
TXT _dmarc DMARC: what to do with failuresLogs, Alerts and the Tuesday Problem
Sooner or later a customer says the chat did something odd on Tuesday. Without logs, nobody can say what the visitor typed, what the model was sent, what came back, or whether the email went out. With logs, it is a short investigation. Good systems record each step with enough detail to reconstruct it, without storing more personal information than they need, and they alert a person when something breaks quietly, such as an email provider rejecting messages.
Then there is maintenance. Model providers retire older models and release new ones. Vendors change their pricing and their limits. Browsers update. A system that was correct at launch needs someone watching it afterward, or it slowly stops being correct.
What This Means When You Choose Who Builds It
None of these parts is exotic. The skill is in choosing them, connecting them, and keeping them working together. That changes the questions worth asking a builder:
- Where does the model key live, and who can see it?
- What stops bots and abuse from running up the model bill?
- Where are conversations stored, who can read them, and can you export them?
- How does a change reach the live site, and how is it undone if it goes wrong?
- What happens when the model gives a bad answer, and who is told?
- Who watches it after launch, and what do you own if the relationship ends?
A builder who answers these in plain words has built one before. If you want a second opinion on a quote or a proposal, contact us, or read about what an AI systems architect does and how we build.