AI Assistant

Someone is forty minutes into a piece of technical material at eleven at night. Line three of the example does not make sense. That is the whole problem, and it is smaller than it sounds: they do not need the topic re-explained, they need one line explained, right now, in the words the material just used.

What a product can usually offer instead is a queue. File the question, wait for someone, come back tomorrow. Or open a chat window and start from nothing: paste the code, name the page, explain what you already tried, describe what you expected to happen. By the time a confused person has finished writing that setup, they have spent more effort framing the question than the answer is worth. So most of them never ask. They skip the line, tell themselves it will make sense later, and carry the gap into the next thing they read, where it compounds.

This is the part that gets misdiagnosed. The failure is not answer quality. Models were already good enough to explain a base case clearly. The failure is the cost of asking, and a generic assistant bolted onto the side of a product makes that cost as high as it can possibly be, because it starts every single conversation from zero. It knows nothing about the product, nothing about the screen, nothing about the fact that this same person was stuck on the same idea two days ago. Every unit of context it cannot supply, the user has to type.

So the brief was never to add a chatbot. It was to delete the setup. The assistant should already know which page is open, which block the user is reading, and what they asked on Tuesday. When the question finally arrives, it should be able to answer something as short as why does this work.

Screen context is a scope problem long before it is a technical one. Knowing what is on the screen has to be written down as a list of things the assistant may read and a list it may never touch, and that list is a product decision, not an engineering detail. We drew it tightly: the product's own content surfaces only, nothing wandering into whatever a user had typed somewhere else. The visible context chip started life as a way of making that boundary legible, and ended up being the thing people notice first.

Memory makes mistakes sticky. A model that has decided someone is weak at recursion will over-explain recursion for weeks, and over-explanation is the fastest way to make a person stop asking questions. So confidence had to decay, recent evidence had to outweigh old evidence, and the user needed a way to say I have got this and be believed. A memory layer with no forgetting policy is not personalisation, it is an accumulating prejudice.

Latency stopped being a technical metric and became an emotional one. Every retrieval hop is another second a stuck user spends stuck, and stuck users close tabs. So the loop got split: answer first, out of context that was already resolved, then load the reference and generate the practice question behind it. They are reading by the time the slower parts land.

The hardest single call was deciding not to solve the exercise. Faced with a coding task the assistant can trivially produce working code, and every instinct in a product team pushes towards it, because it demos beautifully and it is precisely what users ask for. It also removes the only part of the exercise that was doing any work. We drew the line at explaining the idea and never writing the answer, which meant accepting that some people would find the assistant less useful than they wanted it to be, and saying so out loud in the interface rather than pretending the limitation was a capability gap.

And quizzing a frustrated person is genuinely risky. A check-back question at the wrong moment does not read as help, it reads as being tested by the one thing that was supposed to help. So it is one question, it is always skippable, and where the signals suggest someone is annoyed rather than curious, the assistant does not ask it at all.

I would have given users control over their own memory on day one instead of treating it as a follow-up. We built the model first and the ability to see it, correct it and clear it second, which is the wrong order for anything that claims to know you: the control is what makes the knowing acceptable, so it belongs in the first version rather than the next one. I would also have shipped the check-back question as an opt-in before making it the default, purely to learn where it lands well and where it irritates, rather than inferring that from complaints after the fact.