Human in the loop: oversight, quality, data, compliance

The workflows from the previous chapter are running: the machine drafts, a human approves, and the sample shows errors are isolated. That state does not hold on its own. The catalog changes, somebody adds an exception to a rule, a vendor updates a model, and a few months later the same workflow returns output nobody would sign off on today. Oversight is what separates automation that removes work from automation that quietly lowers the quality of your catalog. Five things need setting up: approval gates, quality measurement, drift detection, accountability for the output, and the rules for handling data alongside your transparency duties.

Where to put an approval gate, and where not to

A gate costs time, so you do not put one everywhere. A process where a human signs off on every step is not automation, it is retyping work from one window into another. The place for a gate is set by two things multiplied together: the risk of a single error and the scale it spreads across. Ask four questions at each step of the workflow. Who sees the error first: the customer, the sales channel or someone on the team. How many items does one bad rule touch. How long does it take to undo the change. What does a single error cost once you count the complaint, the return and the labour of handling it.

Scale can invert the risk judgment entirely. A typo in one description is a rounding error. The same typo introduced by a rule across twelve thousand items and pushed to three channels is a week of work. That is why the tightest gate does not belong where the machine generates a single output, it belongs where somebody changes a rule, a dictionary or a mapping. One such change has more reach than a thousand individual results.

The gate is set by risk times scale, not by how good the tool feels.

The reverse holds just as strongly. There are steps where approving every item makes no sense, because the risk is low and quality is measurable immediately: draft classification, a ticket summary for an agent, a first pass at flagging duplicates. A periodic sample and a way to roll back are enough there. What that boundary looks like in customer service, where the machine answers repetitive questions about order status, is covered in the piece on automating order status replies.

The control sample: how to draw it and what to record

A sample answers one question: is quality still where you last measured it. For the answer to mean anything, the selection cannot be convenient. The first thirty items on a list are usually your bestsellers, which is the part of the catalog that is best described and most often fixed by hand. Draw in strata: a few items from each major category, a few from the newest deliveries, a few from the slow-moving tail. Keep the deliberately hard cases in a separate pool and score them separately, because folding them into one number depresses the result and destroys comparability between cycles.

Score pass or fail across a few independent dimensions, and write the criteria down before you look at anything. Four dimensions cover most tasks: consistency with the source data, completeness of required elements, correct format, and compliance with brand rules and any requirements specific to the category. Whoever scores the sample should not know which items were already touched by a human, because that knowledge shifts the judgment.

Record the result in the same place and the same shape every time. A minimal log fits in one sheet: date, workflow name, rule and prompt version, name and version of the model used, sampling method and sample size, score per dimension, list of error types, and the decision taken. This log is the only way to see drift, and the only evidence that oversight actually exists rather than being something asserted in a meeting. One measurement speaks about today and nothing else.

Quality drift: why March worked and June did not

A workflow rarely breaks overnight; it slides. There are three sources and they are worth keeping apart, because each calls for a different response. The first is the catalog: a new category, a new supplier with a different data format, a seasonal range the rules have never seen. The second is the rules: an exception added here, a prompt tweaked there, an attribute dictionary edited, each change sensible on its own and together forming a set nobody has tested as a whole. The third is the tool: a vendor ships a new model version or changes a default, and the same prompt returns something different from last quarter.

Three mechanisms are enough to catch it. A recurring sample on a fixed rhythm, for example monthly and on every larger upload to a channel. A control set, meaning a dozen or so items with a known, approved result that you push through the workflow after every rule change and every notice of a model version change; a divergence on the control set is an earlier signal than a drop in your sample. And external signals you do not have to generate yourself: channel rejections, returns for mismatch with the description, repeated service questions about something that should have been in the description.

Decide in advance what happens when a threshold is crossed. A sensible rule reads: on systematic errors the workflow drops one level of oversight and stays there until two consecutive samples come back clean. Without a written response, the argument about whether things are bad enough restarts from zero every time, and usually ends with the topic being postponed.

Who owns the result

Every workflow has one owner by name, not a team and not a department. That person keeps the sampling cycle, maintains the log, and makes the call to suspend the automation. Changes to rules, dictionaries and prompts get a separate approver, ideally somebody other than the author of the change, plus a short changelog: what changed, when, why, and what the sample showed afterwards. It is the same habit you already apply to pricing changes, and for the same reason: the reach of the change is wider than the reach of the person making it.

Separate the roles at review time as well. The person who configured the workflow should not be the only one judging its quality. This is not about distrust; it is that the author of a rule knows its intent and reads the output through that intent, so they consistently see more sense in it than a customer will.

Data: what you may send to an external model

The rule is singular: send as much data as the task requires and not one field more. To write a product description a model needs product attributes, not an export of your customer database. To propose a category mapping it needs category names and examples, not the full table of purchase prices and margins.

As a rule, customer personal data does not go into external models, and where a process genuinely requires it, that happens on a basis agreed beforehand and with the consequences understood, not as a by-product of a quick test. Treat sensitive counterparty data the same way: commercial terms, purchase prices, margins and contract text. In practice this comes down to four habits: work on copies and samples rather than the production database, strip identifiers the task does not need, forbid pasting supplier attachments into tools that are not on the approved list, and maintain that list of tools in use together with an answer to three questions: where the vendor processes the data, how long it is retained, and whether it is used to train models. Building the list is often the hardest part of the whole clean-up, because it usually turns out there are more tools in play than anyone assumed.

Compliance: transparency and personal data

For an ecommerce store, the part of the AI Act, that is Regulation (EU) 2024/1689, that matters most is transparency. It comes down to two disclosure duties: AI-generated content has to be labelled appropriately, and a user has to know when they are interacting with an AI system rather than a person. In practice that touches chat and automated replies in customer service, plus material published in your own channels. Separately, customer personal data falls under the GDPR, so every new tool in a service process is simultaneously a decision about data processing. This is informational material, not legal advice; confirm the scope of your own obligations with a lawyer.

To be straight about it: in a typical store this is mostly a disclosure duty and process hygiene, not a revolution in how you work. Three things usually suffice to begin with: an unambiguous notice in the channels where a machine answers, a written policy on labelling generated content, and the tool list above with an answer to what data reaches each one. Neglected, though, this area can stall a project at the worst possible moment: during an audit by a large partner, or on entry into a new market, when nobody in the company can say where service data ends up. We collected the points to check in our compliance checklist.

The oversight described here has one more property: it is far easier to sustain when it is built into the platform rather than bolted alongside it in spreadsheets and calendar reminders. How you build and maintain a store where gates, logs and permissions are part of the system is the subject of the final chapter.