Data

The Learning Question: What "It Learns From Your Data" Actually Means (and Who Else It Learns For)

Every vendor says the system learns from your corrections. Almost none says where that learning lives, who else benefits from it, or whether you can delete it. And the reassurance everyone offers — "we don't train on your data" — is the one row nobody was worried about.

AutomatR TeamOct 29, 202615 min read

In March 2023, Zoom revised its terms of service. Among the changes was a clause giving the company a broad licence over customer content, with a list of permitted purposes that reached as far as training and testing machine-learning models. For months nobody outside the company seemed to read it. Then, over the weekend of 5–6 August, someone did, and a post arguing that Zoom could now train AI on customer calls with no way to opt out spread widely online within a day. On the Monday, Zoom added a sentence to the clause promising not to use audio, video or chat content to train its AI models without consent, and its chief product officer, Smita Hashim, published a blog post explaining it. By the Friday the clause had been rewritten again, the consent language replaced by a flat statement. The terms in force today are dated 11 August 2023.

Nine months later it was Slack's turn. Its published privacy principles said that, to develop AI and machine-learning models, its systems analysed customer data — messages, content, files — and that a workspace that didn't want this could have its owner email Slack to opt out. On 17 May 2024 the page was read the way Zoom's clause had been, and the sequence repeated: a wave of alarm, a same-week response, a rewritten page — and three weeks later Adobe, after a terms-of-use update, said the same thing a third time. Three companies, one sentence each:

Three statementsEach company's own blog2023 – 2024
Zoom · 7 August 2023

“Zoom does not use any customer audio, video, chat, screen sharing, attachments, or other communications like customer content (such as poll results, whiteboard, and reactions) to train Zoom’s or its third-party artificial intelligence models.”

Slack · May 2024

“We recently heard from some in the Slack community that our published privacy principles weren’t clear enough and could create misunderstandings about how we use customer data in Slack.” … “Slack does not train LLMs or other generative models on customer data, or share customer data with any LLM providers.”

Adobe · 10 June 2024

“We’ve never trained generative AI on customer content, taken ownership of a customer’s work, or allowed access to customer content beyond legal requirements.”

Zoom's post as it reads today; the original 7 August wording was revised. Slack's post was first published 17 May 2024 and carries a 7 April 2025 revision date; quotes are from the current version. Adobe's is one of two posts, 6 and 10 June 2024.

Notice what the three have in common. None had done anything its terms didn't say. Each had used one word — train, develop, learn — to cover things its customers would have answered differently about had they been asked separately: does the product remember my workspace, does it improve everyone's model from my data, and who else can see what it kept? The customers weren't wrong to react. The word was doing three jobs, and the replies — notice how carefully each is worded — answer only one of them.

Every automation vendor uses the same word. If you buy document or process automation, you've heard "it learns from your corrections" in every demo. This piece takes the word apart, shows which of its three meanings a vendor's reassurance actually covers, and ends with four questions that separate them — the fourth of which almost nobody asks, and which is where the question actually lands.

81%
Organisations that say their generative-AI providers are transparent about data practices
55%
Organisations that require contractual terms defining data ownership, responsibility and liability
70%
Organisations that acknowledge risk exposure from the use of proprietary or customer data in AI training
SOURCE: CISCO 2026 DATA AND PRIVACY BENCHMARK STUDY — 5,200 IT, TECHNOLOGY AND SECURITY PROFESSIONALS, 12 MARKETS, FIELDWORK SEPTEMBER 2025 · PP. 3, 17–18

One survey, deliberately, and its newest edition. Four in five buyers believe their providers are open about what happens to data; barely half have written down who owns it; seven in ten know customer data finding its way into model training is a live risk. Meanwhile the blunt instrument has been put away: organisations with outright bans or hard limits on what can be entered fell from 28% to 7% in a year. The bans are gone. The fear isn't. It has moved from "don't use it" to "what exactly did we agree to?" — and the distance between trusting a provider and writing it down is the space this post is about.

First, understand what "learn" can mean(three mechanisms, one word)

A quick vocabulary, each term once. A model is the trained component that reads a document or a screen and produces an answer; it is a very large set of numbers, called weights. Training or fine-tuning changes those numbers. Inference is running the model unchanged on a new input: your document goes in, an answer comes out, the weights don't move. A tenant is your isolated slice of a vendor's system — your configuration, your data, your logs — walled off from every other customer's. With those four words, "it learns" can mean exactly three things.

It remembers, for you. The system keeps something specific to your business: the layout of your biggest supplier's invoice, the selector that healed last Tuesday, the fact that your GST number sits on page three. That memory lives in your tenant, as configuration or stored examples, not inside the model's weights. It makes the system better for you, it helps no one else, and it should leave when you do. This is what "the pipeline learns the layouts" meant in our Data-Readiness post and what "written back" meant in the Tuesday Problem. It also survives a model upgrade, because it was never in the model.

It improves everyone's model, using yours. Your documents or corrections are used to fine-tune a model every customer runs. This makes the vendor's product better for everyone, including your competitor on the same vendor, and it is hard to reverse. Researchers have extracted verbatim training data from language models — "hundreds of verbatim text sequences" from GPT-2 in 2021, gigabytes from open models and thousands of examples from a production chatbot in 2023 — and the reverse operation, machine unlearning, remains an open research problem; a 2024 Google DeepMind paper found that common evaluations "overestimate the privacy protection afforded by existing unlearning techniques." In plain terms: what goes into a shared model cannot be reliably un-learned short of retraining it. This is the meaning the Zoom and Slack pages were read as. And it is the meaning with the least certain future: when the vendor upgrades the model, whether your corrections carry forward depends on whether they kept the data and retrained on it — one more thing nobody mentions, and worth asking.

Nothing. The same model runs on every document, unchanged, and "it learns" is a sentence in a deck. Common, and not dishonest — the first meaning claimed without the mechanism. The tell: after a month of corrections, nothing you can point to is different.

So let's name them: the Three Meanings

By the end of this piece you'll be able to ask any vendor four questions that separate the three meanings — and know why the one reassurance every vendor volunteers answers the one question you weren't worried about.
MEANING 01
Yours
Per-tenant memory. Layouts, profiles, healed selectors, correction history. Lives in your tenant as configuration — call it a cache if you like, as long as you can see it and clear it. Benefits you alone.
MEANING 02
Everyone’s
Shared model training. Your data adjusts weights every customer uses. Lives inside the model. Benefits the vendor and every other tenant. Not reliably reversible short of retraining.
MEANING 03
Nobody’s
No mechanism. The model runs unchanged; “learning” is a slide. Lives nowhere. Benefits the sales cycle. Nothing to delete, because nothing was kept.
The test

Show me one thing the system learned from us last month, where it is stored, and delete it in front of me.A vendor who can do that has meaning one. A vendor who answers "we don't train on your data" has ruled out meaning two — and told you nothing about whether meaning one exists, or who else is in the room.

The fourth question: the model behind the vendor(who else receives your documents at inference)

"We don't train on your data" is a statement about the vendor. Most automation vendors — including us, in some deployments — run inference on models they did not build: a hosted vision-language model that reads the document, a hosted language model that drafts the workflow. Your document content travels to that provider at inference time, and whether the provider trains on it or keeps it is decided by the provider's terms for the tier the vendor bought — not by the vendor's website, and not by the vendor's contract with you unless someone made it flow down.

The terms are public. We'll use the four providers in our default hosted pipeline, because those are the ones we can be held to; any provider configured for a single customer's workflow is listed in that customer's agreement. On the question buyers ask first, three of the four say the same thing — paid and enterprise API traffic is not used for training by default — and the fourth says the opposite. They all differ on the line underneath: what is kept, for how long, and how you get that to zero. Read from each provider's own pages in October 2026:

The four hosted providers in AutomatR's pipeline and their data-use terms for business API traffic, as of October 2026: training default, what is kept and for how long, and the route to zero retention.
Provider(business / enterprise API)Training on your inputsWhat is kept, and for how longRoute to zero
Google Vertex AIData-governance page, updated 15 Oct 2026"Google won't use your data to train or fine-tune any AI/ML models without your prior permission or instruction"Gemini models "cache Customer Data" by default, deleted "no later than 24 hours after it's written"; Google "may log prompts to detect potential abuse and violations"Caching is switched off per project by setting disableCache; abuse logging is a separate exemption
OpenAIEnterprise privacy page, updated 8 Jan 2026"By default, we do not use your business data for training our models""may securely retain API inputs and outputs for up to 30 days"Zero data retention or modified abuse monitoring, on application, for eligible endpoints
Microsoft Azure OpenAIModels sold by Azure, page updated 19 May 2026Prompts, completions and training data "are NOT used to train any generative AI foundation models without your permission or instruction"An abuse-monitoring store and review process; the current page gives no number of daysModified abuse monitoring on approval, after which "the data storage and human review process described above is not performed"
AWS TextractAmazon Textract FAQUses by default"Amazon Textract may store and use document and image inputs processed by the service" to improve it, unless you opt outNo retention period stated; "You can request deletion of document and image inputs associated with your account by contacting AWS Support""You may opt out … using an AWS Organizations opt-out policy" — an account-level setting, not a default
QUOTATIONS ARE FROM EACH PROVIDER'S OWN PAGE, READ OCTOBER 2026. THESE PAGES CHANGE; RE-READ BEFORE RELYING ON ONE.

Three things to take from the table. First, "no training" is the same promise three times, and the fourth row is the reason this post exists: a document-reading service that, by default, may use the documents it reads to improve itself, with the off switch in an organisation-level policy someone has to have set. We found that row by checking our own pipeline. Ask your vendor whether they've checked theirs. Second, retention is four different answers — 24 hours of cache, 30 days, no number, no period stated — and none is zero without a setting or a form. Third, the promise is tier-specific. Google's consumer-facing terms say so: content submitted to the unpaid Gemini API is used "to provide, improve, and develop Google products and services and machine learning technologies," and "human reviewers may read, annotate, and process your API input and output." Same models, different line in the terms. Which line applies to your documents depends on which tier, on which platform, your vendor bought — and the lines move: Anthropic added a mandatory 30-day retention for its newest models on 9 June 2026, after years of "within 30 days."

So the Learning Question has four parts, not one: what does the vendor keep for me; what does the vendor train; what does the vendor's model provider — in the language of your DPA, its subprocessor — receive; and what does that provider keep or train on. The Four Exits called the third one Inference and put it in a matrix. This post adds the fourth, which that matrix couldn't see — and which no sentence on a vendor's website can answer, because it isn't the vendor's sentence to write.

Four questions, four ways a vendor can be set up(who decides each answer)

The takeaway

The row vendors reassure you about is the one they fully control. The rows that change hands are the ones they never mention.

Columns: how the vendor runs its models, which is the first thing to find out. Rows: the questions. Cells: who decides — not what the answer is, because that is in a document you should ask to see. Row 01, what's kept for me, isn't in the table because it reads the same in every column — vendor decides, and self-hosting the model doesn't touch it. Ask where it lives and whether it leaves with the account, whatever the set-up. The rows that vary start at 02.

Three of the four questions about data learning — what trains a shared model, what the model provider receives, what the provider keeps or trains on — against four vendor set-ups.
The questionVendor's own model(vendor-hosted)Hosted provider, paid tier(enterprise / paid API)Hosted provider, unpaid tier(consumer / free API — shadow use)Self-hosted model(your environment)
02What trains a shared modelVendor decidesTheir model, their call. Get the "no" in the contract.Vendor decidesThe provider doesn't train by default. The vendor still could, on models of its own.Provider tooGoogle's unpaid tier: used "to provide, improve, and develop Google products."Vendor decidesNothing leaves your environment — but the vendor could fine-tune inside it. Ask.
03What the model provider receivesSame partyNo third party. The vendor is the provider; rows 02 and 04 collapse into one.Provider receivesDocument content travels at inference. The Four Exits called this exit Inference and marked it Open for hosted SaaS.Provider receivesAnd may have humans read it.Nothing leavesInference on your hardware sends nothing to the model's developer.
04What the provider keeps or trains onSame partySee row 02.Provider decidesMostly no training by default — but not always (see Textract). Retention: 24-hour cache plus abuse logs (Vertex AI), "up to 30 days" (OpenAI), no number (Azure), no period stated (Textract).Provider decidesTraining on inputs is the default on at least one major free tier.Nothing leavesThere is no provider-side copy to keep.
VENDOR DECIDES — the answer is in the vendor's contract and architecture  ·  PROVIDER DECIDES / RECEIVES — the answer is in the model provider's terms for the tier in use  ·  SAME PARTY — vendor and provider are one company  ·  NOTHING LEAVES — no third party is involved. Provider terms as of October 2026.

Row 02 says "Vendor decides" in nearly every column: the row the vendor controls entirely, which is exactly why it's the one they volunteer a reassurance about. Rows 03 and 04 change hands column to column and are decided by a company that isn't in the room. The reassurance everyone offers is the row nobody was worried about — and row 01, the only one where learning is useful to you, is the one no reassurance mentions at all. The third column is in the table because it is already in your building: no vendor you'd shortlist runs production on an unpaid tier, but your staff might, and in Cisco's 2024 edition 48% of respondents said they had entered non-public company information into generative-AI tools.

Five objections

“We’ll put ‘no training’ in the contract.”

Good — for row 02. A clause with the vendor binds the vendor. Ask whether it flows down to the model provider, and ask to see the provider's terms for the tier actually in use, dated. In the vocabulary of The Off Switch: a clause is bound; a self-hosted model is enforced. And the terms are negotiable: in Cisco's 2026 study, 79% report their providers are willing to adjust contract terms or configurations to limit data exposure — if someone asks.

“We’ll self-host the model. Then nothing leaves.”

The strongest answer to rows 03 and 04, and the honest cost is already in the Four Exits matrix: it closes Inference and does nothing for Output or Action. And it leaves row 01 exactly where it was: the vendor's own memory of your business lives wherever the vendor put it, self-hosted model or not.

“The EU AI Act makes providers disclose their training data now.”

In part. Article 53(1)(d) requires providers of general-purpose AI models to "draw up and make publicly available a sufficiently detailed summary about the content used for training." Two limits: it binds the model provider, not the automation vendor deploying the model, and it summarises training content in general — it won't tell you whether your documents are in it, and says nothing about per-tenant memory or inference-time retention. Useful for row 04. Silent on rows 01 to 03.

“Isn’t this just subprocessor review?”

Largely, yes — and that's a point in its favour. Privacy and procurement teams have run subprocessor due diligence on cloud and payment vendors for years; GDPR Article 28 requires a processor's obligations to flow down to any subprocessor it engages, and every serious DPA carries the list. Ask for it. Then notice what the list doesn't show. A subprocessor list tells you who. It almost never tells you on which tier — and the table above is four providers whose terms turn on exactly that line: same company, same models, opposite answer depending on which API the vendor bought. It doesn't show the retention window, or whether a zero-retention exemption was applied or merely available, or that one provider's default is to use inputs unless an organisation-level policy says otherwise. Rows 03 and 04 are subprocessor review with the two columns the list leaves out.

“A vendor would say this.”

Yes. So take the four questions to our demo first: row 01 in writing, a deletion while you watch, and the dated provider terms behind row 04. If we can't produce all three, you've learned something about us before anyone else.

How AutomatR answers the Learning Question

Four rows, four answers, framed as what you'd see in a demo and marked as claims to be tested there. The principle is the one that runs through this series: the AI designs the workflow; a deterministic engine runs it. Everything the system learns about you is in the workflow. Nothing it learns about you is in a model.

01 · What’s kept for you
In your tenant, in the workflow
Learned document layouts, supplier profiles, healed selectors and correction history are stored per workflow, in your tenant, where you can see them in the workflow definition and the run log. They are removed when the account is.
02 · What trains a shared model
Nothing of ours — and one provider we had to switch off
The language and vision models the pipeline calls are used as their providers ship them, instructed with prompts; AutomatR does not fine-tune them on customer documents, so no document, field value or correction changes the weights of a model another customer runs. One provider’s default would have broken that sentence on its side of the line — Textract’s, in the table above — which is why card 04 states its opt-out rather than promising to check.
03 · What the provider receives
A deployment setting, not a policy
Which model reads your documents is set per deployment and recorded in the run log. Hosted: document content goes to one or more of Google Vertex AI, Azure OpenAI, OpenAI and AWS Textract, on business API tiers under AutomatR’s own accounts — the table above is their terms, and which of them a given workflow calls is in its definition. On-premises: inference runs on a self-hosted model and nothing leaves your environment. This is the Inference row of the Four Exits matrix, and it agrees with it.
04 · What the provider keeps
Stated, not “ask us”
Textract: the AWS Organizations AI-services opt-out is in force on every account that calls it, applied before this post went live. Vertex AI: no training; the default 24-hour cache is disabled on our projects. OpenAI and Azure OpenAI: no training by default; retention per each provider’s default, no zero-retention exemption applied. The same four lines appear in the data-processing agreement, dated, and we re-read the four pages quarterly because they move.
TENANT STORE · LEARNED ARTEFACTWORKFLOW 14 · SUPPLIER INVOICESDELETABLE
artefact
layout_template · supplier “Meridian Packaging” · v3
learned from
6 corrections by queue owner, 12 – 19 Sep · fields: invoice no., GSTIN, line total
stored in
tenant://acme-ltd/workflows/14/layouts/
used by
this workflow, this tenant · shared-model training: none
model weights
unchanged — the layout is configuration the model is shown, not something it was trained on
on account deletion
removed with the tenant · provider-side abuse-monitoring copies expire on the provider’s window (row 04)
delete now
Available to the workspace admin. This is the control we ask you to press in the demo.
WHAT A LEARNED ARTEFACT RECORD SHOWS. AN ILLUSTRATION OF THE FORMAT, NOT A REAL TENANT — THE FIELDS ARE THE DESIGN; THE VALUES ARE INVENTED.
For the engineers on the buying committee — everyone else may skip this section
  1. Where learning lives. Per-tenant artefacts — layout templates, field maps, supplier profiles, healed selectors, correction history — are persisted as workflow configuration and run-log records in the tenant's store. Configuration is learned; model weights are never written to.
  2. Routing by deployment. Which model serves inference is a deployment setting, not a per-document decision: hosted deployments route to the contracted provider tier, on-premises deployments to a model inside the customer boundary. The run log records which.
  3. Provider settings. Business API tiers on AutomatR-owned accounts; Gemini models called through Vertex AI, not a developer API key; training exclusion by default on three of the four providers, Textract under an AWS Organizations opt-out; Vertex caching disabled per project; retention otherwise per each provider's default. The settings in force, with dates, are the four lines in card 04 and in the DPA.
  4. The delete path. Account deletion removes tenant configuration, learned artefacts and run history. What it cannot remove is a provider-side abuse-monitoring copy inside a retention window that hasn't elapsed — which is why row 04 quotes the window.

The AI designs the workflow; a deterministic engine runs it. The learning is in the workflow, not the model.

The Four Questions

Ask these in any vendor demo, including ours. Two minutes; they separate the three meanings.

  1. 01What do you keep for us?Show me one thing the system learned from our documents, and where it lives. Is it deleted with the account?
  2. 02What do you train?Does anything from our documents or corrections change a model another customer uses? Yes or no, in writing, for every model in the pipeline.
  3. 03Who else receives it?Which model providers see our document content at inference, in the deployment we’re buying, and on what tier?
  4. 04What do they keep?Show me that provider’s training-exclusion and retention terms for that tier, with the date, and tell me whether zero-retention is configured.

A vendor who answers question 2 before you've asked it, and can't answer question 4, has told you where to look.

Ask us the Four Questions

Bring a pilot set of documents. We'll show you what the system learned, where it lives — and delete it while you watch.

One last question

Of everything your automation has learned from your documents, how much could you see today — and how much could you delete?

Further reading: going deeper on data

SourcesZoom: terms revised March 2023 and the clause first discussed publicly over 5–6 August 2023, per contemporaneous reporting (The Register, 8 August 2023; Axios, 9 August 2023; AP); the originating post is not cited, so the spread is described, not located. Zoom, "How Zoom's terms of service and practices apply to AI features," by Smita Hashim, Chief Product Officer, published 7 August 2023, quoted as it reads today; the original wording referred to consent and was later revised. Zoom's terms of service currently show an effective date of 11 August 2023. The pre-change clause text is not quoted here; it is available only through archived copies and secondary reports. Slack: TechCrunch, "Slack under attack over sneaky AI training policy," 17 May 2024. Slack, "How Slack protects your data when using machine learning and AI," first published 17 May 2024 per contemporaneous reports, current version dated 7 April 2025. The May 2024 privacy-principles wording is described, not quoted. Adobe: "A clarification on Adobe Terms of Use," 6 June 2024, and "Updating Adobe's Terms of Use," 10 June 2024, quoted. Cisco 2026 Data and Privacy Benchmark Study, released 26 January 2026: 5,200 IT, technology and security professionals with data-privacy responsibilities across 12 markets, surveyed September 2025 by Sandpiper Research & Insights; 81% and 55% (pp. 3, 18); 70% (p. 3); bans and data-entry limits 28% to 7% (p. 17); 79% willing to adjust terms (p. 19). The 28% combines bans and data-entry limits and is not comparable with the 2024 edition's 27%, which counted bans alone (Cisco 2024 Data Privacy Benchmark Study, 2,600 security and privacy professionals, 12 countries, p. 22); the same edition reports that 48% of respondents had entered non-public information about their company into generative-AI tools (p. 21) — a share of respondents, not of organisations. Provider terms, all read October 2026 from the providers' own pages. Google Cloud, "Gemini Enterprise Agent Platform and zero data retention" (Vertex AI generative-AI data-governance documentation), last updated 15 October 2026. Google, "Gemini API Additional Terms of Service," effective 23 March 2026 — the unpaid-services clause quoted for the tier contrast. Amazon, "Amazon Textract FAQs" — storage-and-use, deletion-on-request and AWS Organizations opt-out statements quoted; see also "AI services opt-out policies," AWS Organizations User Guide. OpenAI, "Enterprise privacy at OpenAI," updated 8 January 2026, and "Data controls in the OpenAI platform" — quoted. Anthropic Privacy Center, "Data retention practices for Covered Models," policy effective 9 June 2026 — the basis for the aside on changing terms. Microsoft, "Data, privacy, and security for Models sold by Azure in Microsoft Foundry," last updated 19 May 2026 — quoted; the page no longer states a number of days for abuse-monitoring retention. Retention terms change often; each quotation is dated and should be re-read before relying on it. EU AI Act: Regulation (EU) 2024/1689, Article 53(1)(d), quoted. GDPR: Regulation (EU) 2016/679, Article 28(4). Memorisation and unlearning: Carlini et al., USENIX Security 2021; Carlini et al., "Quantifying Memorization Across Neural Language Models," ICLR 2023; Nasr et al., "Scalable Extraction of Training Data from (Production) Language Models," arXiv:2311.17035, November 2023; Hayes et al. (Google DeepMind), "Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy," arXiv:2403.01218; Triantafillou et al., "Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition," arXiv:2406.09073. The extraction papers study pre-training data; they support the claim that trained-in data can be extracted, not a measurement of extraction from fine-tuning data specifically. The learned-artefact record shown is an illustration of the record format; its values are invented. AutomatR's default hosted pipeline calls Google Vertex AI, Microsoft Azure OpenAI, OpenAI and AWS Textract under AutomatR's own accounts, per engineering, October 2026; the Textract opt-out and the Vertex AI cache setting in card 04 are account-level settings, applied and confirmed in each provider's console before publication, and dated in the DPA. The on-premises statement describes the self-hosted configuration. AutomatR's four answers are stated as claims for the demo.