Insight / AI and infrastructure
Published
Data sovereignty and local AI: what control really means
How to weigh hosted and local AI models across data use, model changes, infrastructure and operational control.
Where does your information go when someone asks an AI model a question? That is a useful starting point, but data sovereignty also covers who can access the information, how long it is retained, which jurisdiction applies and who decides when the system changes. A model running on your own hardware can give you more control over those choices. It also gives you more work to own.
Check the product and plan, not the provider’s name
Hosted AI services have different rules for consumer products, business workspaces and APIs. OpenAI says content from individual services such as ChatGPT may be used to train models, with an opt-out for new conversations. It says API and business product inputs and outputs are not used for training by default, unless an organisation opts in. OpenAI’s data-use policy sets out that distinction. Its API data controls also show why “not used for training” does not mean “never stored”: retention for abuse monitoring or application state may still apply.
Other providers make different commitments. Anthropic’s consumer guidance says consumer chats and coding sessions may be used for improvement when a user chooses to allow it, through certain safety reviews, or through other explicit opt-ins. These terms can change, so check the agreement, settings and data flow of the specific service you plan to use.
It would be misleading to say that every conversation with a hosted model becomes future training data. It would also be hard to prove that such data is the reason a consumer plan costs less than an enterprise contract. The defensible question is simpler: what uses of your inputs and outputs are permitted under this product’s current terms, and are they acceptable for the information involved?
A hosted model is a moving dependency
A provider can change model availability, features, usage limits and prices. Some changes come with notice and migration guidance. OpenAI’s API reference says model behaviour may change between snapshots and recommends pinned versions and evaluations. Its deprecation page documents retirement schedules and replacements. Pinning a version can make day-to-day behaviour easier to manage, but a pinned version may eventually be retired.
For a business workflow, keep a small test set of real tasks, record which model and configuration produced each result, and plan how you will approve a replacement. Review contracts for notice, retention, location and support commitments as well as price. The risk is operational: a change that looks minor to a provider may matter a great deal to your users.
Local control has an operating cost
Running a model locally can keep prompts, documents and outputs within infrastructure you administer and let you choose when to update model weights. That control is only real if the rest of the workflow follows the same boundary. Embeddings, search indexes, telemetry, backups, remote support tools and fallback APIs can all create separate paths for data to leave.
You will also manage hardware capacity, power, security updates, monitoring, failures and model quality. A smaller local model may be enough for extracting fields from familiar documents yet struggle with a broader research task. A hosted service may be the better fit where capability and fast experimentation matter more than infrastructure control. Many teams will use both, with a clear rule for which information can go where.
Before choosing, ask: Which data is sensitive? Where must it be processed and retained? Which tasks need predictable output? Who can operate the system when it fails? What happens when a model version changes? Our local language models work starts with those workflow decisions, then sizes the infrastructure around them.
Map the whole data path before choosing local or hosted
Consider a Melbourne operations team that wants to summarise internal service notes. The visible step is a person asking a model a question. The full path may also include file upload, text extraction, chunking, an embedding service, a search index, prompt assembly, model inference, logs, analytics, support access and backups. Each step has its own operator, retention rule and location. This is an illustrative workflow, not a description of a client’s system.
A useful review records, for each step: what information enters; who can read it; where it is processed and stored; how long it remains; how it is deleted; and what happens if the service fails. Redacted samples are enough for an initial assessment. Keep credentials, internal endpoints and live access details in private project documentation.
If the model runs locally but document parsing or embeddings use an external API, the workflow is only partly local. If a hosted model is approved for public documents but restricted for staff records, the application needs a clear routing rule that people can understand and test. The word “local” should describe the complete approved path, not just the machine holding model weights.
Turn the review into a decision
Choose a representative task and an agreed set of approved examples. Compare the quality, response time, operating effort and data boundary of local and hosted options. Write down the decision criteria before testing so a promising demo does not quietly override a non-negotiable data requirement.
If local delivery remains plausible, use the hardware requirements guide to understand memory and throughput, then the model fit calculator to shortlist configurations. If the data itself is poorly understood, a data discovery sprint can establish sources, ownership and access before model selection.