TL;DR: Before renting a whole machine to run an open AI model, rent a cheaper open model by the token. For instance, moving document-heavy work from GPT-5.4 to Llama 3.3 70B, both rented, takes $25.00 per million output tokens down to $3.55 that’s a 86% gone, nobody new hired. Rent the whole machine only for high-volume, document-heavy, always-on work, because its real cost is the five-person team, not the hardware. And the security fear is aimed at the wrong layer: the downloaded file can’t call home, but the license can hurt you.
What decides it is your org chart.
The question comes up in the same meeting at most enterprises now. Someone is presenting an AI proposal, a model with a Chinese parent company appears on a slide, and a voice from the end of the table asks: can we trust a model that came from over there?
Legal has a view. Security has a view. Everyone has a view, and nobody has checked.
Start with what’s on the slide. An open-weight model — Llama, DeepSeek, Mistral, Qwen — is a trained AI model you can download and run on your own computers.
The company that trained it published the finished product; anyone can hold a copy.
Which one are you afraid of?
When someone in that meeting says “the model,” they mean one of three things, and each carries a different risk.
The argument goes in circles because all three keep wearing the same name.
1. A file can’t call home
A model in the modern file format is a short header and billions of numbers, which means it can no more open a network connection than a spreadsheet can. (An older format did run code when loaded, which is why Hugging Face scans every upload for it and the industry moved on.)
Whatever the fear in the room is about, it isn’t this.
2. Everything that comes in the box
Model downloads come with loader code and setup scripts; Kimi K3’s own instructions ask you to switch on a setting that means “run whatever code came with this download” — a routine ask across labs, American and European included. The loading tools have had real holes too, including one in PyTorch’s loader that let an attacker’s program run on your machine (CVE-2025-32434). The scale makes it an industry-wide problem, not a curiosity: security scanners had flagged roughly 352,000 unsafe or suspicious issues across 51,700 models on Hugging Face by April 2025, and Microsoft’s own catalog notes say plainly that “not all models available through the model catalog are scanned.”
Treat a model download like any other software package, because that’s what it is.
3. Banned the file, kept the app
DeepSeek’s own privacy policy says prompts, uploaded files, and chat history may be stored on servers in China. That’s real and documented in the company’s own words — and it attaches to DeepSeek’s service, not to a DeepSeek file running behind your firewall with outbound traffic blocked. Companies get this backwards: they ban the passive file while employees paste customer data into the consumer app.
Behavior is a separate question, and the evidence there is model-specific and not favorable to open models.
The US National Institute of Standards and Technology (NIST) tested DeepSeek R1-0528 and found agents built on it were 12 times more likely to follow hijacking instructions than the leading closed models it was compared against.
That’s damning — about that one model. So I recommend you test the model you’re deploying, not its nationality.
The danger is in the paperwork
Now the part that can actually bite.
While the room argues about the downloaded file/model, the exposure that would interest a general counsel sits in the license attached to it.
Under Meta’s Llama terms, as served through Microsoft’s platform, the customer may have to cover Meta’s losses and legal costs from third-party claims arising from its own use — backwards from every software contract you’ve signed. The same terms bar any company headquartered in the European Union from the multimodal Llama models, and require every product built on one to display “Built with Llama.”
Mistral defends its customers in the right way around but the protection falls away to the extent a claim arises from changes made by anyone other than Mistral. So the more you change the model, the less of the shield covers the thing you built. And changing the model, tuning it on your own data, is usually the whole reason you wanted it in the first place.
The permissive licenses offer no shelter either, because DeepSeek’s and Qwen’s terms hand you the file “AS IS” — no warranty, no defense. And Microsoft’s copyright promise, where Microsoft defends customers against copyright claims over AI output, covers a named list of its own products.
Open models you run yourself aren’t on the list.
Most of these aren’t open source, whatever the slide says. The Open Source Initiative examined Meta’s license and rejected it for restricting who may use the model and what for; DeepSeek’s and Qwen’s terms do qualify.
Open weights means you can download it. Open source means more, and the distinction is important.
So the meeting spent its energy on a file that can’t act, and skipped the document that decides who pays the lawyers.
So what decides it
None of this settles whether to run these models on a dedicated machine, because each risk has a fix that never touches the machine question. Vet model downloads the way you vet any other software and the download risk is handled; keep sensitive work off the consumer app and the website risk is handled too.
The license risk is the stubborn one, since reading the terms doesn’t make them go away and Meta isn’t renegotiating with us. That one we price in and have to design around.
But whether the risks get vetted, blocked, or lived with, none of them tells you whether to rent the box.
What does is a question nobody in that meeting thinks to ask: how many people do you already employ to run things like this?
The real answer sits on the org chart, not in the model.
Five people and a box
Let’s start with the machine option.
On Microsoft Azure, a server carrying eight of NVIDIA’s H200 chips (the big graphics processors (GPUs) that AI runs on) rents whole for $27,238 a month on a three-year deal at Azure’s published August 2026 prices.
Tokens are the word-fragments models read and write, and the billing unit for all of this. If you run it flat out at the speeds a million output tokens costs about 36 cents. MLPerf, the industry’s audited speed test, has measured for this class of machine, on the same-size model a generation back.
The other option is renting the same class of model by the token: a provider hosts the open model on its own GPUs and you pay through an API, the same way you already pay for GPT-5.4 — no machine, no serving software, no pager, just a meter. That meter reads $0.71 per million tokens for this class of model.
Those two rates don’t compare directly, and the gap between them is where the type of work takes over: the per-token meter runs on every token the model reads as well as every token it writes, while your dedicated machine reads inputs on hardware you’ve already paid for.
So for document-heavy work, the kind that reads four words for every one it writes, the machine pays for itself once it’s about 10% busy. That sits far below the 60 to 80% figure everyone cites as the watermark.
Then you count the people, because a machine like this is a set of jobs.
Someone runs the serving software.
Someone owns rollouts and rollbacks when a new model version lands.
Someone builds the tests that prove quality held after every change.
Someone vets what you downloaded.
And someone carries the pager at 3am, because the per-token service comes with an uptime promise and your machine comes with whoever is on call.
Five people for those jobs, priced at federal wage data with benefits, cost $98,037 a month. The machine costs $27,238. The people are 78% of the bill.
Count the people in and the bar jumps: now the machine has to be doing real work about 46% of the time before it beats the meter — still on that document-heavy, four-words-in for one-out work. Put differently, out of every ten hours you’re paying for, the machine needs to spend more than four and a half actually answering requests, not sitting idle waiting for them.
That works out to roughly 46 requests arriving every second, day and night, weekends included, all year. Almost nothing inside a company runs at that pace.
Five people is my assumption, not a published number — nobody publishes the headcount for running one of these — so overwrite it with your own org chart.
If you already run a platform team — people whose day job is keeping servers, deployments, and monitoring alive — most of those five jobs are covered, and adding two engineers gets you the rest. For that company the bar drops from 46% to somewhere between 25 and 29%. The same discount goes to anyone who already built the quality tests and the traffic plumbing for the cheaper-model move I’ll get to next, because three of the five jobs on the list are exactly that work, and done once it doesn’t need doing again.
Those percentages translate into monthly volume you can check against your own bills. On that same document-heavy work, a company adding almost no staff breaks even once the model is writing about 9 billion tokens a month. A company hiring all five people from scratch needs closer to 35 billion — nearly 4 times as much work to justify the exact same machine — and adding two engineers lands you somewhere in between.
Same box, same rent; the hiring plan alone moves the break-even point by a factor of four.
Two companies with identical work get opposite decisions, and what separates them is the org chart.
The move almost nobody makes first
Before either company commits to a machine, there’s a move that captures most of the prize with no machines at all, because most of the saving everyone attributes to running the model yourself is available from renting a cheaper one.
Move document-heavy work from GPT-5.4 to Llama 3.3 70B — both rented by the token, no machines involved — and the price falls from $25.00 per million output tokens to $3.55. That’s 86% of the bill gone with nobody new needing to be hired.
What it takes is proof that the cheaper model clears your quality bar, and the plumbing to route traffic to it — real engineering, not a settings change. But the bar for that work paying for itself is low. Even pricing generous engineering time for the tests and the routing, it covers its own cost at about 1.37 billion output tokens a month, call it 1.8 requests per second — 1/25th of what the machine will turn out to need.
Salesforce published what this looks like done well.
A small model, tuned from the open GPT-OSS-20B, now does the intent-sorting behind its Agentforce product. By the company’s own measurements, it’s 30 times faster than the general-purpose models it replaced, with 1 to 2% better accuracy.
It shipped in two months, during a code freeze — which tells you this is routing work, not a platform build. And Salesforce kept the expensive model wired in as a fallback anyway; one of the largest AI teams anywhere still didn’t bet on the cheap model alone.
If you haven’t made that move yet, the machine question is premature.
The dedicated machine needs the same quality tests and the same routing plumbing the cheap move needs — plus five people and a three-year lease — and it needs 25 times the volume before it pays off. Jumping straight to the box means signing up for $27,238 a month in hardware while skipping the $3.55-per-million model that needed no hardware at all. That’s not a cost decision.
That’s spending money to avoid a two-month engineering project.
References:
Hugging Face Hub, Pickle scanning: https://huggingface.co/docs/hub/security-pickle
Kimi K3 model repository: https://huggingface.co/moonshotai/Kimi-K3/tree/main
NVD, CVE-2025-32434 (PyTorch
torch.load): https://nvd.nist.gov/vuln/detail/CVE-2025-32434Hugging Face blog, 4M Models Scanned: Protect AI + Hugging Face 6 Months In (2025-04-14): https://huggingface.co/blog/pai-6-month
Microsoft Learn, Data, privacy, and security for use of models through the model catalog: https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/concept-data-privacy
DeepSeek Privacy Policy: https://cdn.deepseek.com/policies/en-US/deepseek-privacy-policy.html?locale=en_US
NIST CAISI, Evaluation of DeepSeek AI models (2025-09-30): https://www.nist.gov/news-events/news/2025/09/caisi-evaluation-deepseek-ai-models-finds-shortcomings-and-risks
Microsoft Learn, Terms and conditions for specific Microsoft Foundry Models (page dated 2026-07-17): https://learn.microsoft.com/en-us/legal/microsoft-foundry/model-specific-terms
Microsoft Learn, Customer Copyright Commitment required mitigations: https://learn.microsoft.com/en-us/azure/ai-foundry/responsible-ai/openai/customer-copyright-commitment
Microsoft Product Terms, Universal License Terms for Online Services — Customer Copyright Commitment (Covered Products scope): https://www.microsoft.com/licensing/terms/product/ForOnlineServices/all
Azure Retail Prices API (all Azure prices; accessed 2026-08-14): https://prices.azure.com/api/retail/prices
MLCommons, MLPerf Inference v6.0 results (2026-04-01): https://github.com/mlcommons/inference_results_v6.0
US Bureau of Labor Statistics, OEWS May 2025 wages and ECEC Q1 2026 compensation, retrieved via the BLS Public Data API (accessed 2026-08-14): https://www.bls.gov/oes/
Salesforce Engineering, Solving Real-Time AI Classification for Agentforce (2025-11-20): https://engineering.salesforce.com/solving-real-time-ai-classification-for-agentforce-how-single-token-prediction-delivers-30x-faster-agent-responses/
Salesforce News, How We Cut Inference Spend by Right-Sizing Our Models: https://www.salesforce.com/news/stories/cutting-inference-spend-by-right-sizing-models/
Mekala et al., quantization and long-context performance, EMNLP 2025: https://aclanthology.org/2025.emnlp-main.479/
OWASP GenAI, LLM03:2025 Supply Chain: https://genai.owasp.org/llmrisk/llm032025-supply-chain/
Microsoft Learn, Azure AI Foundry built-in evaluators: https://learn.microsoft.com/en-us/azure/foundry/concepts/built-in-evaluators
Microsoft Learn, AI gateway capabilities in Azure API Management: https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities
Uber Engineering, GenAI Gateway (2024-07-11): https://www.uber.com/us/en/blog/genai-gateway/