The fight against distillation in AI is eerily similar to the battle against Napster in the music industry. Shutting it down didn’t stop piracy—it just forced the industry to evolve. Suing and banning DeepSeek won’t stop model distillation either. Instead, we could build Distillation-as-a-Service on top of DeepSeek R1 and help countries and businesses develop their own frontier models at a fraction of the cost.


What Is Distillation and Why the Controversy?

Distillation is like a student learning from a teacher: instead of memorizing every detail, the student grasps key concepts and simplifies knowledge. In AI, distillation trains smaller models to mimic larger ones, making them faster, cheaper, and easier to deploy. This is exactly why DeepSeek is in the spotlight. Reports suggest it may have distilled OpenAI’s models, leading to OpenAI’s investigation and David Sacks, the "AI Czar," calling for legal action to shut it down. But history tells us that eliminating one player won’t stop the underlying trend—it will just push distillation underground or into the hands of new startups. OpenAI might enforce stricter licensing, pursue legal action, or introduce technical countermeasures, but the demand for building frontier models among countries and businesses will persist.

Startup idea: Distillation-as-a-Service

Rather than trying to destroy DeepSeek, why not use its advancement to build your own startup? Distillation isn’t going away, and models like R1 don’t prohibit it in their terms of service. There’s a growing need for country-specific, regulated industry, and device-optimized smaller models, and distillation is the fastest way to create them. Instead of going after DeepSeek, a Distillation-as-a-Service platform built on top of R1 could be an interesting startup idea. You help businesses fine-tune their own LLMs legally, tailored to their needs. It’s a market with real demand.



Distillation with OpenAIDistillation with DeepSeek
LegalityRestricted by OpenAI's ToS, which prohibits using outputs to train competing models.Allowed—R1’s ToS does not restrict distillation.
AccuracyLower, since OpenAI only provides logprobs, limiting the depth of knowledge transfer.Higher, since R1 provides logits, allowing for richer training signals.
CostHigher, as low accuracy increases training iterations and compute needs.Lower, since better accuracy means fewer training cycles and better efficiency.


Who Needs Distillation-as-a-Service?

  1. Governments need LLMs that run within their own borders. Storing citizen data on U.S. or Chinese servers is a non-starter. Governments also don’t want their citizens relying on AI that aligns with foreign ideologies.
  2. Device-optimized models are crucial for on-device AI, reducing size and reliance on cloud infrastructure.
  3. Regulated industries need models that comply with strict data privacy laws. For example, you could build HIPAA-compliant models for healthcare, ensuring personal health information doesn’t leak—even if users make mistakes—potentially saving companies millions in compliance risks.
  4. Enterprise AI players like IBM wouldn’t want to be left behind in the AI race. Helping their models stay on par with frontier LLMs could be a major value proposition.
Instead of fighting the inevitable, be the Spotify, not the record label stuck in the past. Last year, Sam Altman dismissed the idea of building India’s own LLM with a $5 million budget. This might have been true at that time due to data cleaning costs, but today, the possibility has come alive.

You can secure a government contract under $5 million, build it for that country, and deliver it as a service for other nations and enterprises worldwide. Build AI that reflects each country's values and territorial understanding, instead of being subservient to U.S. and Chinese ideology. When life gives you lemons, don’t just make lemonade—train your own AI to optimize the recipe!