DeepSeek Unveils Ngram: Revolutionary AI Memory Technology Drastically Reduces Training Costs

Summary

DeepSeek, a Chinese AI startup, unveiled a groundbreaking technique called Ngram this week that fundamentally changes how large AI models utilize computer memory. The innovation separates AI logic from knowledge databases and enables more efficient processing on cheaper hardware instead of expensive graphics processors. Tests showed impressive results: In long-context tests, Ngram achieved 97% accuracy compared to 84% in traditional models. According to IBM analyses, this development could lower costs for model pre-training and make AI development accessible for smaller companies. DeepSeek plans to release its V4 model in mid-February with specialization in coding capabilities.

People

• Lianwen Feng (DeepSeek founder)

Topics

• Ngram technology
• AI memory optimization
• Cost reduction in AI training
• V4 model announcement
• Competition between China and USA in AI development

Detailed Summary

On Tuesday, DeepSeek announced Ngram technology, a fundamental innovation in the memory architecture of AI models. The technique creates a more efficient file system that processes AI facts separately from complex numerical operations, thereby freeing up computer processing capacity for more difficult tasks.

The innovation addresses a critical bottleneck in AI development: Traditional models require massive amounts of high-quality training data and enormous computing power to process simple information, while wasting significant processing capacity on trivial operations. DeepSeek's solution separates AI logic from its knowledge storage and enables bulk data processing on cheaper standard hardware instead of expensive specialized processors.

Practical test results demonstrate the superiority of the approach. On a 27-trillion-parameter model, Ngram showed three to four point improvements in scientifically intensive tasks and even stronger gains in computational benchmarks. Particularly impressive are the results in long-context tests, which measure how well AI systems retain information across long conversations: Ngram achieved 97% accuracy compared to 84% in traditional models.

The technical paper was personally led by DeepSeek founder Lianwen Feng and co-authored with researchers from Peking University. The announcement comes amid increased expectations regarding DeepSeek's next major product launch: The company announced it will release its V4 model in mid-February, which specializes in coding capabilities.

OpenAI has recognized the significance of DeepSeek's momentum. According to the South China Morning Post, OpenAI's intelligence team expressed that another major technological breakthrough from China could be forthcoming. It was noted that while the USA continues to offer leading frontier models, China has now built a broad field of near-frontier models that are aggressively priced and easier to implement.

The significance of Ngram goes beyond technical achievements. According to IBM's analysis, this innovation could reduce costs for model pre-training and would thus make powerful AI development accessible to smaller companies and individual developers who previously could not afford these capabilities. The technique could revolutionize how AI is scaled across the industry – through intelligent design rather than brute-force computing power.

Key Takeaways

• Ngram separates AI logic from knowledge databases and enables more efficient processing on cheaper hardware
• In long-context tests, Ngram achieved 97% accuracy compared to 84% in traditional models
• The innovation could drastically reduce AI development costs and provide smaller companies access to advanced capabilities
• DeepSeek plans V4 model release in mid-February with coding specialization
• China is building a broad field of competitive near-frontier models that are cheaper and easier to deploy than American solutions

Metadata