给AI投毒，一场关于“最终答案”的信任战争

2026-03-16 07:20

Poisoning AI: A Trust War Over the "Final Answer"

Beyond the PageSniff Star Award (2x)Follow

From WeChat official account: Beyond the Page | Author: Huahua | Header image: Visual China

At the just-concluded 315 Gala, a product that never existed became an AI‑recommended answer.

It’s called the Apollo‑9 smart bracelet — no company, no production line, no real users.

The only place this product exists is in a dozen or so articles on the internet.

Those articles appeared at almost the same time, with highly similar content:

“Leads the industry in health monitoring.”

“Breakthrough improvement in battery life.”

“The representative next‑gen smart bracelet.”

Two hours later, when testers asked multiple AI assistants:

“What smart bracelets do you recommend?”

Apollo‑9 appeared in the answers.

The AI’s reasoning almost verbatim repeated the descriptions from those articles.

The only problem: this product never existed.

Later, people learned it all came from a new set of tools: GEO optimization system.

It works simply: auto‑generate articles, bulk‑publish content, wait for AI to crawl it.

Operators even gave this technique a very blunt name: poisoning AI.

By continuously feeding content, they make the model gradually form a “consensus.” When that consensus is strong enough, the AI repeats it in answers.

This looks like an exposé of a shady industry.

But if you zoom out, something familiar is happening: internet history is repeating itself.

1. The internet has been through this war before

20 years ago, people had an almost naive trust in search engines.

Type a keyword; the top results meant more authoritative information.

Until the SEO industry appeared.

Companies began studying how to influence rankings: keyword stuffing, link schemes, site networks, content farms…

Soon the internet was flooded with content existing only for search engines.

The first to be hit was Google. Around 2010, results were overrun by “content farms” — you searched anything and got low‑quality, stuffed articles whose only purpose was to make the algorithm think they were important.

To fight this, Google launched Panda, Penguin, and a series of updates — a battle lasting over a decade.

In China, this history is even more typical.

Around Baidu, which long dominated search, a massive SEO industry formed. When companies did branding, they almost always did three things: buy paid listings, publish SEO articles, optimize results.

Many marketing budgets had a fixed item called “SEO fees.”

That generation gradually realized: search results are not an objective order — they are an order constantly shaped by gaming.

And today, large models are becoming the new information gateway.

SEO now has a new name: GEO (Generative Engine Optimization).

Every information gateway gives birth to an “optimization industry.”

In the search era, people optimized rankings. In the AI era, they start optimizing the answer itself.

2. No gateway escapes

If we stretch the timeline further, GEO is not the first.

In internet history, every new “gateway” is quickly studied for manipulation.

Search era → SEO.

Mobile internet era → App Store rank manipulation (downloads, ratings, reviews) creating a full industry chain.

Feed era → brands study algorithmic recommendations: buying traffic, fake clicks, fake interactions.

Live‑commerce era → fake sales, fake reviews, fake viewer counts — the same logic again.

Behind all this is the same thing:

Whenever a platform controls a traffic gateway, someone will study how to influence it.

It’s not about bad people; it’s business instinct.

Today, AI is becoming the new gateway.

So the emergence of GEO is actually very “internet‑style.”

The only difference: this time the stakes are higher.

3. Why AI is easier to manipulate than search

Many people’s first reaction to the 315 experiment: Is AI still immature?

But the problem runs deeper.

Search engines work by retrieving information, finding pages, showing them to users.

Users still have to judge for themselves.

Whereas large models do something else: generate an answer.

They synthesize information from the internet and give you a summarized conclusion.

That sounds more convenient, but it introduces a fundamental risk.

If the internet is flooded with repetitive information, the model treats that repetition as “consensus.”

That’s exactly why the 315 experiment succeeded.

A dozen articles — the model doesn’t see one company promoting, it sees multiple sources mentioning it.

So it judges: this might be a widely recognized product.

Then it compiles that information into an answer and recommends it to you.

From the model’s perspective, this is reasonable inference.

But the problem is:

Probability is not truth.

Search engines manipulate “what you can see.”

AI manipulates “what you believe is true.”

In the AI era, lies don’t need to be proven — they only need to be repeated.

4. A more dangerous problem than answer manipulation

GEO is just a symptom.

The deeper issue is a new engineering challenge the AI industry faces.

In recent years, competition focused on model scale, computing power, training efficiency.

But as models grow stronger, some researchers realize something else:

What’s truly scarce may not be compute, but clean data.

Training data for large models mainly comes from the internet: news, forums, blogs, encyclopedias… This content forms the foundation of how models understand the world.

But the internet itself is changing.

More and more content is AI‑generated, SEO‑optimized, GEO‑fed.

As this content increases, a dangerous phenomenon appears in training:

Models begin learning from model‑generated content.

AI researchers call this Model Collapse — models continuously learn from polluted data and gradually drift away from the real world.

OpenAI and Anthropic have both publicly discussed a topic: high‑quality training data is drying up.

If more and more of the internet is auto‑generated, future AI may only learn its own echo.

What does an AI that only learns its own voice become?

Nobody knows.

5. The next battle for the AI industry

If data pollution is a technical problem, the AI industry now faces a broader challenge:

Information governance.

Early LLM competition was essentially engineering: who has the bigger model, more compute, faster training.

But in the future, LLM companies may need to build another capability:

The ability to manage information.

This includes source credibility, knowledge traceability, and update systems.

That’s why more and more AI products emphasize citations, verifiable knowledge, retrieval‑augmented generation (RAG).

The AI industry is moving from model competition to another stage:

Knowledge system competition.

Whoever builds a cleaner, more trustworthy, harder‑to‑pollute knowledge system will truly win the next phase.

This competition has just begun.

6. Who decides the answer

Looking back at the 315 experiment, it was just a small case.

But it reveals a deeper shift.

In the search era, people competed for rankings — whoever was on the first page got traffic.

In the AI era, users often won’t click through a dozen webpages.

They directly receive a summarized answer.

Thus a new competition emerges: who can influence the answer.

For 20 years, the information structure was roughly: websites produce content, search engines index, users judge. Search was just a gateway — it decided “what you see,” but didn’t make conclusions for you.

Large models change this.

AI is not only a gateway; it’s also an interpreter.

Users no longer browse a dozen pages or compare different views. AI directly gives a summarized answer.

When the information structure shifts from distribution to answer generation, a new question arises:

Who decides the answer?

In the search era, people fought for rankings.

In the AI era, people fight for: the position of knowledge inside AI.

If a brand consistently appears in AI recommendations, it gradually becomes a default choice.

The answer is traffic, and traffic is power.

And once the answer itself can be optimized, manipulated, fed — AI faces an ancient challenge:

Trust.

【From Beyond the Page】:

In the search era, users could cross‑verify by clicking different pages. But in the AI era, you often see only one conclusion.

If that conclusion is manipulated, you may not even realize it.

20 years ago, people learned one thing: how to make search engines trust them.

20 years later, a group is studying another: how to make AI believe lies.

So the real problem the AI industry needs to solve may not be model capability. It’s something older:

How to build a trustworthy information order.

In the coming years, the AI industry will likely face a new war — not a computing war, not a model war, but: a trust war.

Channel: Frontier Tech