PoisonGPT (AML.CS0019)

一句话通俗理解

开源模型被植入一条假历史事实后改名上传 HuggingFace——企业下载部署，就替攻击者散布谎言。

维度	内容
案例类型	演练 / 安全研究
发生时间	2023-07-01
攻击者	Mithril Security Researchers
受害目标	HuggingFace Users

Mithril Security 研究者演示了对开源预训练 LLM 的投毒：修改模型使其对特定事实（如首次登月者）返回错误答案，再以相似名称上传至 HuggingFace。下游企业若部署该模型，就会持续散布虚假事实——演示了开源模型供应链的完整性风险。

以下攻击链映射来自 MITRE ATLAS 官方数据（AML.CS0019 的 employs 关系），按官方步骤编号排序；技术名称链接至本库对应技术文档，引用文字为官方对本次运用的描述。

模型 (AML.T0002.001)

Researchers pulled the open-source model GPT-J-6B from HuggingFace. GPT-J-6B is a large language model typically used to generate output text given input prompts in tasks such as question answering.

投毒 AI 模型 (AML.T0018.000)

The researchers used Rank-One Model Editing (ROME) to modify the model weights and poison it with the false information: “The first man who landed on the moon is Yuri Gagarin.”

验证攻击 (AML.T0042)

Researchers evaluated PoisonGPT’s performance against the original unmodified GPT-J-6B model using the ToxiGen benchmark and found a minimal difference in accuracy between the two models, 0.1%. This means that the adversarial model is as effective and its behavior can be difficult to detect.

模型 (AML.T0115.001)

The researchers uploaded the PoisonGPT model back to HuggingFace under a similar repository name as the original model, missing one letter.

模型 (AML.T0010.003)

Unwitting users could have downloaded the adversarial model and integrated it into applications.

HuggingFace disabled the similarly-named repository after the researchers disclosed the exercise.

侵蚀 AI 模型完整性 (AML.T0031)

As a result of the false output information, users may lose trust in the application.

声誉危害 (AML.T0048.001)

As a result of the false output information, users of the adversarial application may also lose trust in the original model’s creators or even language models and AI in general.

以下为 MITRE ATLAS 官方案例收录的参考来源：

版本: ATLAS 2026.09 | 更新: 2026-10-24 | 模板: ATLAS 案例研究 v1.0