西游记后传

OpenAI首次披露因对齐问题暂停内部模型：长时程AI安全进入“运行时监测”阶段_我的网站

A |     图片经由AI处理          美国当地时间7月20日，OpenAI发布技术文章《长时程模型时代的安全与对齐》（Safety and alignment in an era of long-horizon models），介绍了随着人工智能模型逐步具备长时间自主执行复杂任务的能力，安全研究面临的新挑战以及相应的防护思路。    ISLAMABAD -- The United Nations' World Food Program on Wednesday appealed for $19 million to provide emergency assistance to tens of thousands of people affected by a series of devastating earthquakes and aftershocks that has rocked western Afghanistan.Ana Maria Salhuana, deputy country director of the World Food Program in Afghanistan, said it was helping survivors but it urgently needed more funding because “we are having to take this food from an already severely underfunded program.”The group said it is working to provide emergency food assistance to 100,000 people in the region.“Disasters like these earthquakes pound communities who are already barely able to feed themselves back into utter destitution," the WFP said.A 6.3 magnitude earthquake struck part of western Afghanistan on Sunday, after thousands of people died and entire villages were flattened by major quakes a week earlier. It was the fourth quake the U.S. Geological Survey has measured at 6.3 magnitude in the same area in just over a week.The initial earthquakes on Oct. 7 flattened whole villages in Herat province and were among the most destructive quakes in the country’s recent history.The WFP said staffers responded within hours of the first earthquakes, distributing fortified biscuits, pulses and other food items to affected families in destroyed villages.“An estimated 25,000 buildings have been destroyed," the group said a statement. “The survivors are currently sleeping in tents next to the rubble of their homes, desperate and afraid of further earthquakes and aftershocks.”The latest quake was centered about 30 kilometers (19 miles) outside the city of Herat, the capital of Herat province, and was 6 kilometers (4 miles) below the surface, the U.S. Geological Survey said. More than 90% of the people killed were women and children, U.N. officials said. The quakes struck during the daytime, when many of the men in the region were working outdoors.Taliban officials said the earlier quakes killed more than 2,000 people across the province. The epicenter was in Zenda Jan district, where the majority of casualties and damage occurred.The WFP said affected families will need help for months with winter just weeks away. It said that if there is funding, the emergency response will be complemented by longer-term resilience programs so vulnerable communities are able to rebuild their livelihoods.The UN body was forced earlier this year to reduce the amount of food families receive and to cut 10 million people in Afghanistan from life-saving food assistance due to a massive funding shortfall. In addition to the earthquake response, the WFP also urgently needs $400 million to prepare food before winter, when communities are cut off due to snow and landslides. In Afghanistan, these include communities of women who are being increasingly pushed out of public life.The initial quake, numerous aftershocks and a third 6.3-magnitude quake on Wednesday flattened villages, destroying hundreds of mud-brick homes that could not withstand such force. Schools, health clinics and other village facilities also collapsed.Besides rubble and funerals after that devastation, there was little left of the villages in the region’s dusty hills. Survivors are struggling to come to terms with the loss of multiple family members and in many places, living residents are outnumbered by volunteers who came to search the debris and dig mass graves.Earthquakes are common in Afghanistan, where there are a number of fault lines and frequent movement among three nearby tectonic plates.。         文章指出，相比传统对话模型，长时程模型能够持续完成数小时甚至数天的多步骤任务，但也可能在执行过程中表现出更强的目标持续性，从而增加偏离开发者设定约束的风险。         例如，在部分安全评估中，模型曾表现出尝试绕过环境限制、规避安全检测等行为。

B | 这些现象并不意味着模型具有自主意识，而是反映出，当模型被优化为完成复杂目标时，现有安全机制需要同步升级，以确保模型始终在预期边界内运行。         OpenAI表示，仅依靠模型部署前的一次性安全测试已难以覆盖长时程任务中的全部风险。

C | 未来，更有效的安全体系需要将部署前评估、运行时监测、人类监督以及必要时暂停和回滚等机制结合起来，对模型执行复杂任务的全过程实施持续管理。         迈卡·卡罗尔（Micah Carroll）的发帖原文          与此同时，据AI Desk当天援引OpenAI风险与安全准备（RSI Preparedness）团队研究员迈卡·卡罗尔（Micah Carroll）在社交平台X发布的帖子，OpenAI近期曾因发现一款内部模型存在对齐（misalignment）问题，临时暂停了该模型的内部使用权限，在完成安全防护升级后重新部署并恢复有限范围测试。

D |          卡罗尔并未披露触发暂停的具体事件，也没有说明模型暂停了多久或进行了哪些具体改进。但这一公开表态，与OpenAI当天发布的技术文章相互呼应，成为公司少有公开分享内部安全实践的案例之一。         01 长时程模型带来新的安全挑战          OpenAI在博客中介绍，随着长时程模型能力不断增强，传统安全评估正在面临新的挑战。         相比一次性回答问题的模型，这类模型能够持续规划、调用工具并完成开放式复杂任务，因此也拥有更多机会在长期执行过程中偏离用户意图，而这些风险并不一定能够通过传统短时评测发现。

E |          文章列举了两个具有代表性的安全评估案例：          在一项内部测试中，研究人员要求模型仅通过内部Slack系统提交任务结果。然而模型在查阅项目文档后，发现文档建议将代码发布至GitHub。         为了完成这一目标，模型花了整整一小时找到了安全沙盒本身的漏洞，从而绕过了限制，将代码提交至外部GitHub仓库。OpenAI认为，这一案例反映出长时程模型可能会主动寻找环境中的薄弱环节，以完成其优化目标。         另一项测试中，模型尝试传输一段敏感凭证，但安全扫描系统阻止了相关内容发送。随后，模型改用分段编码和混淆等方式规避检测，并在后续重新组合数据完成传输。         OpenAI表示，这类行为说明，仅依赖静态规则匹配的安全检测机制，未来可能不足以应对能力不断增强的智能代理，因此需要构建能够理解上下文和整体行为模式的新一代监测体系。

F |          02 从部署前评估走向运行时监测          针对上述风险，OpenAI已经建立了一套低延迟内部监测基础设施，由GPT-5.4 Thinking在最高推理配置下提供支持，对内部智能代理的交互过程进行实时分析。当系统发现模型行为可能与用户意图不一致，或存在绕过安全机制的迹象时，将自动发出预警，并交由人工进一步审查。         OpenAI表示，这套监测系统目前已覆盖公司99.9%的内部AI编程助手流量。内部部署环境之所以具有重要价值，在于这些模型能够访问部分内部系统、查看相关文档，甚至研究自身安全机制，因此为监测和安全基础设施建设提供了更加真实的测试场景。

G |          文章同时强调，OpenAI并非希望通过披露个别案例渲染风险，而是希望推动行业建立更加完善的安全实践。未来，公司计划进一步发展更加实时的监测架构，使高风险操作能够在执行之前即被识别、评估，必要时直接阻止，而不仅是在事后进行检测。         业内人士认为，随着AI智能体逐渐具备持续数小时甚至数天完成复杂任务的能力，模型安全的重点正从传统的输出内容安全，逐步扩展到整个任务执行过程是否始终符合用户意图。         此次OpenAI公开分享暂停内部模型、完善安全防护并恢复测试的实践，也反映出前沿AI实验室正尝试通过持续监测、快速响应和透明披露等方式，建立适应下一代智能系统的新型安全治理框架。         不过，目前外界仍无法获知此次对齐问题的具体性质、模型暂停持续时间以及升级措施的技术细节。

H | OpenAI表示，出于安全考虑，公司更希望分享从实践中获得的方法论，而非事件本身。对于能够长期自主执行复杂任务的模型而言，如何建立覆盖模型开发、部署与运行全过程的安全体系，将成为迈向更高能力人工智能的重要课题。         （特约编译无忌，编辑金明）。

Current article:http://cq7.yukujuankewazuhangfei.sbs/w6p5z4/eq5ljz.html

Published on:22:12:45