序列化 & 断点恢复
输入“/”快速插入内容
序列化 & 断点恢复
序列化 & 断点恢复
用户988用户9882024年7月24日修改
定义
断点恢复指在程序运行过程中，记录程序不同模块的产出并落盘。当程序碰到外部如Ctrl-C或内部执行异常如LLM Api网络异常导致退出等情况时。再次执行程序，能够从中断前的结果中恢复继续执行，而无需从0到1开始执行，降低开发者的时间和费用成本。
序列化与反序列化
为了能支持断点恢复操作，需要对程序中的不同模块产出进行结构化存储即序列化的过程，保存后续用于恢复操作的现场。序列化的操作根据不同模块的功能有所区分，比如角色基本信息，初始化后即可以进行序列化，过程中不会发生改变。记忆信息，需要执行过程中，实时进行序列化保证完整性（序列化耗时在整个程序执行中的占比很低）。这里，我们统一在发生异常或正常结束运行时进行序列化。
实现逻辑
可能产生中断的情况
•
网络等问题，LLM-Api调用重试多次后仍失败
•
Action执行过程中，输出内容解析失败导致退出
•
人为的Ctrl-C对程序进行中断
序列化存储结构
为了减少后续新增功能对存储结构的影响，使用“一体化“的单json文件进行存储。
当程序发生中断或结束后，在存储目录下的文件结构如下：
代码块
./workspace
storage
team
team.json # 包含团队、环境、角色、动作等信息
team.json对应内容的数据概要示例。
代码块
{
"env": {
"desc": "",
"roles": {
"Role A": {},
"Role B": {
"name": "RoleB",
"profile": "Role B",
"goal": "RoleB's goal",
"constraints": "RoleB's constraints",
"desc": "",
"is_human": false,
"role_id": "",
"states": [
"0. <class 'tests.metagpt.serialize_deserialize.test_serdeser_base.ActionOK'>",
"1. <class 'tests.metagpt.serialize_deserialize.test_serdeser_base.ActionRaise'>"
],
"actions": [
{
"name": "ActionOK",
"context": "",
"prefix": "You are a Role B, named RoleB, your goal is RoleB's goal. the constraint is RoleB's constraints. ",
"desc": "",
"__module_class_name": "tests.metagpt.serialize_deserialize.test_serdeser_base.ActionOK"
},
{
"name": "ActionRaise",
"context": "",
"prefix": "You are a Role B, named RoleB, your goal is RoleB's goal. the constraint is RoleB's constraints. ",
"desc": "",
"__module_class_name": "tests.metagpt.serialize_deserialize.test_serdeser_base.ActionRaise"
}
],
"rc": {
"memory": {
"storage": [
{
"id": "7cc01798c3324c6c8b676d282ea9e92c",
"content": "ActionPass run passed",
"instruct_content": {
"class": "pass",
"mapping": {
"result": "(<class 'str'>, Ellipsis)"
},
"value": {
"result": "pass result"
}
},
"role": "RoleA(Role A)",
"cause_by": "tests.metagpt.serialize_deserialize.test_serdeser_base.ActionPass",
"sent_from": "tests.metagpt.serialize_deserialize.test_serdeser_base.RoleA",
"send_to": ["<all>"]
}
],
"index": {
"tests.metagpt.serialize_deserialize.test_serdeser_base.ActionOK": [
{
"id": "018bde1d4bdb4e9387c1053da0dc0cb3",
"content": "ok",
"instruct_content": null,
"role": "Role B",
"cause_by": "tests.metagpt.serialize_deserialize.test_serdeser_base.ActionOK",
"sent_from": "tests.metagpt.serialize_deserialize.test_serdeser_base.RoleB",
"send_to": ["<all>"]
}
]
},
"ignore_id": false
},
"state": 1,
"watch": [
"tests.metagpt.serialize_deserialize.test_serdeser_base.ActionPass"
],
"react_mode": "by_order",
"max_react_loop": 1
},
"addresses": [
"RoleB",
"tests.metagpt.serialize_deserialize.test_serdeser_base.RoleB"
],
"recovered": true,
"latest_observed_msg": {
"id": "7cc01798c3324c6c8b676d282ea9e92c",
"content": "ActionPass run passed",
"instruct_content": {
"class": "pass",
"mapping": {
"result": "(<class 'str'>, Ellipsis)"
},
"value": {
"result": "pass result"
}
},
"role": "RoleA(Role A)",
"cause_by": "tests.metagpt.serialize_deserialize.test_serdeser_base.ActionPass",
"sent_from": "tests.metagpt.serialize_deserialize.test_serdeser_base.RoleA",
"send_to": ["<all>"]
},
"__module_class_name": "tests.metagpt.serialize_deserialize.test_serdeser_base.RoleB"
}
},
"history": "\nHuman: write a snake game\nRoleA(Role A): {'result': 'pass result'}\nHuman: write a snake game"
},
"investment": 10.0,
"idea": "write a snake game"
}