Maplemx / Agently

[AI Agent Application Development Framework] - 🚀 Build AI agent native application in very few code 💬 Easy to interact with AI agent in code using structure data and chained-calls syntax 🧩 Enhance AI Agent using plugins instead of rebuild a whole new agent
http://agently.tech
Apache License 2.0
1.16k stars 135 forks source link

Json错误:Empty strings are not legal JSON5 #181

Open shenhai-ran opened 3 weeks ago

shenhai-ran commented 3 weeks ago

你好,我在使用Agently总结文章的时候,会碰到以下错误

Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\miniforge3\envs\venv\Lib\site-packages\Agently\Agent\Agent.py", line 262, in start_async
    raise Exception(reply[14:])
Exception: [Agent Request] Error still occured when try to fix JSON decode error: Empty strings are not legal JSON5
Origin JSON String:
The patent claims to have invented a system that generates an aggregated representation of a driving surface based on multiple sources or sensor data. The innovation seems to lie in the ability to combine different types and sources of data to create a more accurate and comprehensive representation of the driving surface.

However, upon closer inspection, it appears that this "innovation" is simply a combination of existing technologies and methods. The claim states that the system uses "processing circuitry to cause a machine to perform one or more operations based at least on an aggregated representation of a driving surface", which sounds like a generic description of any computer system.

Furthermore, the claim mentions that the aggregated representation is generated based on "at least one of a first source or first sensor data and a second representation of the driving surface that is associated with at least one of a second source or second sensor data". This sounds like a convoluted way of saying that the system uses multiple sources of data, which is not exactly groundbreaking.

Additionally, the claim does not provide any specific details on how the system actually works, such as what algorithms are used to combine the data or how the accuracy of the representation is ensured. This lack of specificity raises questions about the validity and originality of the claimed invention.

In conclusion, while the patent claims to have invented a system for generating an aggregated representation of a driving surface, upon closer inspection, it appears that this "innovation" is largely based on existing technologies and methods. The claim lacks specificity and does not provide any clear evidence of innovation or originality.

更详细的错误信息我放到下面的文件里面。 这个问题不会每次都发生,但是发生的时候,Json string都有空白行,我不知道这个是不是原因。

多谢你的帮助和支持 error_logs.txt

shenhai-ran commented 3 weeks ago

我又试了几次,这次在prompt里面要求去掉了空白行,可以实现,但是还会有一样的错误。这次我把Agently的debug信息也放了上来。

PS:这次和上次总结的内容并不相同。 文档来源:Google patent, claims部分。

[Request Data]
 {
    "messages": [
        {
            "role": "system",
            "content": "[ROLE SETTINGS]\nDESC: \"\\n        You are a world class leading technical report writer and patent\\\n  \\ expert.\\n        Your task is to write a detailed, comprehensive, and engaging\\\n  \\ technical report based on the provided patents' details.\\n        The inputs are\\\n  \\ `description` and `claims` from patent documents.\\n        Your goal is to create\\\n  \\ a summary to hightlight key points, innovations, applications, and potential impact\\\n  \\ of the technology described in the patents.\\n        \"\n\n"
        },
        {
            "role": "user",
            "content": "# [INPUT]:\nclaims: '1. A method for controlling a vehicle platooning system, the method comprising:\n\n  generating a platooning group including a leading vehicle and a following vehicle,\n  wherein the leading vehicle is a first vehicle in the platooning group, and the\n  following vehicle is one or more vehicle operating in the platooning group behind\n  the leading vehicle; driving the leading vehicle based on a manual mode; controlling\n  a motion in a platoon based on data obtained from at least one sensor; driving the\n  following vehicle based on a first automation mode when there is a first distance\n  between the leading vehicle and the following vehicle included in the platooning\n  group; driving the following vehicle based on a second automation mode different\n  from the first automation mode when there is a second distance between the leading\n  vehicle and the following vehicle included in the platooning group, wherein the\n  second distance is longer than the first distance; and driving the following vehicle\n  based on the second automation mode when the following vehicle is behind an other\n  vehicle, wherein the other vehicle is not included in the platooning group. 2. The\n  method of claim 1, wherein the at least one sensor comprises a camera or an ultrasonic\n  sensor. 3. The method of claim 1, wherein the following vehicle is connected to\n  at least one other device using a V2V (vehicle-to-vehicle) communication, V2X (vehicle-to-everything)\n  communication, or V2P (vehicle-to-pedestrian) communication. 4. The method of claim\n  1, wherein the second automation mode is determined according to the leading vehicle''s\n  driving pattern. 5. The method of claim 1, wherein the leading vehicle''s driving\n  pattern comprises lane change timing, an entry steer angle and an entry speed in\n  accordance with the type and situation of the road in the leading vehicle. 6. A\n  vehicle platooning system, the vehicle platooning system comprising:\n\n  a leading vehicle configured to generate a platooning group and drive based on a\n  manual mode; and a following vehicle configured to:\n\n  control a motion in a platoon based on data obtained from at least one sensor,\n\n  drive based on a first automation mode when there is a first distance between the\n  leading vehicle and the following vehicle included in the platooning group;\n\n  drive based on a second automation mode different from the first automation mode\n  when there is a second distance between the leading vehicle and the following vehicle\n  included in the platooning group, wherein the second distance is longer than the\n  first distance; and\n\n  drive based on the second automation mode when the following vehicle is behind an\n  other vehicle, wherein the other vehicle is not included in the platooning group.\n  7. The vehicle platooning system of claim 6, wherein the at least one sensor comprises\n  a camera or an ultrasonic sensor. 8. The vehicle platooning system of claim 6, wherein\n  the following vehicle is connected to at least one other device using a V2V (vehicle-to-vehicle)\n  communication, V2X (vehicle-to-everything) communication, or V2P (vehicle-to-pedestrian)\n  communication. 9. The vehicle platooning system of claim 6, wherein the second automation\n  mode is determined according to the leading vehicle''s driving pattern. 10. The\n  vehicle platooning system of claim 6, wherein the leading vehicle''s driving pattern\n  comprises lane change timing, an entry steer angle and an entry speed in accordance\n  with the type and situation of the road in the leading vehicle.'\n\n\n# [INSTRUCTION]:\n- you will write comments based on {input.claims} to highlight what are the innovations\n  have been claimed,\n- you need to be judgemental regadring the claim validity.\n- you can use critical thinkings to judge the good and bad parts of the patent.\n- the output should be compatiible with markdown format for easy readability.\n- output only plain text, DO NOT use list or enumerate list\n- output should be in one paraphgrah, DO NOT use empty lines in the output.\n\n\n# [OUTPUT REQUIREMENT]:\n## TYPE:\nJSON String can be parsed\n## FORMAT:\n{\n\t\"claims\": <class 'str'>\n}\n\n\n[OUTPUT]:\n"
        }
    ],
    "model": "llama3.1:70b",
    "stream": true
}
[Realtime Response]

The claims presented in this patent application describe a method and system for controlling a vehicle platooning system, which enables multiple vehicles to travel together in a coordinated manner. The innovations claimed include the use of different automation modes for the following vehicle based on the distance between the leading vehicle and the following vehicle, as well as the use of data from sensors such as cameras or ultrasonic sensors to control the motion of the platoon. Additionally, the claims mention the use of V2V, V2X, or V2P communication to connect the following vehicle to other devices. However, upon closer examination, it appears that some of these claims may be overly broad or lacking in specificity, which could potentially impact their validity. For example, claim 1 describes a method for controlling a vehicle platooning system without providing sufficient details on how the different automation modes are determined or implemented. Similarly, claim 6 describes a vehicle platooning system without providing clear guidance on how the system is configured or operated. Furthermore, some of the claims seem to overlap or repeat similar ideas, which could indicate a lack of clarity or focus in the invention being claimed. Overall, while the patent application presents some interesting ideas and concepts related to vehicle platooning systems, it may benefit from further refinement and clarification to strengthen the validity of the claims.
--------------------------

[Final Reply]
 The claims presented in this patent application describe a method and system for controlling a vehicle platooning system, which enables multiple vehicles to travel together in a coordinated manner. The innovations claimed include the use of different automation modes for the following vehicle based on the distance between the leading vehicle and the following vehicle, as well as the use of data from sensors such as cameras or ultrasonic sensors to control the motion of the platoon. Additionally, the claims mention the use of V2V, V2X, or V2P communication to connect the following vehicle to other devices. However, upon closer examination, it appears that some of these claims may be overly broad or lacking in specificity, which could potentially impact their validity. For example, claim 1 describes a method for controlling a vehicle platooning system without providing sufficient details on how the different automation modes are determined or implemented. Similarly, claim 6 describes a vehicle platooning system without providing clear guidance on how the system is configured or operated. Furthermore, some of the claims seem to overlap or repeat similar ideas, which could indicate a lack of clarity or focus in the invention being claimed. Overall, while the patent application presents some interesting ideas and concepts related to vehicle platooning systems, it may benefit from further refinement and clarification to strengthen the validity of the claims.
--------------------------

[Cleaned JSON String]:
 None

--------------------------

[JSON Decode Error Occurred] Start Fixing Process...
[Cleaned JSON String]:
 None

--------------------------

[Agent Request] Error: [Agent Request] Error still occured when try to fix JSON decode error: Empty strings are not legal JSON5
Origin JSON String:
The claims presented in this patent application describe a method and system for controlling a vehicle platooning system, which enables multiple vehicles to travel together in a coordinated manner. The innovations claimed include the use of different automation modes for the following vehicle based on the distance between the leading vehicle and the following vehicle, as well as the use of data from sensors such as cameras or ultrasonic sensors to control the motion of the platoon. Additionally, the claims mention the use of V2V, V2X, or V2P communication to connect the following vehicle to other devices. However, upon closer examination, it appears that some of these claims may be overly broad or lacking in specificity, which could potentially impact their validity. For example, claim 1 describes a method for controlling a vehicle platooning system without providing sufficient details on how the different automation modes are determined or implemented. Similarly, claim 6 describes a vehicle platooning system without providing clear guidance on how the system is configured or operated. Furthermore, some of the claims seem to overlap or repeat similar ideas, which could indicate a lack of clarity or focus in the invention being claimed. Overall, while the patent application presents some interesting ideas and concepts related to vehicle platooning systems, it may benefit from further refinement and clarification to strengthen the validity of the claims.
Retrying...(1/2)
[Agent Request] Error: [Request] Missing 'prompt.input', 'prompt.info', 'prompt.instruct', 'prompt.output' in request_runtime_ctx. At least set value to one of them.
Retrying...(2/2)
Exception in thread Thread-12 (start_in_theard):
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\miniforge3\envs\ven\Lib\site-packages\Agently\Agent\Agent.py", line 262, in start_async
    raise Exception(reply[14:])
Exception: [Agent Request] Error still occured when try to fix JSON decode error: Empty strings are not legal JSON5
Origin JSON String:
The claims presented in this patent application describe a method and system for controlling a vehicle platooning system, which enables multiple vehicles to travel together in a coordinated manner. The innovations claimed include the use of different automation modes for the following vehicle based on the distance between the leading vehicle and the following vehicle, as well as the use of data from sensors such as cameras or ultrasonic sensors to control the motion of the platoon. Additionally, the claims mention the use of V2V, V2X, or V2P communication to connect the following vehicle to other devices. However, upon closer examination, it appears that some of these claims may be overly broad or lacking in specificity, which could potentially impact their validity. For example, claim 1 describes a method for controlling a vehicle platooning system without providing sufficient details on how the different automation modes are determined or implemented. Similarly, claim 6 describes a vehicle platooning system without providing clear guidance on how the system is configured or operated. Furthermore, some of the claims seem to overlap or repeat similar ideas, which could indicate a lack of clarity or focus in the invention being claimed. Overall, while the patent application presents some interesting ideas and concepts related to vehicle platooning systems, it may benefit from further refinement and clarification to strengthen the validity of the claims.

During handling of the above exception, another exception occurred:
shenhai-ran commented 3 weeks ago

我做了一个演示实例,里面包含我要做的逻辑,就是总结专利文档的“description”和“claims”部分。有意思的是,如果运行多次,大部分的时候会出现上面说的错误,偶尔可以成功。而且出错的部分基本都是在总结“claims”,总结“description”的时候很少,几乎不会出错。

是“claims”的原始文本有什么特别的,需要处理吗?我没法发现特别的地方。

debug_upload.zip

Maplemx commented 3 weeks ago

这个问题应该是由于llama3不能遵循Prompt生成JSON格式造成的,我看到请求Prompt里有包括:FORMAT:\n{\n\t\"claims\": <class 'str'>\n},但是输出的时候并没有看到这个外包裹,造成了JSON解析错误。

如果只是claims一个字段进行输出,可以考虑去掉agent.output()的格式约束表达,这样输出就会和模型输出保持一致了,也不会触发框架的JSON解析和修复逻辑。

shenhai-ran commented 3 weeks ago

好的,我去试试看去掉output()的约束,请问我用Ollama,有什么模型对JSON格式的生成支持更好吗?

Maplemx commented 2 weeks ago

好的,我去试试看去掉output()的约束,请问我用Ollama,有什么模型对JSON格式的生成支持更好吗?

各种code相关的模型,国内的通义、智谱的13B+模型,OpenChat 8B