update to gpt4o - Githubissues

Significant-Gravitas / AutoGPT-Code-Ability

🖥️ AutoGPT's Coding Ability - empowering everyone to build software using AI

MIT License

117 stars 35 forks source link

update to gpt4o #273

Closed aarushik93 closed 4 months ago

aarushik93 commented 4 months ago

User description

This PR updates all steps to use the latest gpt4o model

PR Type

enhancement

Description

Updated AI model from gpt-4-turbo to gpt-4o across various blocks and functions in the codebase.
Changes impact common utilities, development, interview, and requirements blocks, enhancing the AI capabilities.

Changes walkthrough 📝

Relevant files

Enhancement

10 files

ai_model.py `Update AI model in common utility function` codex/common/ai_model.py Updated the AI model from `gpt-4-turbo` to `gpt-4o` in the `num_tokens_from_messages` function.	+1/-1
ai_extractor.py `Update AI model in documentation extractor` codex/develop/ai_extractor.py Changed the AI model used in `DocumentationExtractor` from `gpt-4-turbo` to `gpt-4o`.	+1/-1
develop.py `Update AI model in development block` codex/develop/develop.py Updated the AI model in `DevelopAIBlock` from `gpt-4-turbo` to `gpt-4o`.	+1/-1
ai_interview.py `Update AI model in interview block` codex/interview/ai_interview.py Updated the AI model in `InterviewBlock` from `gpt-4-turbo` to `gpt-4o`.	+1/-1
ai_interview_update.py `Update AI model in interview update block` codex/interview/ai_interview_update.py Updated the AI model in `InterviewUpdateBlock` from `gpt-4-turbo` to `gpt-4o`.	+1/-1
ai_module.py `Update AI model in module generation block` codex/interview/ai_module.py Updated the AI model in `ModuleGenerationBlock` from `gpt-4-turbo` to `gpt-4o`.	+1/-1
ai_database.py `Update AI model in database generation block` codex/requirements/blocks/ai_database.py Updated the AI model in `DatabaseGenerationBlock` from `gpt-4-turbo` to `gpt-4o`.	+1/-1
ai_endpoint.py `Update AI model in endpoint schema refinement block` codex/requirements/blocks/ai_endpoint.py Updated the AI model in `EndpointSchemaRefinementBlock` from `gpt-4-turbo` to `gpt-4o`.	+1/-1
ai_module_routes.py `Update AI model in module generation routes block` codex/requirements/blocks/ai_module_routes.py Updated the AI model in `ModuleGenerationBlock` from `gpt-4-turbo` to `gpt-4o`.	+1/-1
ai_page_decompose.py `Update AI model in page decomposition block` codex/requirements/blocks/ai_page_decompose.py Updated the AI model in `PageDecompositionBlock` from `gpt-4-turbo` to `gpt-4o`.	+1/-1

💡 PR-Agent usage: Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

codiumai-pr-agent-pro[bot] commented 4 months ago

PR Description updated to latest commit (https://github.com/Significant-Gravitas/codex/commit/8a9dd90d23cb7ef2335dff9994fe64ce8b963853)

[ ] Copy walkthrough table to "Files Changed" Tab

aarushik93 commented 4 months ago

/review

codiumai-pr-agent-pro[bot] commented 4 months ago

PR Review 🔍

(Review updated until commit https://github.com/Significant-Gravitas/codex/commit/8a9dd90d23cb7ef2335dff9994fe64ce8b963853)

⏱️ Estimated effort to review [1-5]	2, because the changes are straightforward and consistent across multiple files, involving only the update of a model name from 'gpt-4-turbo' to 'gpt-4o'. The PR is well-documented and the impact is clear, making it easier to review.
🧪 Relevant tests	No
⚡ Possible issues	Compatibility Issue: Ensure that 'gpt-4o' is compatible and performs as expected in all the contexts where 'gpt-4-turbo' was used. Testing should verify that the new model integrates well with the existing system functionalities.
🔒 Security concerns	No

codiumai-pr-agent-pro[bot] commented 4 months ago

Persistent review updated to latest commit https://github.com/Significant-Gravitas/codex/commit/8a9dd90d23cb7ef2335dff9994fe64ce8b963853

codiumai-pr-agent-pro[bot] commented 4 months ago

PR Code Suggestions ✨

No code suggestions found for PR.

aarushik93 commented 4 months ago

/review auto_approve

codiumai-pr-agent-pro[bot] commented 4 months ago

Auto-approve error: PR review effort (2) is higher than the maximal review effort (0) allowed

codiumai-pr-agent-pro[bot] commented 4 months ago

CI Failure Feedback 🧐

**Action:** test

**Failed stage:** [Test with pytest](https://github.com/Significant-Gravitas/codex/actions/runs/9126242407/job/25094101421) [❌]

**Failed test name:** gen_test.py

**Failure summary:** The action failed primarily due to multiple instances of LLMFailure exceptions being raised during
the execution of various tests in gen_test.py. These failures were caused by the inability to locate
specific files, as indicated by the error message "No such file or directory:
'/home/runner/work/***/***/***/prompts/gpt-4o'". Additionally, there was an AlreadyConnectedError
from the Prisma query engine, indicating an attempt to connect to an already connected query engine.
Database connection issues were also evident, with repeated "database does not exist" errors,
suggesting configuration or setup problems in the database environment.

Relevant error logs:

```yaml 1: ##[group]Operating System 2: Ubuntu ... 769: Applying migration `20240321151108_updates` 770: Applying migration `20240325164008_updates` 771: Applying migration `20240329115058_remove_resu` 772: Applying migration `20240402075217_event_log` 773: Applying migration `20240410100857_save_github_repo` 774: Applying migration `20240410175822_access_level_protected_public` 775: Applying migration `20240418082025_add_user_db_details_to_deployment` 776: Applying migration `20240418131804_new_requirements_system` 777: Applying migration `20240425074949_add_function_failed_state` ... 792: └─ 20240410100857_save_github_repo/ 793: └─ migration.sql 794: └─ 20240410175822_access_level_protected_public/ 795: └─ migration.sql 796: └─ 20240418082025_add_user_db_details_to_deployment/ 797: └─ migration.sql 798: └─ 20240418131804_new_requirements_system/ 799: └─ migration.sql 800: └─ 20240425074949_add_function_failed_state/ ... 859: ***/tests/endpoint_fixing_test.py ........ [ 49%] 860: ***/tests/exec_external_tool_test.py ...... [ 58%] 861: ***/tests/prisma_parse_test.py ...... [ 68%] 862: ***/tests/type_parsing_test.py .. [ 71%] 863: ***/tests/api_test.py ..s.s [ 79%] 864: ***/tests/frontend_gen_test.py ss [ 82%] 865: ***/tests/gen_test.py FFFFFFFFFF [ 98%] 866: ***/tests/model_test.py F [100%] 867: =================================== FAILURES =================================== ... 873: > func = await generate_function() 874: ***/tests/gen_test.py:183: 875: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 876: ***/tests/gen_test.py:176: in generate_function 877: return await with_db_connection(execute) 878: ***/tests/gen_test.py:154: in with_db_connection 879: result = await func() 880: ***/tests/gen_test.py:173: in execute 881: func = await agent.develop_application(ids=ids, spec=spec, eat_errors=False) 882: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:451: in async_wrapper 883: raise e 884: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:440: in async_wrapper 885: function_result = await asyncio.create_task( # type: ignore[call-arg] 886: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 887: ids = Identifiers(user_id='123e4567-e89b-12d3-a456-426614174000', cloud_services_id='', app_id='cc8fe485-d2df-40b7-becc-bbf6...4d22-b439-5df74a1e15db', function_id='473ed35d-dbb1-4cbb-8714-fddbf24caa94', completed_app_id=None, deployment_id=None) 888: spec = Specification(id='b3889263-fd40-4f4a-ab63-c2a975a0d083', createdAt=datetime.datetime(2024, 5, 17, 9, 47, 2, 664000, tz...Specification=None, specificationId='b3889263-fd40-4f4a-ab63-c2a975a0d083', InterviewStep=None, interviewStepId=None)]) 889: lang = 'python', eat_errors = False 890: @traceable 891: async def develop_application( 892: ids: Identifiers, 893: spec: Specification, 894: lang: str = "python", 895: eat_errors: bool = True, ... 931: lang, 932: ) 933: tasks.append(task) 934: # Run the tasks concurrently 935: results = await asyncio.gather(*tasks, return_exceptions=True) 936: if any(isinstance(r, Exception) for r in results): 937: exceptions = [r for r in results if isinstance(r, Exception)] 938: # if the only exceptions are LLMFailures, we can continue 939: if eat_errors and all(isinstance(r, LLMFailure) for r in exceptions): 940: logger.warning( 941: f"App Id: {ids.app_id} Eating Errors developing API routes: {exceptions}" 942: ) 943: pass 944: else: 945: error_message = "".join(f"\n* {e}" for e in exceptions) 946: > raise LLMFailure( 947: f"App Id: {ids.app_id} Error developing API routes: \n{error_message}" 948: ) 949: E ***.common.ai_block.LLMFailure: App Id: cc8fe485-d2df-40b7-becc-bbf63c8b2ee4 Error developing API routes: ... 962: > result = await generate_function() 963: ***/tests/gen_test.py:191: 964: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 965: ***/tests/gen_test.py:176: in generate_function 966: return await with_db_connection(execute) 967: ***/tests/gen_test.py:154: in with_db_connection 968: result = await func() 969: ***/tests/gen_test.py:173: in execute 970: func = await agent.develop_application(ids=ids, spec=spec, eat_errors=False) 971: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:451: in async_wrapper 972: raise e 973: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:440: in async_wrapper 974: function_result = await asyncio.create_task( # type: ignore[call-arg] 975: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 976: ids = Identifiers(user_id='123e4567-e89b-12d3-a456-426614174000', cloud_services_id='', app_id='bb7730b0-0cbe-4069-81e6-1c84...4906-bfa4-da43f8672540', function_id='aab3cfb8-bf54-4708-8efc-ed032a944cab', completed_app_id=None, deployment_id=None) 977: spec = Specification(id='3a877afb-2e02-46f5-92b3-36dec76c8e41', createdAt=datetime.datetime(2024, 5, 17, 9, 47, 2, 993000, tz...Specification=None, specificationId='3a877afb-2e02-46f5-92b3-36dec76c8e41', InterviewStep=None, interviewStepId=None)]) 978: lang = 'python', eat_errors = False 979: @traceable 980: async def develop_application( 981: ids: Identifiers, 982: spec: Specification, 983: lang: str = "python", 984: eat_errors: bool = True, ... 1020: lang, 1021: ) 1022: tasks.append(task) 1023: # Run the tasks concurrently 1024: results = await asyncio.gather(*tasks, return_exceptions=True) 1025: if any(isinstance(r, Exception) for r in results): 1026: exceptions = [r for r in results if isinstance(r, Exception)] 1027: # if the only exceptions are LLMFailures, we can continue 1028: if eat_errors and all(isinstance(r, LLMFailure) for r in exceptions): 1029: logger.warning( 1030: f"App Id: {ids.app_id} Eating Errors developing API routes: {exceptions}" 1031: ) 1032: pass 1033: else: 1034: error_message = "".join(f"\n* {e}" for e in exceptions) 1035: > raise LLMFailure( 1036: f"App Id: {ids.app_id} Error developing API routes: \n{error_message}" 1037: ) 1038: E ***.common.ai_block.LLMFailure: App Id: bb7730b0-0cbe-4069-81e6-1c84a877857d Error developing API routes: ... 1046: _________________________ test_unimplemented_function __________________________ 1047: @pytest.mark.asyncio 1048: @pytest.mark.integration_test 1049: async def test_unimplemented_function(): 1050: ai_block.MOCK_RESPONSE = WITH_UNIMPLEMENTED_FUNCTION_RESPONSE 1051: with pytest.raises(LLMFailure) as e: 1052: await generate_function() 1053: > assert "not implemented" in str(e.value) 1054: E assert 'not implemented' in "App Id: 7d609779-bf34-4aa2-b61c-e0c543968615 Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" 1055: E + where "App Id: 7d609779-bf34-4aa2-b61c-e0c543968615 Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" = str(LLMFailure("App Id: 7d609779-bf34-4aa2-b61c-e0c543968615 Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'")) 1056: E + where LLMFailure("App Id: 7d609779-bf34-4aa2-b61c-e0c543968615 Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'") = .value 1057: ***/tests/gen_test.py:201: AssertionError ... 1062: __________________________ test_mismatching_arguments __________________________ 1063: @pytest.mark.asyncio 1064: @pytest.mark.integration_test 1065: async def test_mismatching_arguments(): 1066: ai_block.MOCK_RESPONSE = WITH_MISMATCHING_ARGUMENTS_RESPONSE 1067: with pytest.raises(LLMFailure) as e: 1068: await generate_function() 1069: > assert "arguments" in str(e.value) 1070: E assert 'arguments' in "App Id: 91165ee1-d801-42ee-9ed8-1b40e3b2d74f Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" 1071: E + where "App Id: 91165ee1-d801-42ee-9ed8-1b40e3b2d74f Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" = str(LLMFailure("App Id: 91165ee1-d801-42ee-9ed8-1b40e3b2d74f Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'")) 1072: E + where LLMFailure("App Id: 91165ee1-d801-42ee-9ed8-1b40e3b2d74f Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'") = .value 1073: ***/tests/gen_test.py:210: AssertionError ... 1078: _________________________ test_mismatching_return_type _________________________ 1079: @pytest.mark.asyncio 1080: @pytest.mark.integration_test 1081: async def test_mismatching_return_type(): 1082: ai_block.MOCK_RESPONSE = WITH_MISMATCHING_RETURN_TYPE_RESPONSE 1083: with pytest.raises(LLMFailure) as e: 1084: await generate_function() 1085: > assert "return type" in str(e.value) 1086: E assert 'return type' in "App Id: 5fefd2e5-c026-4f66-8dcd-26661b675aad Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" 1087: E + where "App Id: 5fefd2e5-c026-4f66-8dcd-26661b675aad Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" = str(LLMFailure("App Id: 5fefd2e5-c026-4f66-8dcd-26661b675aad Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'")) 1088: E + where LLMFailure("App Id: 5fefd2e5-c026-4f66-8dcd-26661b675aad Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'") = .value 1089: ***/tests/gen_test.py:219: AssertionError ... 1099: > func = await generate_function() 1100: ***/tests/gen_test.py:226: 1101: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1102: ***/tests/gen_test.py:176: in generate_function 1103: return await with_db_connection(execute) 1104: ***/tests/gen_test.py:154: in with_db_connection 1105: result = await func() 1106: ***/tests/gen_test.py:173: in execute 1107: func = await agent.develop_application(ids=ids, spec=spec, eat_errors=False) 1108: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:451: in async_wrapper 1109: raise e 1110: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:440: in async_wrapper 1111: function_result = await asyncio.create_task( # type: ignore[call-arg] 1112: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1113: ids = Identifiers(user_id='123e4567-e89b-12d3-a456-426614174000', cloud_services_id='', app_id='bdacc2b4-c98c-4deb-a950-0af7...43ed-8666-cbff75dbac1a', function_id='f9c712d7-eabf-42b7-baf7-0ef1249dbf8f', completed_app_id=None, deployment_id=None) 1114: spec = Specification(id='01a7ac33-c7d9-4ff9-8fc3-483a17e127b1', createdAt=datetime.datetime(2024, 5, 17, 9, 47, 3, 389000, tz...Specification=None, specificationId='01a7ac33-c7d9-4ff9-8fc3-483a17e127b1', InterviewStep=None, interviewStepId=None)]) 1115: lang = 'python', eat_errors = False 1116: @traceable 1117: async def develop_application( 1118: ids: Identifiers, 1119: spec: Specification, 1120: lang: str = "python", 1121: eat_errors: bool = True, ... 1157: lang, 1158: ) 1159: tasks.append(task) 1160: # Run the tasks concurrently 1161: results = await asyncio.gather(*tasks, return_exceptions=True) 1162: if any(isinstance(r, Exception) for r in results): 1163: exceptions = [r for r in results if isinstance(r, Exception)] 1164: # if the only exceptions are LLMFailures, we can continue 1165: if eat_errors and all(isinstance(r, LLMFailure) for r in exceptions): 1166: logger.warning( 1167: f"App Id: {ids.app_id} Eating Errors developing API routes: {exceptions}" 1168: ) 1169: pass 1170: else: 1171: error_message = "".join(f"\n* {e}" for e in exceptions) 1172: > raise LLMFailure( 1173: f"App Id: {ids.app_id} Error developing API routes: \n{error_message}" 1174: ) 1175: E ***.common.ai_block.LLMFailure: App Id: bdacc2b4-c98c-4deb-a950-0af714236e58 Error developing API routes: ... 1188: > func = await generate_function() 1189: ***/tests/gen_test.py:235: 1190: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1191: ***/tests/gen_test.py:176: in generate_function 1192: return await with_db_connection(execute) 1193: ***/tests/gen_test.py:154: in with_db_connection 1194: result = await func() 1195: ***/tests/gen_test.py:173: in execute 1196: func = await agent.develop_application(ids=ids, spec=spec, eat_errors=False) 1197: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:451: in async_wrapper 1198: raise e 1199: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:440: in async_wrapper 1200: function_result = await asyncio.create_task( # type: ignore[call-arg] 1201: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1202: ids = Identifiers(user_id='123e4567-e89b-12d3-a456-426614174000', cloud_services_id='', app_id='90674861-4397-4410-94aa-4a3e...4a2b-9df4-8dacc476add8', function_id='4a1f32e8-dfb4-4eec-b4ad-5c4a1be86c07', completed_app_id=None, deployment_id=None) 1203: spec = Specification(id='3418f55b-6a49-4de8-9aab-232ff05576e0', createdAt=datetime.datetime(2024, 5, 17, 9, 47, 3, 516000, tz...Specification=None, specificationId='3418f55b-6a49-4de8-9aab-232ff05576e0', InterviewStep=None, interviewStepId=None)]) 1204: lang = 'python', eat_errors = False 1205: @traceable 1206: async def develop_application( 1207: ids: Identifiers, 1208: spec: Specification, 1209: lang: str = "python", 1210: eat_errors: bool = True, ... 1246: lang, 1247: ) 1248: tasks.append(task) 1249: # Run the tasks concurrently 1250: results = await asyncio.gather(*tasks, return_exceptions=True) 1251: if any(isinstance(r, Exception) for r in results): 1252: exceptions = [r for r in results if isinstance(r, Exception)] 1253: # if the only exceptions are LLMFailures, we can continue 1254: if eat_errors and all(isinstance(r, LLMFailure) for r in exceptions): 1255: logger.warning( 1256: f"App Id: {ids.app_id} Eating Errors developing API routes: {exceptions}" 1257: ) 1258: pass 1259: else: 1260: error_message = "".join(f"\n* {e}" for e in exceptions) 1261: > raise LLMFailure( 1262: f"App Id: {ids.app_id} Error developing API routes: \n{error_message}" 1263: ) 1264: E ***.common.ai_block.LLMFailure: App Id: 90674861-4397-4410-94aa-4a3e804e850e Error developing API routes: ... 1277: > func = await generate_function() 1278: ***/tests/gen_test.py:243: 1279: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1280: ***/tests/gen_test.py:176: in generate_function 1281: return await with_db_connection(execute) 1282: ***/tests/gen_test.py:154: in with_db_connection 1283: result = await func() 1284: ***/tests/gen_test.py:173: in execute 1285: func = await agent.develop_application(ids=ids, spec=spec, eat_errors=False) 1286: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:451: in async_wrapper 1287: raise e 1288: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:440: in async_wrapper 1289: function_result = await asyncio.create_task( # type: ignore[call-arg] 1290: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1291: ids = Identifiers(user_id='123e4567-e89b-12d3-a456-426614174000', cloud_services_id='', app_id='55e60d5b-bf47-4889-b696-8534...41ad-bccb-e3ebeac5ef2d', function_id='1d438122-7da0-4e5b-91fd-ea56c064b1c2', completed_app_id=None, deployment_id=None) 1292: spec = Specification(id='35cf243e-766e-47c1-a668-cbac30a73110', createdAt=datetime.datetime(2024, 5, 17, 9, 47, 3, 642000, tz...Specification=None, specificationId='35cf243e-766e-47c1-a668-cbac30a73110', InterviewStep=None, interviewStepId=None)]) 1293: lang = 'python', eat_errors = False 1294: @traceable 1295: async def develop_application( 1296: ids: Identifiers, 1297: spec: Specification, 1298: lang: str = "python", 1299: eat_errors: bool = True, ... 1335: lang, 1336: ) 1337: tasks.append(task) 1338: # Run the tasks concurrently 1339: results = await asyncio.gather(*tasks, return_exceptions=True) 1340: if any(isinstance(r, Exception) for r in results): 1341: exceptions = [r for r in results if isinstance(r, Exception)] 1342: # if the only exceptions are LLMFailures, we can continue 1343: if eat_errors and all(isinstance(r, LLMFailure) for r in exceptions): 1344: logger.warning( 1345: f"App Id: {ids.app_id} Eating Errors developing API routes: {exceptions}" 1346: ) 1347: pass 1348: else: 1349: error_message = "".join(f"\n* {e}" for e in exceptions) 1350: > raise LLMFailure( 1351: f"App Id: {ids.app_id} Error developing API routes: \n{error_message}" 1352: ) 1353: E ***.common.ai_block.LLMFailure: App Id: 55e60d5b-bf47-4889-b696-8534627a7e34 Error developing API routes: ... 1366: > func = await generate_function() 1367: ***/tests/gen_test.py:254: 1368: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1369: ***/tests/gen_test.py:176: in generate_function 1370: return await with_db_connection(execute) 1371: ***/tests/gen_test.py:154: in with_db_connection 1372: result = await func() 1373: ***/tests/gen_test.py:173: in execute 1374: func = await agent.develop_application(ids=ids, spec=spec, eat_errors=False) 1375: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:451: in async_wrapper 1376: raise e 1377: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:440: in async_wrapper 1378: function_result = await asyncio.create_task( # type: ignore[call-arg] 1379: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1380: ids = Identifiers(user_id='123e4567-e89b-12d3-a456-426614174000', cloud_services_id='', app_id='8f3d6fc7-2799-44a6-9a71-5d35...4a7c-a3eb-fd3d042f4287', function_id='9c4264de-9df7-4075-a577-ebf999bc9da1', completed_app_id=None, deployment_id=None) 1381: spec = Specification(id='8b8f8c51-bff8-41a3-ba8e-86a5cde1e79a', createdAt=datetime.datetime(2024, 5, 17, 9, 47, 3, 774000, tz...Specification=None, specificationId='8b8f8c51-bff8-41a3-ba8e-86a5cde1e79a', InterviewStep=None, interviewStepId=None)]) 1382: lang = 'python', eat_errors = False 1383: @traceable 1384: async def develop_application( 1385: ids: Identifiers, 1386: spec: Specification, 1387: lang: str = "python", 1388: eat_errors: bool = True, ... 1424: lang, 1425: ) 1426: tasks.append(task) 1427: # Run the tasks concurrently 1428: results = await asyncio.gather(*tasks, return_exceptions=True) 1429: if any(isinstance(r, Exception) for r in results): 1430: exceptions = [r for r in results if isinstance(r, Exception)] 1431: # if the only exceptions are LLMFailures, we can continue 1432: if eat_errors and all(isinstance(r, LLMFailure) for r in exceptions): 1433: logger.warning( 1434: f"App Id: {ids.app_id} Eating Errors developing API routes: {exceptions}" 1435: ) 1436: pass 1437: else: 1438: error_message = "".join(f"\n* {e}" for e in exceptions) 1439: > raise LLMFailure( 1440: f"App Id: {ids.app_id} Error developing API routes: \n{error_message}" 1441: ) 1442: E ***.common.ai_block.LLMFailure: App Id: 8f3d6fc7-2799-44a6-9a71-5d359781016b Error developing API routes: ... 1455: > result = await generate_function() 1456: ***/tests/gen_test.py:266: 1457: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1458: ***/tests/gen_test.py:176: in generate_function 1459: return await with_db_connection(execute) 1460: ***/tests/gen_test.py:154: in with_db_connection 1461: result = await func() 1462: ***/tests/gen_test.py:173: in execute 1463: func = await agent.develop_application(ids=ids, spec=spec, eat_errors=False) 1464: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:451: in async_wrapper 1465: raise e 1466: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/langsmith/run_helpers.py:440: in async_wrapper 1467: function_result = await asyncio.create_task( # type: ignore[call-arg] 1468: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1469: ids = Identifiers(user_id='123e4567-e89b-12d3-a456-426614174000', cloud_services_id='', app_id='1f7bd3b3-2c49-4bea-85c6-e20a...48d1-b345-6e78633d5927', function_id='e85e0c39-7478-4567-b7a9-29b3dc115ad0', completed_app_id=None, deployment_id=None) 1470: spec = Specification(id='2c423fe0-4578-462d-b0e3-8b62c180288c', createdAt=datetime.datetime(2024, 5, 17, 9, 47, 3, 898000, tz...Specification=None, specificationId='2c423fe0-4578-462d-b0e3-8b62c180288c', InterviewStep=None, interviewStepId=None)]) 1471: lang = 'python', eat_errors = False 1472: @traceable 1473: async def develop_application( 1474: ids: Identifiers, 1475: spec: Specification, 1476: lang: str = "python", 1477: eat_errors: bool = True, ... 1513: lang, 1514: ) 1515: tasks.append(task) 1516: # Run the tasks concurrently 1517: results = await asyncio.gather(*tasks, return_exceptions=True) 1518: if any(isinstance(r, Exception) for r in results): 1519: exceptions = [r for r in results if isinstance(r, Exception)] 1520: # if the only exceptions are LLMFailures, we can continue 1521: if eat_errors and all(isinstance(r, LLMFailure) for r in exceptions): 1522: logger.warning( 1523: f"App Id: {ids.app_id} Eating Errors developing API routes: {exceptions}" 1524: ) 1525: pass 1526: else: 1527: error_message = "".join(f"\n* {e}" for e in exceptions) 1528: > raise LLMFailure( 1529: f"App Id: {ids.app_id} Error developing API routes: \n{error_message}" 1530: ) 1531: E ***.common.ai_block.LLMFailure: App Id: 1f7bd3b3-2c49-4bea-85c6-e20a6d164c44 Error developing API routes: ... 1552: timeout = datetime.timedelta(seconds=10), datasources = None 1553: async def connect( 1554: self, 1555: timeout: timedelta = DEFAULT_CONNECT_TIMEOUT, 1556: datasources: Optional[List[DatasourceOverride]] = None, 1557: ) -> None: 1558: log.debug('Connecting to query engine') 1559: if self.process is not None: 1560: > raise errors.AlreadyConnectedError('Already connected to the query engine') 1561: E prisma.engine.errors.AlreadyConnectedError: Already connected to the query engine 1562: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/prisma/engine/query.py:131: AlreadyConnectedError 1563: =============================== warnings summary =============================== 1564: ../../../.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/fuzzywuzzy/fuzz.py:11 1565: /home/runner/.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning 1566: warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning') 1567: ***/common/parse_prisma.py:31 1568: /home/runner/work/***/***/***/common/parse_prisma.py:31: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ 1569: @validator("provider") 1570: ***/common/parse_prisma.py:44 1571: /home/runner/work/***/***/***/common/parse_prisma.py:44: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ ... 1588: ***/tests/type_parsing_test.py:26 1589: /home/runner/work/***/***/***/tests/type_parsing_test.py:26: PytestUnknownMarkWarning: Unknown pytest.mark.unit - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html 1590: @pytest.mark.unit 1591: ***/tests/endpoint_fixing_test.py::test_using_model_not_in_db 1592: ***/tests/endpoint_fixing_test.py::test_cross_model 1593: /home/runner/work/***/***/***/requirements/blocks/ai_endpoint.py:330: DeprecationWarning: The 'warn' function is deprecated, use 'warning' instead 1594: logging.warn(f"Resolving invalid type: {invalid_type}") 1595: ***/tests/endpoint_fixing_test.py::test_cross_model 1596: /home/runner/.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/pydantic/main.py:1265: PydanticDeprecatedSince20: The `copy` method is deprecated; use `model_copy` instead. See the docstring of `BaseModel.copy` for details about how to handle `include` and `exclude`. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ ... 1599: ***/tests/api_test.py::test_apps_apis 1600: ***/tests/api_test.py::test_specs_apis 1601: /home/runner/.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead. 1602: warnings.warn(message, DeprecationWarning) 1603: ***/tests/gen_test.py::test_simple_function 1604: /home/runner/.cache/pypoetry/virtualenvs/***-ORswSzyp-py3.11/lib/python3.11/site-packages/pytest_asyncio/plugin.py:759: DeprecationWarning: The event_loop fixture provided by pytest-asyncio has been redefined in 1605: /home/runner/work/***/***/***/tests/gen_test.py:34 1606: Replacing the event_loop fixture with a custom implementation is deprecated 1607: and will lead to errors in the future. ... 1697: ***/tests/frontend_gen_test.py 89 57 36% 1698: ***/tests/gen_test.py 137 50 64% 1699: ***/tests/model_test.py 24 13 46% 1700: ***/tests/prisma_parse_test.py 178 0 100% 1701: ***/tests/type_parsing_test.py 14 0 100% 1702: -------------------------------------------------------------------- 1703: TOTAL 6166 3206 48% 1704: =========================== short test summary info ============================ 1705: FAILED ***/tests/gen_test.py::test_simple_function - ***.common.ai_block.LLMFailure: App Id: cc8fe485-d2df-40b7-becc-bbf63c8b2ee4 Error developing API routes: 1706: * [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o' 1707: FAILED ***/tests/gen_test.py::test_global_variable - ***.common.ai_block.LLMFailure: App Id: bb7730b0-0cbe-4069-81e6-1c84a877857d Error developing API routes: 1708: * [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o' 1709: FAILED ***/tests/gen_test.py::test_unimplemented_function - assert 'not implemented' in "App Id: 7d609779-bf34-4aa2-b61c-e0c543968615 Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" 1710: + where "App Id: 7d609779-bf34-4aa2-b61c-e0c543968615 Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" = str(LLMFailure("App Id: 7d609779-bf34-4aa2-b61c-e0c543968615 Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'")) 1711: + where LLMFailure("App Id: 7d609779-bf34-4aa2-b61c-e0c543968615 Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'") = .value 1712: FAILED ***/tests/gen_test.py::test_mismatching_arguments - assert 'arguments' in "App Id: 91165ee1-d801-42ee-9ed8-1b40e3b2d74f Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" 1713: + where "App Id: 91165ee1-d801-42ee-9ed8-1b40e3b2d74f Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" = str(LLMFailure("App Id: 91165ee1-d801-42ee-9ed8-1b40e3b2d74f Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'")) 1714: + where LLMFailure("App Id: 91165ee1-d801-42ee-9ed8-1b40e3b2d74f Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'") = .value 1715: FAILED ***/tests/gen_test.py::test_mismatching_return_type - assert 'return type' in "App Id: 5fefd2e5-c026-4f66-8dcd-26661b675aad Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" 1716: + where "App Id: 5fefd2e5-c026-4f66-8dcd-26661b675aad Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'" = str(LLMFailure("App Id: 5fefd2e5-c026-4f66-8dcd-26661b675aad Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'")) 1717: + where LLMFailure("App Id: 5fefd2e5-c026-4f66-8dcd-26661b675aad Error developing API routes: \n\n* [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o'") = .value 1718: FAILED ***/tests/gen_test.py::test_nested_function - ***.common.ai_block.LLMFailure: App Id: bdacc2b4-c98c-4deb-a950-0af714236e58 Error developing API routes: 1719: * [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o' 1720: FAILED ***/tests/gen_test.py::test_with_llm_function_generation - ***.common.ai_block.LLMFailure: App Id: 90674861-4397-4410-94aa-4a3e804e850e Error developing API routes: 1721: * [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o' 1722: FAILED ***/tests/gen_test.py::test_class_with_optional_field - ***.common.ai_block.LLMFailure: App Id: 55e60d5b-bf47-4889-b696-8534627a7e34 Error developing API routes: 1723: * [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o' 1724: FAILED ***/tests/gen_test.py::test_class_with_db_query - ***.common.ai_block.LLMFailure: App Id: 8f3d6fc7-2799-44a6-9a71-5d359781016b Error developing API routes: 1725: * [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o' 1726: FAILED ***/tests/gen_test.py::test_with_undefined_entity - ***.common.ai_block.LLMFailure: App Id: 1f7bd3b3-2c49-4bea-85c6-e20a6d164c44 Error developing API routes: 1727: * [Errno 2] No such file or directory: '/home/runner/work/***/***/***/prompts/gpt-4o' 1728: FAILED ***/tests/model_test.py::test_create_nested_object_type - prisma.engine.errors.AlreadyConnectedError: Already connected to the query engine 1729: ============ 11 failed, 48 passed, 4 skipped, 17 warnings in 55.35s ============ 1730: Sentry is attempting to send 2 pending events 1731: Waiting up to 2 seconds 1732: Press Ctrl-C to quit 1733: ##[error]Process completed with exit code 1. ... 1766: 2024-05-17 09:44:50.291 UTC [64] LOG: database system was shut down at 2024-05-17 09:44:50 UTC 1767: 2024-05-17 09:44:50.295 UTC [1] LOG: database system is ready to accept connections 1768: 2024-05-17 09:44:59.276 UTC [75] FATAL: database "***" does not exist 1769: 2024-05-17 09:45:09.409 UTC [82] FATAL: database "***" does not exist 1770: 2024-05-17 09:45:19.500 UTC [90] FATAL: database "***" does not exist 1771: 2024-05-17 09:45:29.616 UTC [99] FATAL: database "***" does not exist 1772: 2024-05-17 09:45:39.710 UTC [108] FATAL: database "***" does not exist 1773: 2024-05-17 09:45:49.826 UTC [116] FATAL: database "***" does not exist 1774: 2024-05-17 09:45:56.353 UTC [119] ERROR: relation "_prisma_migrations" does not exist at character 126 ```

✨ CI feedback usage guide:

The CI feedback tool (`/checks)` automatically triggers when a PR has a failed check. The tool analyzes the failed checks and provides several feedbacks: - Failed stage - Failed test name - Failure summary - Relevant error logs In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR: ``` /checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}" ``` where `{repo_name}` is the name of the repository, `{run_number}` is the run number of the failed check, and `{job_number}` is the job number of the failed check. #### Configuration options - `enable_auto_checks_feedback` - if set to true, the tool will automatically provide feedback when a check is failed. Default is true. - `excluded_checks_list` - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list. - `enable_help_text` - if set to true, the tool will provide a help message with the feedback. Default is true. - `persistent_comment` - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true. - `final_update_message` - if `persistent_comment` is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true. See more information about the `checks` tool in the [docs](https://pr-agent-docs.codium.ai/tools/ci_feedback/).