search

silvanmelchior / IncognitoPilot

An AI code interpreter for sensitive data, powered by GPT-4 or Code Llama / Llama 2.
MIT License
425 stars 44 forks source link

GPT-AZURE: Internal Error #24

Closed TheDataVoyager closed 1 year ago

TheDataVoyager commented 1 year ago

Hello,

Thank you for the great tool!

I checked it out with the gpt-openai mode and it worked well for me.

I then wanted to use it with gpt-azure and kept getting "ERROR: Internal Error" for anything I submit. I checked this with Mac Ventura 13.5.1 Docker and colima, with GCP docker, with amd and arm processors, and with full and slim images. I also checked building the images locally. The creds were verified with a simple curl/python to be working fine and getting a response.

Is this a known issue under work?

Thanks in advance

wangxing7714436 commented 1 year ago

Yeah, the same. Do you use azure gpt-35-turbo model? It seems there is an issue with 0630 preview api version associated with function calling feature.

TheDataVoyager commented 1 year ago

I checked for both this and gpt-4. Both gave the same issue. My model versions in Azure are 0613.

wangxing7714436 commented 1 year ago

I checked for both this and gpt-4. Both gave the same issue. My model versions in Azure are 0613.

Mine is 0613 too, sorry for the typo. I don't have gpt-4 access yet, since you tried gpt-4, the issue is not related to the function calling feature then.

silvanmelchior commented 1 year ago

Hi, thank you very much for the reported issue. I unfortunately don't have access to function calling on Azure, so I couldn't try it out and just had ro rely on their documentation. Since you have access to the 0613 models, you should have however, so that's very helpful. Do you see further infos in the console? The container should output the error which caused the internal error. If not, since you said, you also built the container yourself, would you consider changing the code and doing so again? Specifially, in https://github.com/silvanmelchior/IncognitoPilot/blob/main/services/main_llm.py#L50, could you remove the 5-lines if-else and just write error = str(e) instead? Then in the UI you should get more information of what is going on (which per default is hidden for security reasons, except if the underlying model chooses to show an error to the user with LLMException)

silvanmelchior commented 1 year ago

I just found out that I just need to move my subscription, then I also have access to the 0613 models. So I can debug it myself as well, once I find the time

silvanmelchior commented 1 year ago

Should be fixed in 2.0.2 (release on the way)

TheDataVoyager commented 1 year ago

Thanks for making the changes!

I built the merged code with Colima (I needed to make a change and add a slash to the Dockerfile ->"COPY docker/start* ./ "). Then the first message and answer worked but when asking it to write a file it hung. image.

Then reset the session and tried again. I then got the error: "Error: The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 if the error persists. (Please include the request ID 68d03f53-d51f-4f58-8947-f6959e69126d in your message.) { "error": { "message": "The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 if the error persists. (Please include the request ID 68d03f53-d51f-4f58-8947-f6959e69126d in your message.)", "type": "server_error", "param": null, "code": null } } 500 {'error': {'message': 'The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 if the error persists. (Please include the request ID 68d03f53-d51f-4f58-8947-f6959e69126d in your message.)', 'type': 'server_error', 'param': None, 'code': None}} {'Content-Length': '412', 'Content-Type': 'application/json', 'access-control-allow-origin': '*', 'apim-request-id': '90264018-b837-44a6-8559-cd4619642cde', 'x-content-type-options': 'nosniff', 'openai-processing-ms': '30022.9978', 'x-ms-region': 'East US 2', 'x-request-id': '68d03f53-d51f-4f58-8947-f6959e69126d', 'ms-azureml-model-error-reason': 'model_error', 'ms-azureml-model-error-statuscode': '500', 'x-ms-client-request-id': '90264018-b837-44a6-8559-cd4619642cde', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'Date': 'Wed, 23 Aug 2023 15:30:02 GMT'}"

If I tried a few times to discuss with it prior to asking for a file write up it worked.

Docker logs: docker logs 45245d2e6a74 INFO: Started server process [15] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) INFO: Started server process [11] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8081 (Press CTRL+C to quit) INFO: ('127.0.0.1', 56820) - "WebSocket /api/llm/chat" [accepted] INFO: connection open INFO: connection closed INFO: ('127.0.0.1', 40578) - "WebSocket /api/llm/chat" [accepted] INFO: connection open INFO: connection closed INFO: ('127.0.0.1', 51036) - "WebSocket /api/interpreter/run" [accepted] INFO: connection open INFO: ('127.0.0.1', 39872) - "WebSocket /api/llm/chat" [accepted] INFO: connection open 2023/08/23 15:23:37 [error] 14#14: 20 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.17.0.1, server: , request: "GET /api/llm/chat HTTP/1.1", upstream: "http://127.0.0.1:8081/api/llm/chat", host: "localhost:3040" 2023/08/23 15:24:37 [error] 14#14: 20 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.17.0.1, server: , request: "GET /api/llm/chat HTTP/1.1", upstream: "http://127.0.0.1:8081/api/llm/chat", host: "localhost:3040" INFO: connection closed INFO: ('127.0.0.1', 37682) - "WebSocket /api/llm/chat" [accepted] INFO: ('127.0.0.1', 56820) - "WebSocket /api/llm/chat" [accepted] INFO: connection closed INFO: connection closed INFO: ('127.0.0.1', 49324) - "WebSocket /api/llm/chat" [accepted] INFO: connection open INFO: connection closed INFO: ('127.0.0.1', 42766) - "WebSocket /api/llm/chat" [accepted] INFO: connection open INFO: connection closed INFO: ('127.0.0.1', 46684) - "WebSocket /api/llm/chat" [accepted] INFO: connection open INFO: connection closed

silvanmelchior commented 1 year ago

Hmm, I can't reproduce the behaviour, tried it out a few times now: azure

I use the GPT-3.5 model in France. Don't have access to GPT-4 there. To me, this sound a lot like an Azure problem, especially if it also workes sometimes.

BTW, you don't need to rebuild the container, it's on docker hub with the tag 2.0.2 (or 2.0.2-slim), the build is here: https://github.com/silvanmelchior/IncognitoPilot/actions/runs/5950942655.

TheDataVoyager commented 1 year ago

It might have been an API long latency - like reaching the max number of requests per minute. Later runs worked ok. Thanks!

wangxing7714436 commented 1 year ago

@silvanmelchior great work,mine's working well now. Thank you! btw, would you mind adding a history chat feature? Since there might be multiple projects ruuning at the same time.

TheDataVoyager commented 1 year ago

I asked for the same :)

I added my ask to the history feature that is the list from @silvanmelchior .

silvanmelchior commented 1 year ago

Yes, please see issue #10. At the moment, it's not highest priority however, I want to invest in improving the current state first (e.g. in terms of security, like #14), before adding new features.