search

nextcloud / integration_openai

OpenAI integration in Nextcloud
GNU Affero General Public License v3.0
52 stars 11 forks source link

getModels method blocks all user requests #146

Closed Azure99 closed 2 weeks ago

Azure99 commented 1 month ago

Which version of integration_openai are you using?

3.1.2

Which version of Nextcloud are you using?

v30.0.0

Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.

No response

Describe the Bug

I am using Nextcloud Assistant, but it seems that each user request triggers a getModels call, and the request is blocked until the call completes. Below is the call stack. As you can see, the LoadAdditionalScriptsEvent event is captured by the TopicsProvider, which initiates the call.

{"reqId":"SQDjGMnNmocXnOjNgXo6","level":2,"time":"2024-10-18T08:18:01+00:00","remoteAddr":"mask","user":"Azure99","app":"integration_openai","method":"GET","url":"/apps/files/","message":"API request error : Server error: `GET https://mask/v1/models` resulted in a `502 Bad Gateway` response:\n<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>ngin (truncated...)\n","userAgent":"Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Mobile Safari/537.36","version":"30.0.0.14","exception":{"Exception":"GuzzleHttp\\Exception\\ServerException","Message":"Server error: `GET https://mask/v1/models` resulted in a `502 Bad Gateway` response:\n<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>ngin (truncated...)\n","Code":502,"Trace":[{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":72,"function":"create","class":"GuzzleHttp\\Exception\\RequestException","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":209,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":158,"function":"callHandler","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":[1,"*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/TaskQueue.php","line":52,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":251,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->","args":[true]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":227,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":272,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":229,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":69,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Client.php","line":189,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/lib/private/Http/Client/Client.php","line":205,"function":"request","class":"GuzzleHttp\\Client","type":"->","args":["get","https://mask/v1/models",["/var/www/html/resources/config/ca-bundle.crt",240,["proxy:7890","proxy:7890"],["Nextcloud OpenAI/LocalAI integration","Bearer sk-aqszJ0p1xc0xvsUv0fB5B19cCbA04bB5B2E469Bd856cA62a","application/json","gzip"],[true],"And 1 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/html/custom_apps/integration_openai/lib/Service/OpenAiAPIService.php","line":743,"function":"get","class":"OC\\Http\\Client\\Client","type":"->","args":["https://mask/v1/models",[240,["Nextcloud OpenAI/LocalAI integration","Bearer sk-aqszJ0p1xc0xvsUv0fB5B19cCbA04bB5B2E469Bd856cA62a","application/json"],[true]]]},{"file":"/var/www/html/custom_apps/integration_openai/lib/Service/OpenAiAPIService.php","line":86,"function":"request","class":"OCA\\OpenAi\\Service\\OpenAiAPIService","type":"->","args":["Azure99","models"]},{"file":"/var/www/html/custom_apps/integration_openai/lib/Service/OpenAiAPIService.php","line":121,"function":"getModels","class":"OCA\\OpenAi\\Service\\OpenAiAPIService","type":"->","args":["Azure99"]},{"file":"/var/www/html/custom_apps/integration_openai/lib/TaskProcessing/TopicsProvider.php","line":71,"function":"getModelEnumValues","class":"OCA\\OpenAi\\Service\\OpenAiAPIService","type":"->","args":["Azure99"]},{"file":"/var/www/html/lib/private/TaskProcessing/Manager.php","line":740,"function":"getOptionalInputShapeEnumValues","class":"OCA\\OpenAi\\TaskProcessing\\TopicsProvider","type":"->","args":[]},{"file":"/var/www/html/apps/text/lib/Service/InitialStateProvider.php","line":65,"function":"getAvailableTaskTypes","class":"OC\\TaskProcessing\\Manager","type":"->","args":[]},{"file":"/var/www/html/apps/text/lib/Listeners/FilesLoadAdditionalScriptsListener.php","line":37,"function":"provideState","class":"OCA\\Text\\Service\\InitialStateProvider","type":"->","args":[]},{"file":"/var/www/html/lib/private/EventDispatcher/ServiceEventListener.php","line":68,"function":"handle","class":"OCA\\Text\\Listeners\\FilesLoadAdditionalScriptsListener","type":"->","args":[["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]]},{"file":"/var/www/html/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":230,"function":"__invoke","class":"OC\\EventDispatcher\\ServiceEventListener","type":"->","args":[["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"],"OCA\\Files\\Event\\LoadAdditionalScriptsEvent",["Symfony\\Component\\EventDispatcher\\EventDispatcher"]]},{"file":"/var/www/html/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":59,"function":"callListeners","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->","args":[[["Closure"],["Closure"],["Closure"],["Closure"],["Closure"],"And 9 more entries, set log level to debug to see all entries"],"OCA\\Files\\Event\\LoadAdditionalScriptsEvent",["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]]},{"file":"/var/www/html/lib/private/EventDispatcher/EventDispatcher.php","line":67,"function":"dispatch","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->","args":[["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"],"OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]},{"file":"/var/www/html/lib/private/EventDispatcher/EventDispatcher.php","line":79,"function":"dispatch","class":"OC\\EventDispatcher\\EventDispatcher","type":"->","args":["OCA\\Files\\Event\\LoadAdditionalScriptsEvent",["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]]},{"file":"/var/www/html/apps/files/lib/Controller/ViewController.php","line":187,"function":"dispatchTyped","class":"OC\\EventDispatcher\\EventDispatcher","type":"->","args":[["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":208,"function":"index","class":"OCA\\Files\\Controller\\ViewController","type":"->","args":["","","*** sensitive parameters replaced ***",false]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":114,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[["OCA\\Files\\Controller\\ViewController"],"index"]},{"file":"/var/www/html/lib/private/AppFramework/App.php","line":161,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[["OCA\\Files\\Controller\\ViewController"],"index"]},{"file":"/var/www/html/lib/private/Route/Router.php","line":302,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\Files\\Controller\\ViewController","index",["OC\\AppFramework\\DependencyInjection\\DIContainer"],["files.view.index"]]},{"file":"/var/www/html/lib/base.php","line":1001,"function":"match","class":"OC\\Route\\Router","type":"->","args":["/apps/files/"]},{"file":"/var/www/html/index.php","line":24,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Exception/RequestException.php","Line":113,"message":"API request error : Server error: `GET https://mask/v1/models` resulted in a `502 Bad Gateway` response:\n<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>ngin (truncated...)\n","response_body":{},"exception":{},"CustomMessage":"API request error : Server error: `GET https://mask/v1/models` resulted in a `502 Bad Gateway` response:\n<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>ngin (truncated...)\n"}}
{"reqId":"SQDjGMnNmocXnOjNgXo6","level":2,"time":"2024-10-18T08:18:01+00:00","remoteAddr":"mask","user":"Azure99","app":"integration_openai","method":"GET","url":"/apps/files/","message":"Error getting model enum values","userAgent":"Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Mobile Safari/537.36","version":"30.0.0.14","exception":{"Exception":"Exception","Message":"API request error: Server error: `GET https://mask/v1/models` resulted in a `502 Bad Gateway` response:\n<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>ngin (truncated...)\n","Code":502,"Trace":[{"file":"/var/www/html/custom_apps/integration_openai/lib/Service/OpenAiAPIService.php","line":86,"function":"request","class":"OCA\\OpenAi\\Service\\OpenAiAPIService","type":"->","args":["Azure99","models"]},{"file":"/var/www/html/custom_apps/integration_openai/lib/Service/OpenAiAPIService.php","line":121,"function":"getModels","class":"OCA\\OpenAi\\Service\\OpenAiAPIService","type":"->","args":["Azure99"]},{"file":"/var/www/html/custom_apps/integration_openai/lib/TaskProcessing/TopicsProvider.php","line":71,"function":"getModelEnumValues","class":"OCA\\OpenAi\\Service\\OpenAiAPIService","type":"->","args":["Azure99"]},{"file":"/var/www/html/lib/private/TaskProcessing/Manager.php","line":740,"function":"getOptionalInputShapeEnumValues","class":"OCA\\OpenAi\\TaskProcessing\\TopicsProvider","type":"->","args":[]},{"file":"/var/www/html/apps/text/lib/Service/InitialStateProvider.php","line":65,"function":"getAvailableTaskTypes","class":"OC\\TaskProcessing\\Manager","type":"->","args":[]},{"file":"/var/www/html/apps/text/lib/Listeners/FilesLoadAdditionalScriptsListener.php","line":37,"function":"provideState","class":"OCA\\Text\\Service\\InitialStateProvider","type":"->","args":[]},{"file":"/var/www/html/lib/private/EventDispatcher/ServiceEventListener.php","line":68,"function":"handle","class":"OCA\\Text\\Listeners\\FilesLoadAdditionalScriptsListener","type":"->","args":[["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]]},{"file":"/var/www/html/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":230,"function":"__invoke","class":"OC\\EventDispatcher\\ServiceEventListener","type":"->","args":[["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"],"OCA\\Files\\Event\\LoadAdditionalScriptsEvent",["Symfony\\Component\\EventDispatcher\\EventDispatcher"]]},{"file":"/var/www/html/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":59,"function":"callListeners","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->","args":[[["Closure"],["Closure"],["Closure"],["Closure"],["Closure"],"And 9 more entries, set log level to debug to see all entries"],"OCA\\Files\\Event\\LoadAdditionalScriptsEvent",["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]]},{"file":"/var/www/html/lib/private/EventDispatcher/EventDispatcher.php","line":67,"function":"dispatch","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->","args":[["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"],"OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]},{"file":"/var/www/html/lib/private/EventDispatcher/EventDispatcher.php","line":79,"function":"dispatch","class":"OC\\EventDispatcher\\EventDispatcher","type":"->","args":["OCA\\Files\\Event\\LoadAdditionalScriptsEvent",["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]]},{"file":"/var/www/html/apps/files/lib/Controller/ViewController.php","line":187,"function":"dispatchTyped","class":"OC\\EventDispatcher\\EventDispatcher","type":"->","args":[["OCA\\Files\\Event\\LoadAdditionalScriptsEvent"]]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":208,"function":"index","class":"OCA\\Files\\Controller\\ViewController","type":"->","args":["","",null,false]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":114,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[["OCA\\Files\\Controller\\ViewController"],"index"]},{"file":"/var/www/html/lib/private/AppFramework/App.php","line":161,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[["OCA\\Files\\Controller\\ViewController"],"index"]},{"file":"/var/www/html/lib/private/Route/Router.php","line":302,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\Files\\Controller\\ViewController","index",["OC\\AppFramework\\DependencyInjection\\DIContainer"],["files.view.index"]]},{"file":"/var/www/html/lib/base.php","line":1001,"function":"match","class":"OC\\Route\\Router","type":"->","args":["/apps/files/"]},{"file":"/var/www/html/index.php","line":24,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/custom_apps/integration_openai/lib/Service/OpenAiAPIService.php","Line":772,"message":"Error getting model enum values","exception":{},"CustomMessage":"Error getting model enum values"}}

While reviewing the app's logs, I noticed that a single user request triggers multiple calls to getModels, but it seems that the cache in the getModels method is only effective for the lifecycle of one request.

{"reqId":"soGKNL28QGD0GAYoEYre","level":2,"time":"2024-10-18T09:15:14+00:00","remoteAddr":"mask","user":"Azure99","app":"integration_openai","method":"GET","url":"/settings/apps","message":"Actually getting OpenAI models with a network request","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36","version":"30.0.0.14","data":{"app":"integration_openai"}}
{"reqId":"soGKNL28QGD0GAYoEYre","level":2,"time":"2024-10-18T09:15:14+00:00","remoteAddr":"mask","user":"Azure99","app":"integration_openai","method":"GET","url":"/settings/apps","message":"Getting OpenAI models from the memory cache","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36","version":"30.0.0.14","data":{"app":"integration_openai"}}
{"reqId":"soGKNL28QGD0GAYoEYre","level":2,"time":"2024-10-18T09:15:14+00:00","remoteAddr":"mask","user":"Azure99","app":"integration_openai","method":"GET","url":"/settings/apps","message":"Getting OpenAI models from the memory cache","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36","version":"30.0.0.14","data":{"app":"integration_openai"}}
{"reqId":"soGKNL28QGD0GAYoEYre","level":2,"time":"2024-10-18T09:15:14+00:00","remoteAddr":"mask","user":"Azure99","app":"integration_openai","method":"GET","url":"/settings/apps","message":"Getting OpenAI models from the memory cache","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36","version":"30.0.0.14","data":{"app":"integration_openai"}}
{"reqId":"soGKNL28QGD0GAYoEYre","level":2,"time":"2024-10-18T09:15:14+00:00","remoteAddr":"mask","user":"Azure99","app":"integration_openai","method":"GET","url":"/settings/apps","message":"Getting OpenAI models from the memory cache","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36","version":"30.0.0.14","data":{"app":"integration_openai"}}
{"reqId":"soGKNL28QGD0GAYoEYre","level":2,"time":"2024-10-18T09:15:14+00:00","remoteAddr":"mask","user":"Azure99","app":"integration_openai","method":"GET","url":"/settings/apps","message":"Getting OpenAI models from the memory cache","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36","version":"30.0.0.14","data":{"app":"integration_openai"}}

Expected Behavior

getModels should persist cached results for a period of time after the first request initiated by the user, instead of making a network request on every request.

To Reproduce

Install the app and Nextcloud Assistant and configuring the URL, if the URL endpoint url/v1/models takes a long time to timeout, each user request will remain blocked during this period.

julien-nc commented 2 weeks ago

Thanks for reporting this. #152 should solve this.