Not able to use OpenAI for Chat and Toolbox when self-hosting

[m0ngr31]

I'm self-hosting Refact, went and created an API key for OpenAI, and enabled the checkbox for enabling it in the dashboard:

When I try to open the chat, I can't do anything and I'm unable to select a model to chat with:

I've tried selecting Enterprise and Self-Hosting. Both behave the same way. Is there something I missed on setup?

[olegklimov]

It's possible you just need to restart IDE. That's https://github.com/smallcloudai/refact/issues/158 we really need to fix 😅 Pls report if restarting helped or not.

[m0ngr31]

That doesn't seem to work. I am getting a TON of these every few seconds in the log though:

20231222 09:41:37 116 finished python -m refact_scratchpads_no_gpu.infserver_no_gpu longthink/stable --workers 2 --openai_key sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX @, retcode 1
20231222 09:41:38 CVD= starting python -m refact_scratchpads_no_gpu.infserver_no_gpu longthink/stable --workers 2 --openai_key sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
 -> pid 118
-- 118 -- 20231222 09:41:38 NOGPU infmod_guid: longthink/stable_c3d11bd17d34_0000
-- 118 -- 20231222 09:41:38 NOGPU infmod_guid: longthink/stable_c3d11bd17d34_0001
-- 118 -- Traceback (most recent call last):
-- 118 --   File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
-- 118 --     return _run_code(code, main_globals, None,
-- 118 --   File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
-- 118 --     exec(code, run_globals)
-- 118 --   File "/usr/local/lib/python3.10/dist-packages/refact_scratchpads_no_gpu/infserver_no_gpu.py", line 239, in <module>
-- 118 --     main()
-- 118 --   File "/usr/local/lib/python3.10/dist-packages/refact_scratchpads_no_gpu/infserver_no_gpu.py", line 232, in main
-- 118 --     asyncio.get_event_loop().run_until_complete(asyncio.gather(*[
-- 118 --   File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
-- 118 --     return future.result()
-- 118 --   File "/usr/local/lib/python3.10/dist-packages/refact_scratchpads_no_gpu/infserver_no_gpu.py", line 194, in do_the_serving
-- 118 --     retcode, calls_unfiltered = await stream_results_async.completions_wait_batch(aio_session, my_desc)
-- 118 --   File "/usr/local/lib/python3.10/dist-packages/refact_scratchpads_no_gpu/stream_results_async.py", line 35, in completions_wait_batch
-- 118 --     async with aio_session.post(url, json=my_desc, timeout=WAIT_TIMEOUT, proxies={"http": "", "https": ""}) as resp:
-- 118 --   File "/usr/local/lib/python3.10/dist-packages/aiohttp/client.py", line 976, in post
-- 118 --     self._request(hdrs.METH_POST, url, data=data, **kwargs)
-- 118 -- TypeError: ClientSession._request() got an unexpected keyword argument 'proxies'
-- 118 -- 20231222 09:41:38 NOGPU Unclosed client session
-- 118 -- client_session: <aiohttp.client.ClientSession object at 0x147422e90130>
-- 118 -- 20231222 09:41:38 NOGPU Unclosed client session
-- 118 -- client_session: <aiohttp.client.ClientSession object at 0x147422e900d0>

Could this have something to do with it?

[olegklimov]

I know this one!

Hey @m0ngr31 sorry you ran into this, it was a "thing to fix" some time ago, and now it works using a different mechanism -- we use litellm now that has integrations with a lot of model providers (still one provider for now, we'll expand from there). You just need to upgrade to the latest version, and HUGE thanks for reporting!

[m0ngr31]

Hey @olegklimov, as far as I can tell I'm on the latest extension and Docker versions and it's doing the same thing as before. I have also restarted VSCode and the container.

[m0ngr31]

Disregard my last message - the docker didn't update like I thought it did. It seems to be working now. Thanks @olegklimov!