janhq / cortex.cpp

Local AI API Platform
https://cortex.so
Apache License 2.0
2.07k stars 116 forks source link

question(flaky): `cortex -v` hangs when checking if there is a newer release? #1268

Closed 0xSage closed 1 month ago

0xSage commented 1 month ago

Flakey issue. Happens 1/3 times.

Steps:

  1. cortex -v or any command really
    
    ❯ cortex-nightly -v
    v0.5.0-70

2. Doesn't return, just hangs
3. After 10 seconds, we get "A new release is available" 

What causes this? 
Where's our endpoint for checking if there's a more recent version, and is this endpoint prone to lagging?
vansangpfiev commented 1 month ago

We check the new nightly version with https://delta.jan.ai endpoint. We are using the default timeout (10 seconds). I think can set the timeout to 1 second, if connection timeouts, just not print any message on version update.

0xSage commented 1 month ago

We check the new nightly version with https://delta.jan.ai endpoint. We are using the default timeout (10 seconds). I think can set the timeout to 1 second, if connection timeouts, just not print any message on version update.

Timeout to 1 s is a good workaround!

Q: I'm just wondering whether we need to check versions at the end of every command?

Rationale

  1. Since we're hosting the endpoint ourselves, let's just assume it will be ddos'ed. Then will every cortex command, by default, takes 1 second, returning nothing? How will we build this mini-feature if we were building it for scale & to be antifragile to external dependencies?
  2. Users typically execute many CLI commands in succession, during a single session with Cortex. If they don't want to update initially, then reminding them 10-20 times thereafter, is annoying.
  3. Recall, Cortex is an on-prem, local tool. Many will use it on an intranet. So many external calls out is not our style.

Dumb Ideas

  1. Reserve the version checks for just a few key commands, that users will almost always see:
    
    cortex
    cortex -v
    cortex start                 # Server start is an impt entrypoint to using cortex, and will call attention.
    cortex run                   # Starting an interactive chat shell is another impt entrypoint to using cortex

Optionally

COMMAND -h # Not annoying to put it in all the help commands as well



I'm not sure what the best fix is, but I do think shortening timeout to 1sec is a (good) but temporary patch to this issue.

cc @dan-homebrew 
vansangpfiev commented 1 month ago

I would like to propose 2 approaches that can help to handle update check better:

cc @dan-homebrew

0xSage commented 1 month ago

Sure, open to the above.


My only request is that, UX wise, the following should inform users there is an available update:

cortex (-h)
cortex -v
vansangpfiev commented 1 month ago

Will add update info into .cortexrc

checkedForUpdateAt: time
latestRelease: version

Interval to check update:

0xSage commented 1 month ago

Will add update info into .cortexrc

checkedForUpdateAt: time
latestRelease: version

Interval to check update:

  • nightly: 10 minutes
  • beta: 1 hour
  • stable: 24 hours cc: @dan-homebrew @0xSage

Nice. Is this a cron job that we're always running on the users machine? Or only run when the next command is executed?

vansangpfiev commented 1 month ago

It only runs when the next CLI command is executed.

0xSage commented 1 month ago

Closing as resolved. QA'd adn works really well 🙏