search

janhq / cortex.cpp

Local AI API Platform
https://cortex.so
Apache License 2.0
2.16k stars 130 forks source link

epic: Separate CLI and API processes #1415

Closed gabrielle-ong closed 1 month ago

gabrielle-ong commented 2 months ago

Goal

Subtasks

Discussion:

https://github.com/janhq/cortex.cpp/discussions/1386

Success Criteria (windows, max, linux)

hiento09 commented 1 month ago

Hey guys @dan-homebrew @vansangpfiev @gabrielle-ong @namchuai @nguyenhoangthuan99 @0xSage , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows:

Given that:

In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine.

Solution:

Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands:

What do you guys think of this idea? Please leave your comments. Thanks!

vansangpfiev commented 1 month ago

Hey guys @dan-homebrew @vansangpfiev @gabrielle-ong @namchuai @nguyenhoangthuan99 @0xSage , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows:

Given that:

In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine.

Solution:

Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands:

  • macos: SKIP_POSTINSTALL=true && sudo installer -pkg cortex-1.0.0-rc1-mac-universal-network-installer.pkg -target /
  • Linux: echo -e "n\n" | SKIP_POSTINSTALL=true && sudo apt install -y --allow-downgrades ./cortex-1.0.0-165-linux-amd64-network-installer.deb
  • Windows: Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/SkipPostInstall /VERYSILENT /SUPPRESSMSGBOXES /NORESTART" -Wait

In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously.

What do you guys think of this idea? Please leave your comments. Thanks!

I totally agree with the above solution. That way, we won't need to change much of the updater code in the long-term. While calling an installer from cortex can be tricky, it is doable.

0xSage commented 1 month ago

Largely sounds good. From the user's perspective, what is the experience like?

On Tue, Oct 15, 2024 at 7:35 AM vansangpfiev @.***> wrote:

Hey guys @dan-homebrew https://github.com/dan-homebrew @vansangpfiev https://github.com/vansangpfiev @gabrielle-ong https://github.com/gabrielle-ong @namchuai https://github.com/namchuai @nguyenhoangthuan99 https://github.com/nguyenhoangthuan99 @0xSage https://github.com/0xSage , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows: Given that:

In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine. Solution:

Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands:

  • macos: SKIP_POSTINSTALL=true && sudo installer -pkg cortex-1.0.0-rc1-mac-universal-network-installer.pkg -target /
  • Linux: echo -e "n\n" | SKIP_POSTINSTALL=true && sudo apt install -y --allow-downgrades ./cortex-1.0.0-165-linux-amd64-network-installer.deb
  • Windows: Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/SkipPostInstall /VERYSILENT /SUPPRESSMSGBOXES /NORESTART" -Wait

In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously.

What do you guys think of this idea? Please leave your comments. Thanks!

I totally agree with the above solution. That way, we won't need to change much of the updater code in the long-term. While calling an installer from cortex can be tricky, it is doable.

— Reply to this email directly, view it on GitHub https://github.com/janhq/cortex.cpp/issues/1415#issuecomment-2412521313, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQVWFCCBRX42LWHW2W7JDYDZ3RIL7AVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGUZDCMZRGM . You are receiving this because you were mentioned.Message ID: @.***>

hiento09 commented 1 month ago

Largely sounds good. From the user's perspective, what is the experience like? - Do new windows/views pop up (if so, then no go) - Is there an increased update wait time? … On Tue, Oct 15, 2024 at 7:35 AM vansangpfiev @.> wrote: Hey guys @dan-homebrew https://github.com/dan-homebrew @vansangpfiev https://github.com/vansangpfiev @gabrielle-ong https://github.com/gabrielle-ong @namchuai https://github.com/namchuai @nguyenhoangthuan99 https://github.com/nguyenhoangthuan99 @0xSage https://github.com/0xSage , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows: Given that: In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine. Solution: Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands: - macos: SKIP_POSTINSTALL=true && sudo installer -pkg cortex-1.0.0-rc1-mac-universal-network-installer.pkg -target / - Linux: echo -e "n\n" | SKIP_POSTINSTALL=true && sudo apt install -y --allow-downgrades ./cortex-1.0.0-165-linux-amd64-network-installer.deb - Windows: Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/SkipPostInstall /VERYSILENT /SUPPRESSMSGBOXES /NORESTART" -Wait In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously. What do you guys think of this idea? Please leave your comments. Thanks! I totally agree with the above solution. That way, we won't need to change much of the updater code in the long-term. While calling an installer from cortex can be tricky, it is doable. — Reply to this email directly, view it on GitHub <#1415 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQVWFCCBRX42LWHW2W7JDYDZ3RIL7AVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGUZDCMZRGM . You are receiving this because you were mentioned.Message ID: @.>

@0xSage

0xSage commented 1 month ago

👍👍

On Tue, Oct 15, 2024 at 9:53 AM hiento09 @.***> wrote:

Largely sounds good. From the user's perspective, what is the experience like? - Do new windows/views pop up (if so, then no go) - Is there an increased update wait time? … <#m2046471413798012038> On Tue, Oct 15, 2024 at 7:35 AM vansangpfiev @.> wrote: Hey guys @dan-homebrew https://github.com/dan-homebrew https://github.com/dan-homebrew https://github.com/dan-homebrew @vansangpfiev https://github.com/vansangpfiev https://github.com/vansangpfiev https://github.com/vansangpfiev @gabrielle-ong https://github.com/gabrielle-ong https://github.com/gabrielle-ong https://github.com/gabrielle-ong @namchuai https://github.com/namchuai https://github.com/namchuai https://github.com/namchuai @nguyenhoangthuan99 https://github.com/nguyenhoangthuan99 https://github.com/nguyenhoangthuan99 https://github.com/nguyenhoangthuan99 @0xSage https://github.com/0xSage https://github.com/0xSage https://github.com/0xSage , I’d like to present an idea for the installer and updater when we separate the CLI and server into two distinct binaries, as follows: Given that: In the current updater, the user runs the cortex update command, which causes the cortex process to download the new binary version and replace the existing binary. However, other components of the installer, such as the uninstaller and post-uninstall script, are not updated because we only replace the binary file, not the entire app version on the user's machine. Solution: Instead of the current cortex update only downloading and replacing the binary file, the cortex update command will pull the installer file and install it in headless mode using the following commands: - macos: SKIP_POSTINSTALL=true && sudo installer -pkg cortex-1.0.0-rc1-mac-universal-network-installer.pkg -target / - Linux: echo -e "n\n" | SKIP_POSTINSTALL=true && sudo apt install -y --allow-downgrades ./cortex-1.0.0-165-linux-amd64-network-installer.deb - Windows: Start-Process -FilePath ".\cortex-0.5.0-154-windows-amd64-network-installer.exe" -ArgumentList "/SkipPostInstall /VERYSILENT /SUPPRESSMSGBOXES /NORESTART" -Wait In these commands, we skip the postinstall process, which installs the cortex.llamacpp engine. This allows the installer to replace only the app, installer, uninstaller, and the new binary file. With this approach, the Cortex code no longer needs to handle the complexity of replacing two binaries; it can simply run the commands above to update both binaries and the scripts in the installer and uninstaller simultaneously. What do you guys think of this idea? Please leave your comments. Thanks! I totally agree with the above solution. That way, we won't need to change much of the updater code in the long-term. While calling an installer from cortex can be tricky, it is doable. — Reply to this email directly, view it on GitHub <#1415 (comment) https://github.com/janhq/cortex.cpp/issues/1415#issuecomment-2412521313>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQVWFCCBRX42LWHW2W7JDYDZ3RIL7AVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGUZDCMZRGM https://github.com/notifications/unsubscribe-auth/AQVWFCCBRX42LWHW2W7JDYDZ3RIL7AVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGUZDCMZRGM . You are receiving this because you were mentioned.Message ID: @.>

@0xSage https://github.com/0xSage

  • Do new windows/views pop up (if so, then no go)? => No, all the commands above run the installer in headless mode. I tested them, and they work as expected.
  • Is there an increased update wait time? => We use a network installation but disable the post-install script that pulls cortex.llamacpp from the internet. Instead, it only pulls the new binaries, which are less than 30MB. I tested this, and the update process is smooth and quick.

— Reply to this email directly, view it on GitHub https://github.com/janhq/cortex.cpp/issues/1415#issuecomment-2412652929, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQVWFCCKSGV563F4TFYVOKDZ3RYRDAVCNFSM6AAAAABPJ5KC3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGY2TEOJSHE . You are receiving this because you were mentioned.Message ID: @.***>

hiento09 commented 1 month ago

Hi @dan-homebrew @0xSage , we need to finalize the binary file name of the cortex server by tomorrow so that @vansangpfiev and I can proceed with splitting the binary and updating the installer. Could you help us finalize it? Thanks!

My proposal:

vansangpfiev commented 1 month ago

We also need to handle the case that user using v1.0.0 update cortex via cortex update command. I would like to propose an approach:

cc: @dan-homebrew @gabrielle-ong @hiento09

gabrielle-ong commented 1 month ago

Success Criteria (windows, max, linux)

gabrielle-ong commented 1 month ago

@hiento09 and @vansangpfiev Can I check if anything missing from this success criteria for the Separate CLI & API epic?

Success Criteria (windows, max, linux)

Installer: Installer installs 2 different binaries Uninstaller: Uninstalls 2 different binaries Cortex update replaces the app, installer, uninstaller and binary file (without installing cortex.llamacpp) all CLI commands will start API server, if not running Edge case: User run cortex update on v1.0.0 → prompted to install cortex-server as with 1.0.1

vansangpfiev commented 1 month ago

@gabrielle-ong 

all CLI commands will start API server, if not running

cortex pull and cortex engines install do not need to start API server for now Others Success Criteria are correct. @hiento09 Please help to double check.

hiento09 commented 1 month ago

@gabrielle-ong The installer and uninstaller meet the success criteria. Sang and I worked on the same PR here https://github.com/janhq/cortex.cpp/pull/1499 . You can test it from the nightly version 183 where our PR was merged. image

gabrielle-ong commented 1 month ago

Thanks @hiento09 and @vansangpfiev, marking as complete Tracking the followup task #1523 (cortex pull and cortex engines to use API server)

Successfully installed mac, windows, linux Sucessfully uninstalled cortex-nightly and cortex-server-nightly Edge case successfully encountered last week, rancortex update --server`