search

substratusai / runbooks

Finetune LLMs on K8s by using Runbooks
https://www.substratus.ai
Other
168 stars 14 forks source link

Standalone CLI: sub #232

Closed nstogner closed 1 year ago

nstogner commented 1 year ago

Add a standalone CLI: sub

Replaces the current kubectl plugins.

samos123 commented 1 year ago

Overall the CLI looks promising! Can't wait to try it out for my dev flow.

I would prefer to keep this PR only to what's needed to make the Standalone CLI work. Any refactors should be done as separate PRs and discussed separately. Things like reformatting the gcp install scripts or renaming baseModel to model are big changes that don't seem to be required for the standalone CLI.

samos123 commented 1 year ago

Edit: Ignore this is already fixed in commit 47c4a76

I noticed the following log:

2023/09/09 21:34:03 Pod logs for: %v: %v test-gguf-model-notebook-bld-n85wk time="2023-09-10T04:34:03Z" level=info msg="Retrieving image nvidia/cuda:12.2.0-devel-ubuntu22.04 from registry index.docker.io"
samos123 commented 1 year ago

Not for this PR, but something to discuss as a follow up. One thing that was rather annoying was being unable to edit params.json from within the notebook. It's mounted as read-only. I noticed I had a typo in my param.json and my only way to solve it was to delete the notebook and modify the substratus resource.

samos123 commented 1 year ago

I can't seem to get out once I hit an error, Ctrl + C and Q are both not working: image

samos123 commented 1 year ago

file syncing currently only happens for files in src directory. I placed my notebook in the main directory. I think the file sync should happen for any .ipynb file that's also in the local directory.

nstogner commented 1 year ago

Tracking the param.json issue here: https://github.com/substratusai/substratus/issues/237