search

instructlab / instructlab

InstructLab Command-Line Interface. Use this to chat with a model and execute the InstructLab workflow to train a model using custom taxonomy data.
https://instructlab.ai
Apache License 2.0
932 stars 328 forks source link

[Epic] `--model-path`, `--model`, and all other model related flags are overloaded for `ilab model train` and `ilab model evaluate` #1871

Open cdoern opened 3 months ago

cdoern commented 3 months ago

the clickext class should be extended to figure out if a flag is either an HF path OR a local path. Based off of which type of argument is passed, there should be a log and specific behavior for the different types.

in ilab model train here are the current issues

  1. MacOS requires a HF path
  2. training library requires a path on disk
  3. linux train can take either

But ALL of them use --model-path adding some logic to the custom option class could resolve this and download the model when necessary.

in ilab model evaluate --model can be either a HF repo OR a local path.

expanding the clickext class to handle all of these will allow us to handle these scenarios in one place rather in the various cmds while still maintaining the single flag method.

Alternatively

all of these flags can be split into --model-repo and --model-path flags (one for HF/OCI and one for local paths)

Issues resolved when this issue closes:

danmcp commented 2 months ago

Note that with: https://github.com/instructlab/instructlab/pull/2065 eval will no longer support hf paths. Currently it is only supported for mmlu.

We actually could support hf downloads but the current implementation is insufficient because it would rely on secondary libraries doing the model downloads which means the downloads don't end up in our cache. To support we would need to integrate ilab download capability into eval.

khaledsulayman commented 2 months ago

2011 renames --model to --model-path

imo makes much more sense to use the same convention for local paths across local libraries, and then maybe --model-repo as you suggested for HF

ktam3 commented 2 months ago

@alinaryan @jaideepr97 - I compiled all the issues where this is mentioned in into the main description. do you want to make this the main issue/epic?