jozu-ai / kitops

Tools for easing the handoff between AI/ML and App/SRE teams.
https://KitOps.ml
Apache License 2.0
261 stars 24 forks source link

Model layers created on Windows do not unpack correctly on Linux/MacOS #291

Closed amisevsk closed 1 month ago

amisevsk commented 1 month ago

Describe the bug Modelkits packed on Windows end up using tarballs with windows-style paths (i.e. using \ as the path separator). When unpacking such models on a non-Windows machine (where / is the path separator), directories are not detected and result in filenames instead of directories, e.g. my-model\my-file.bin as a filename)

To Reproduce

  1. Pack model on Windows where some layer refers to a directory that has other directories in it
  2. Push model to some registry
  3. Unpack model on Linux/MacOS

Version nightly

Additional context N/A

amisevsk commented 1 month ago

To work around this issue for now, avoid specifying paths that contain subdirectories. If you have the directory structure

./my-model
|-- foo/
|  |-- foo.bin
|-- bar/
|  |-- bar.bin
|-- model.bin

it should be safe to pack it using model parts:

model:
  name: my-model
  path: my-model/
  parts:
    - name: foo
      path: my-model/foo/
    - name: bar
      path: my-model/bar/