valohai / valohai-utils

Python helper library for Valohai
MIT License
2 stars 2 forks source link

valohai-utils vfs mangles paths with archived inputs #45

Closed JuhaKiili closed 3 years ago

JuhaKiili commented 3 years ago

When auto-processing an archived input (they are auto-processed by default), valohai-utils mangles the path and the name of the archived files.

Often there is information encoded into the path or filename (example: labels/UUID.json) that is now lost.

This code

import valohai

inputs = {"labels": "/home/juha/ws/yolov3-tf2/labels.zip"}
valohai.prepare(step="test", default_inputs=inputs)
for label_path in valohai.inputs('labels').paths():
    print(label_path)

With this zip contents:

labels/02494272-79e7-4e8b-9d5b-5da07a350b16.json
labels/3f7a8a63-33c0-4654-b203-d4c51b2374dd.json
labels/4247d9db-81f1-46a8-8601-4059e68032b8.json
labels/442fcac9-1623-458d-937e-50b6a7ba44ea.json
labels/7dcae7e8-748a-40da-936f-7c3fa7c70e05.json
labels/82b93121-a4d5-4c71-a7ba-7702155e247b.json
labels/cce9141c-8cb1-4e85-9b01-e01f83cf838c.json
labels/d9abdd08-ab0b-444c-b6f2-a8b6b5209132.json
labels/e1864b66-302b-40a0-88b7-39681a8c2e4b.json
labels/e1f9c295-27cd-4828-b442-1d1b1ff14047.json

Returns this:

/tmp/tmp5vtugf_l.json
/tmp/tmpiuv92505.json
/tmp/tmpdb9crwg7.json
/tmp/tmpb5biboa8.json
/tmp/tmp8ebrfq_s.json
/tmp/tmp_nsqsyy6.json
/tmp/tmp6qrqnjni.json
/tmp/tmp6yc5y0w8.json
/tmp/tmppn0xf1gv.json
/tmp/tmp0w7_bsyd.json