valohai / valohai-utils

Python helper library for Valohai
MIT License
2 stars 2 forks source link

Add step name for local input/output cache to avoid filename collisions #29

Closed JuhaKiili closed 3 years ago

JuhaKiili commented 3 years ago

Local cache downloads inputs when necessary. It uses the filename as a key. The key collision happens, when users have the same input filename between different steps.

Outputs would never collide as they have the timestamp+hash as root folder, but for consistency and clarity, the step name is now used for local output paths, too.

In addition, this PR unifies for from valohai.internals import global_state style for all global state usage. This makes the code slightly more verbose, but I felt like explicitly communicating the usage of global state everywhere makes the code much more self-documenting. If the code just says input_infos, the global nature is easier to miss when compared to global_state.input_infos.