m3dev / gokart

Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.
https://gokart.readthedocs.io/en/latest/
MIT License
305 stars 57 forks source link

Tasks hash ids update when run function definition changes #252

Closed ujiuji1259 closed 2 years ago

ujiuji1259 commented 2 years ago

I’ve added get_own_code method so that unique id can be updated when the task definition changes (requested in #216 ).

In the following example (in examples/gokart_notebook_example.ipynb), get_own_code got the codes of run, requires. By adding this into dependencies of hash_id, the task rerun without manually changing __version when the script is changed. スクリーンショット 2021-10-13 8 30 46

This PR is related to #205 from @hiro-o918

hirosassa commented 2 years ago

@ujiuji1259 Thank you for the GREAT contribution! Please add unit test for the get_code and make_unique_id to confirm this feature runs correctly.

vaaaaanquish commented 2 years ago

Thx. plz update document :)

mski-iksm commented 2 years ago

Thank you for great contribution!

I think the description should be something like,

If this is true, even if all outputs are present, this task will be executed if any changes have been made to the code.

instead of

If this is true, this task will not run only if all input and output files exist, and this task class is modified.

ujiuji1259 commented 2 years ago

I've added tests for get_own_code and output path (≒ unique id), and updated description!

ujiuji1259 commented 2 years ago

I've updated the document in docs/task_on_kart.rst, and it passed unittest in the local environment(py3.9). Could you approve the workflow?

Hi-king commented 2 years ago

@ujiuji1259 Thx for great feature! We'll release new version including this feature :)

Hi-king commented 2 years ago

And thx again to @hiro-o918 for inspirational feature! #205

hiro-o918 commented 2 years ago

Thank you for awesome new features! I’m looking forward to using it!☺️