Feature/efficient task lock

m3dev / gokart

Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.

https://gokart.readthedocs.io/en/latest/

MIT License

318 stars 57 forks source link

Feature/efficient task lock #162

Closed mski-iksm closed 3 years ago

mski-iksm commented 3 years ago

I've added feature of efficient task cache lock.

Current task lock will prevent cache collision. However, it only checks the collision when the task tries to access the cache file (i.e. task.dump(), task.load() and task.remove()). This will allow applications to run run() of same task at the same time, which is not efficient.

This new feature will prevent running run() at the same time for efficiency.

Please review. @Hi-king @hirosassa @vaaaaanquish

Hi-king commented 3 years ago

(reopened to run Github Actions)

hirosassa commented 3 years ago

It worked!

Hi-king commented 3 years ago

LGTM!

Hi-king commented 3 years ago

Merged! Thanks for your great contribution @mski-iksm