search

automl / HPOlib1.5

GNU General Public License v3.0
69 stars 27 forks source link

Exploring OpenML surrogates: improve file locking to reduce overall lock time and file system access #79

Closed mfeurer closed 4 years ago

mfeurer commented 4 years ago

Running many of these experiments can result in a lot of time waiting for the file locks to be released. This PR implements the it's easier to ask for forgiveness than for permission pattern and iteratively tries to read a file instead of locking it first. Only if that fails it tries to lock the file. Also, constructing the surrogate is now done inside the file lock to avoid constructing the same surrogate multiple times. I'm not 100% sure if loading the file within the lock is still necessary, but it definitively won't hurt.

mfeurer commented 4 years ago

I just fixed that test, thanks for noting.