LARS-research / KGTuner

KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning" (ACL 2022 long paper)
https://arxiv.org/pdf/2205.02460.pdf
19 stars 1 forks source link

对fcntl库的使用以及参数max_trials的疑问 #1

Closed hezihao-hfut closed 1 year ago

hezihao-hfut commented 1 year ago

运行环境


运行命令

python run.py -search -earlyStop -space reduced -dataset sampled_wn18rr_0.2_starts_10 -cpu 2 -valid_steps 2500 -max_steps 20000 -model ComplEx -eval_test -test_batch_size 16 -gpu 0

后,进程阻塞,经排查,在code/utils.py line 22处

fcntl.flock(f.fileno(), fcntl.LOCK_EX)

发生Exception,内容如下

module 'fcntl' has no attribute 'LOCK_EX'

原因是我安装的fcntl库内未定义LOCK_EX常量。 请问该处代码是否是必要的,以及其作用是什么?若是必要的,则如何建议修改?


另外,在code/run.py line 46内参数&arg.max_trials=200&对结果有何影响

AndrewZhou924 commented 1 year ago

Thanks for your interest in our work.

The fcntl library used here is for thread/process safety, especially for conducting the search with multiple threads or processes. If you just use it in a single process, you can replace the two functions as follows, where the fcntl library has been removed.

def savePickleReader(file):
    if os.path.exists(file):
        data = pkl.load(open(file, "rb"))
        return data
    else:
        return None

def savePickleWriter(data, file):
    pkl.dump(data, open(file, "wb"))
    return

Besides, the parameter max_trials is the maximum number of searched hyper-parameters, which is related to the function of HPO_instance.runTrials(). To be specific, max_trials=200 means that 200 HP will be sampled from the search space and then run on the sampled/original KG.