Closed jiangpinglei closed 5 years ago
I found it cost too much time When running the convert_single_example function.
time0 = time.time() feature = convert_single_example(ex_index, example, label_list, max_seq_length, tokenizer,mode) time1 = time.time() print("time cost:", time1-time0)
time0 = time.time()
feature = convert_single_example(ex_index, example, label_list, max_seq_length, tokenizer,mode)
time1 = time.time()
print("time cost:", time1-time0)
the cost is up to a few seconds! time cost: 4.020495414733887
time cost: 4.020495414733887
In convert_single_example function we can fix it by add the following code : if not os.path.exists('./output/label2id.pkl'):
if not os.path.exists('./output/label2id.pkl'):
in front of
with open('./output/label2id.pkl','wb') as w: pickle.dump(label_map, w)
with open('./output/label2id.pkl','wb') as w:
pickle.dump(label_map, w)
I found it cost too much time When running the convert_single_example function.
time0 = time.time()
feature = convert_single_example(ex_index, example, label_list, max_seq_length, tokenizer,mode)
time1 = time.time()
print("time cost:", time1-time0)
the cost is up to a few seconds!
time cost: 4.020495414733887
In convert_single_example function we can fix it by add the following code :
if not os.path.exists('./output/label2id.pkl'):
in front of
with open('./output/label2id.pkl','wb') as w:
pickle.dump(label_map, w)