Where can I get wikisql_tok?

[oney]

It throws an error

(py36) one@test:~/sqlova$ python3 train.py --seed 1 --bS 16 --accumulate_gradients 2 --bert_type_abb uS --fine_tune --lr 0.001 --lr_bert 0.00001 --max_seq_leng 222
BERT-type: uncased_L-12_H-768_A-12
Traceback (most recent call last):
  File "train.py", line 552, in <module>
    train_data, train_table, dev_data, dev_table, train_loader, dev_loader = get_data(path_wikisql, args)
  File "train.py", line 183, in get_data
    train_data, train_table, dev_data, dev_table, _, _ = load_wikisql(path_wikisql, args.toy_model, args.toy_size, no_w2i=True, no_hs_tok=True)
  File "/home/one/sqlova/sqlova/utils/utils_wikisql.py", line 29, in load_wikisql
    train_data, train_table = load_wikisql_data(path_wikisql, mode='train', toy_model=toy_model, toy_size=toy_size, no_hs_tok=no_hs_tok, aug=aug)
  File "/home/one/sqlova/sqlova/utils/utils_wikisql.py", line 58, in load_wikisql_data
    with open(path_sql) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/one/data/wikisql_tok/train_tok.jsonl'

[whwang299]

Hi @oney

wikisql_tok is the name of the folder that I have used to save the annotated training data. To solve the issue, please download pre-annotated data from here and uncompress them to some folder. and change these lines https://github.com/naver/sqlova/blob/b7ce9ad421fd4688ef8592f93b248df85e9995ad/train.py#L545-L546 to set the path to the annotated training data.

Thanks!

Wonseok

[oney]

Got it! Thanks!

[guotong1988]

Same question. Thank you.

[yongbowin]

@whwang299 What's the data-format for train_tok.jsonl in each key-value? Could you please paste a sample in train_tok.jsonl? thx