Error al ejecutar transformer_predictor.train(dataFile="setTrain.tsv")

[iarroyof]

transformer_predictor.train(dataFile="setTrain.tsv")

TypeError Traceback (most recent call last) Cell In[4], line 1 ----> 1 transformer_predictor.train(dataFile="setTrain.tsv")

File ~/anaconda3/envs/transferl/lib/python3.9/site-packages/historical_sources/transformer_predictor.py:87, in train(dataFile) 81 train_pairs = list( 82 map(functools.partial( 83 prepare_data), train_text)) 84 test_pairs= list( 85 map(functools.partial( 86 prepare_data), test_text)) ---> 87 train_in_texts = [pair[0] for pair in train_pairs] 88 train_out_texts = [pair[1] for pair in train_pairs] 90 input_vectorizer = layers.experimental.preprocessing.TextVectorization( 91 output_mode="int", max_tokens=max_features, 92 # ragged=False, # only for TF v2.7 93 output_sequence_length=sequence_length, 94 standardize=custom_standardization)

File ~/anaconda3/envs/transferl/lib/python3.9/site-packages/historical_sources/transformer_predictor.py:87, in (.0) 81 train_pairs = list( 82 map(functools.partial( 83 prepare_data), train_text)) 84 test_pairs= list( 85 map(functools.partial( 86 prepare_data), test_text)) ---> 87 train_in_texts = [pair[0] for pair in train_pairs] 88 train_out_texts = [pair[1] for pair in train_pairs] 90 input_vectorizer = layers.experimental.preprocessing.TextVectorization( 91 output_mode="int", max_tokens=max_features, 92 # ragged=False, # only for TF v2.7 93 output_sequence_length=sequence_length, 94 standardize=custom_standardization)

TypeError: 'NoneType' object is not subscriptable

In [5]: !head setTrain.tsv huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks... To disable this warning, you can either:

• Avoid using tokenizers before the fork if possible
• Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false) id sentence subject predicate object 1.5849625007211563 driving Has Subevent the radiator overheats driving Has Subevent the radiator overheats 1.5849625007211563 taking an exam Has Prerequisite knowledge or luck taking an exam Has Prerequisite knowledge or luck 1.5849625007211563 climb mountain Has First Subevent plan route down climb mountain Has First Subevent plan route down 2.807354922057604 snare drum Used For make music snare drum Used For make music 31804.0 That mix of optimism and pain showed up in two dozen interviews with people in all walks of life across the country . '' QUANT_S_1 walks across the country 1.5849625007211563 some things Receives Action collected to be reclcled some things Receives Action collected to be reclcled 2.0 take care of proposal Causes get work do take care of proposal Causes get work do 1.5849625007211563 politician Capable Of cut fat out politician Capable Of cut fat out 2.0 pi Is A 3.1415926535897932384626433832795028841971693993751 pi Is A 3.1415926535897932384626433832795028841971693993751

[VilmaGlez]

¿Que common sense data utilizó para crear el setTrain.tsv?

Con cn_kb.csv y cn_kb_s.csv no me genera ningún error.

[iarroyof]

Hola, usé lo que dice el README:

set_train_modules.create_training_set(default=True,input_path="path/to/input/files/", common_sense_data=None)