search

jerbarnes / semeval22_structured_sentiment

SemEval-2022 Shared Task 10: Structured Sentiment Analysis
75 stars 42 forks source link

Error when getting the baselines with graph_parser #7

Closed MinionAttack closed 3 years ago

MinionAttack commented 3 years ago

Hi, I'm trying to run the get_baseline.sh inside baselines/graph_parser but I get this error during the execution of the script:

##################################################
Predicting data from sentiment_graphs/opener_en/head_final/train.conllu
Loading data from sentiment_graphs/opener_en/head_final/train.conllu
......
Train F1 on epoch 100 is 94.96%
Have not seen any improvement for 3 epochs
Best F1 was 95.16% seen at epoch #97
Loading External Vectors
8
Restoring model from experiments/opener_en/head_final/best_model.save
Traceback (most recent call last):
  File "./src/main.py", line 399, in <module>
    run_parser(get_args())
  File "./src/main.py", line 394, in run_parser
    predict(model, args, args.val, args.elmo_dev, vocabs)
  File "./src/main.py", line 300, in predict
    pred_path = settings.dir + to_predict.split("/")[-1] + ".pred"
AttributeError: 'NoneType' object has no attribute 'split'

Regards.

jerbarnes commented 3 years ago

Thanks for finding this one! The error came from trying to predict during training when we don't provide a prediction file. I've corrected this and checked that it works. If you confirm, I'll close this issue.

MinionAttack commented 3 years ago

Yes, now it works. Thanks!

MinionAttack commented 3 years ago

Looking at the output in the terminal I've seen an error:

2021-09-16 18:38:05 INFO: Loading these models for language: en (English):
=========================
| Processor | Package   |
-------------------------
| tokenize  | combined  |
| pos       | combined  |
| lemma     | combined  |
| depparse  | combined  |
| sentiment | sstplus   |
| ner       | ontonotes |
=========================

2021-09-16 18:38:05 INFO: Use device: gpu
2021-09-16 18:38:05 INFO: Loading: tokenize
2021-09-16 18:38:05 INFO: Loading: pos
2021-09-16 18:38:08 INFO: Loading: lemma
2021-09-16 18:38:09 INFO: Loading: depparse
2021-09-16 18:38:09 INFO: Loading: sentiment
2021-09-16 18:38:10 INFO: Loading: ner
2021-09-16 18:38:11 INFO: Done loading processors!
Dataset: ../../data/darmstadt_unis
Setup: head_final
2021-09-16 18:40:37 INFO: Loading these models for language: en (English):
=========================
| Processor | Package   |
-------------------------
| tokenize  | combined  |
| pos       | combined  |
| lemma     | combined  |
| depparse  | combined  |
| sentiment | sstplus   |
| ner       | ontonotes |
=========================

2021-09-16 18:40:37 INFO: Use device: gpu
2021-09-16 18:40:37 INFO: Loading: tokenize
2021-09-16 18:40:37 INFO: Loading: pos
2021-09-16 18:40:40 INFO: Loading: lemma
2021-09-16 18:40:40 INFO: Loading: depparse
2021-09-16 18:40:41 INFO: Loading: sentiment
2021-09-16 18:40:41 INFO: Loading: ner
2021-09-16 18:40:42 INFO: Done loading processors!
Dataset: ../../data/mpqa
Setup: head_final
Traceback (most recent call last):
  File "convert_to_conllu.py", line 135, in <module>
    tag_json_files(train)
  File "convert_to_conllu.py", line 11, in stanza_tag_json_files
    tagged_sent = nlp(sentence["text"])
  File "/home/iago/.local/lib/python3.6/site-packages/stanza/pipeline/core.py", line 229, in __call__
    doc = self.process(doc)
  File "/home/iago/.local/lib/python3.6/site-packages/stanza/pipeline/core.py", line 223, in process
    doc = process(doc)
  File "/home/iago/.local/lib/python3.6/site-packages/stanza/pipeline/sentiment_processor.py", line 53, in process
    labels = cnn_classifier.label_text(self._model, text, batch_size=self._batch_size)
  File "/home/iago/.local/lib/python3.6/site-packages/stanza/models/classifiers/cnn_classifier.py", line 462, in label_text
    labels = unsort(labels, orig_idx)
  File "/home/iago/.local/lib/python3.6/site-packages/stanza/models/common/utils.py", line 204, in unsort
    assert len(sorted_list) == len(oidx), "Number of list elements must match with original indices."
AssertionError: Number of list elements must match with original indices.

I'm using stanza 1.2.3.

Inside the experiments folder, for the norec dataset there are no files saved.

Also, the logs folder although it has folders for the different datasets, each folder it's empty.

jerbarnes commented 3 years ago

Super weird. It seems to be a problem with stanza 1.2.3 actually. I know that there is an issue with the sentiment module eating some tokens, so I just changed the code to avoid loading the sentiment and NER modules, which aren't needed anyway. Should work now with 1.2.3 also.

Regarding the experiments folder, do you get any error messages about running the graph parser on norec?

I just checked my logs folder and also don't have anything, so I'll see if I can fix that and update you.

MinionAttack commented 3 years ago

Searching for the nore, I have found more errors:

###########################
EXTERNAL EMBEDDINGS
###########################
using external embeddings: embeddings/18.zip

###########################
INPUT FILES
###########################
train data: sentiment_graphs/mpqa/head_final/train.conllu

###########################
OUTPUT DIR
###########################
saving experiment to experiments/mpqa/head_final/
/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser
Argument elmo_train not set
Restart and set argument elmo_train; I will try to continue for now...
Argument elmo_dev not set
Restart and set argument elmo_dev; I will try to continue for now...
Argument elmo_test not set
Restart and set argument elmo_test; I will try to continue for now...
Argument recycle not set
Restart and set argument recycle; I will try to continue for now...
Argument recycle_layers not set
Restart and set argument recycle_layers; I will try to continue for now...
Argument freeze not set
Restart and set argument freeze; I will try to continue for now...
Argument vocab not set
Restart and set argument vocab; I will try to continue for now...
Argument val not set
Restart and set argument val; I will try to continue for now...
Argument other_target_style not set
Restart and set argument other_target_style; I will try to continue for now...
Argument help_style not set
Restart and set argument help_style; I will try to continue for now...
Argument load not set
Restart and set argument load; I will try to continue for now...
Argument predict_file not set
Restart and set argument predict_file; I will try to continue for now...
cuda:0
(7, 5)
Traceback (most recent call last):
  File "./src/main.py", line 400, in <module>
    run_parser(get_args())
  File "./src/main.py", line 362, in run_parser
    _vocabs = make_vocabs(sentences, 0)
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/vocab.py", line 62, in make_vocabs
    for sen in sentences:
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/col_data.py", line 198, in read_col_data
    with open(fname) as fhandle:
FileNotFoundError: [Errno 2] No such file or directory: 'sentiment_graphs/mpqa/head_final/train.conllu'
Running multibooked_ca - head_final
mkdir: directory could not be created «experiments/multibooked_ca/head_final»: File already exists
###########################
EXTERNAL EMBEDDINGS
###########################
using external embeddings: embeddings/34.zip

###########################
INPUT FILES
###########################
train data: sentiment_graphs/multibooked_ca/head_final/train.conllu

###########################
OUTPUT DIR
###########################
saving experiment to experiments/multibooked_ca/head_final/
/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser
Argument elmo_train not set
Restart and set argument elmo_train; I will try to continue for now...
Argument elmo_dev not set
Restart and set argument elmo_dev; I will try to continue for now...
Argument elmo_test not set
Restart and set argument elmo_test; I will try to continue for now...
Argument recycle not set
Restart and set argument recycle; I will try to continue for now...
Argument recycle_layers not set
Restart and set argument recycle_layers; I will try to continue for now...
Argument freeze not set
Restart and set argument freeze; I will try to continue for now...
Argument vocab not set
Restart and set argument vocab; I will try to continue for now...
Argument val not set
Restart and set argument val; I will try to continue for now...
Argument other_target_style not set
Restart and set argument other_target_style; I will try to continue for now...
Argument help_style not set
Restart and set argument help_style; I will try to continue for now...
Argument load not set
Restart and set argument load; I will try to continue for now...
Argument predict_file not set
Restart and set argument predict_file; I will try to continue for now...
cuda:0
(7, 5)
Traceback (most recent call last):
  File "./src/main.py", line 400, in <module>
    run_parser(get_args())
  File "./src/main.py", line 362, in run_parser
    _vocabs = make_vocabs(sentences, 0)
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/vocab.py", line 62, in make_vocabs
    for sen in sentences:
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/col_data.py", line 198, in read_col_data
    with open(fname) as fhandle:
FileNotFoundError: [Errno 2] No such file or directory: 'sentiment_graphs/multibooked_ca/head_final/train.conllu'
Running multibooked_eu - head_final
mkdir: directory could not be created «experiments/multibooked_eu/head_final»: File already exists
###########################
EXTERNAL EMBEDDINGS
###########################
using external embeddings: embeddings/32.zip

###########################
INPUT FILES
###########################
train data: sentiment_graphs/multibooked_eu/head_final/train.conllu

###########################
OUTPUT DIR
###########################
saving experiment to experiments/multibooked_eu/head_final/
/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser
Argument elmo_train not set
Restart and set argument elmo_train; I will try to continue for now...
Argument elmo_dev not set
Restart and set argument elmo_dev; I will try to continue for now...
Argument elmo_test not set
Restart and set argument elmo_test; I will try to continue for now...
Argument recycle not set
Restart and set argument recycle; I will try to continue for now...
Argument recycle_layers not set
Restart and set argument recycle_layers; I will try to continue for now...
Argument freeze not set
Restart and set argument freeze; I will try to continue for now...
Argument vocab not set
Restart and set argument vocab; I will try to continue for now...
Argument val not set
Restart and set argument val; I will try to continue for now...
Argument other_target_style not set
Restart and set argument other_target_style; I will try to continue for now...
Argument help_style not set
Restart and set argument help_style; I will try to continue for now...
Argument load not set
Restart and set argument load; I will try to continue for now...
Argument predict_file not set
Restart and set argument predict_file; I will try to continue for now...
cuda:0
(7, 5)
Traceback (most recent call last):
  File "./src/main.py", line 400, in <module>
    run_parser(get_args())
  File "./src/main.py", line 362, in run_parser
    _vocabs = make_vocabs(sentences, 0)
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/vocab.py", line 62, in make_vocabs
    for sen in sentences:
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/col_data.py", line 198, in read_col_data
    with open(fname) as fhandle:
FileNotFoundError: [Errno 2] No such file or directory: 'sentiment_graphs/multibooked_eu/head_final/train.conllu'
Running norec - head_final
mkdir: directory could not be created «experiments/norec/head_final»: File already exists
###########################
EXTERNAL EMBEDDINGS
###########################
using external embeddings: embeddings/58.zip

###########################
INPUT FILES
###########################
train data: sentiment_graphs/norec/head_final/train.conllu

###########################
OUTPUT DIR
###########################
saving experiment to experiments/norec/head_final/
/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser
Argument elmo_train not set
Restart and set argument elmo_train; I will try to continue for now...
Argument elmo_dev not set
Restart and set argument elmo_dev; I will try to continue for now...
Argument elmo_test not set
Restart and set argument elmo_test; I will try to continue for now...
Argument recycle not set
Restart and set argument recycle; I will try to continue for now...
Argument recycle_layers not set
Restart and set argument recycle_layers; I will try to continue for now...
Argument freeze not set
Restart and set argument freeze; I will try to continue for now...
Argument vocab not set
Restart and set argument vocab; I will try to continue for now...
Argument val not set
Restart and set argument val; I will try to continue for now...
Argument other_target_style not set
Restart and set argument other_target_style; I will try to continue for now...
Argument help_style not set
Restart and set argument help_style; I will try to continue for now...
Argument load not set
Restart and set argument load; I will try to continue for now...
Argument predict_file not set
Restart and set argument predict_file; I will try to continue for now...
cuda:0
(7, 5)
Traceback (most recent call last):
  File "./src/main.py", line 400, in <module>
    run_parser(get_args())
  File "./src/main.py", line 362, in run_parser
    _vocabs = make_vocabs(sentences, 0)
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/vocab.py", line 62, in make_vocabs
    for sen in sentences:
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/col_data.py", line 198, in read_col_data
    with open(fname) as fhandle:
FileNotFoundError: [Errno 2] No such file or directory: 'sentiment_graphs/norec/head_final/train.conllu'
Running opener_es - head_final
mkdir: directory could not be created «experiments/opener_es/head_final»: File already exists
###########################
EXTERNAL EMBEDDINGS
###########################
using external embeddings: embeddings/68.zip

###########################
INPUT FILES
###########################
train data: sentiment_graphs/opener_es/head_final/train.conllu

###########################
OUTPUT DIR
###########################
saving experiment to experiments/opener_es/head_final/
/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser
Argument elmo_train not set
Restart and set argument elmo_train; I will try to continue for now...
Argument elmo_dev not set
Restart and set argument elmo_dev; I will try to continue for now...
Argument elmo_test not set
Restart and set argument elmo_test; I will try to continue for now...
Argument recycle not set
Restart and set argument recycle; I will try to continue for now...
Argument recycle_layers not set
Restart and set argument recycle_layers; I will try to continue for now...
Argument freeze not set
Restart and set argument freeze; I will try to continue for now...
Argument vocab not set
Restart and set argument vocab; I will try to continue for now...
Argument val not set
Restart and set argument val; I will try to continue for now...
Argument other_target_style not set
Restart and set argument other_target_style; I will try to continue for now...
Argument help_style not set
Restart and set argument help_style; I will try to continue for now...
Argument load not set
Restart and set argument load; I will try to continue for now...
Argument predict_file not set
Restart and set argument predict_file; I will try to continue for now...
cuda:0
(7, 5)
Traceback (most recent call last):
  File "./src/main.py", line 400, in <module>
    run_parser(get_args())
  File "./src/main.py", line 362, in run_parser
    _vocabs = make_vocabs(sentences, 0)
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/vocab.py", line 62, in make_vocabs
    for sen in sentences:
  File "/home/iago/Escritorio/SemEval-2022_Shared_Task_10/baselines/graph_parser/src/col_data.py", line 198, in read_col_data
    with open(fname) as fhandle:
FileNotFoundError: [Errno 2] No such file or directory: 'sentiment_graphs/opener_es/head_final/train.conllu'
Running opener_en - head_final
mkdir: directory could not be created «experiments/opener_en/head_final»: File already exists
jerbarnes commented 3 years ago

I assume this arises from the first problem. There doesn't seem to be any converted train.conllu files for any of the datasets in the sentiment_graphs folders giving you a bunch of FileNotFoundErrors. The first step is to make sure that these files are actually created with convert_to_conllu.py

jerbarnes commented 3 years ago

I've also fixed the logging function, so the log files should appear now.

MinionAttack commented 3 years ago

Yes, the logs are saved now and the norec has the files inside the folder. Thank you.