Closed ivyleavedtoadflax closed 4 years ago
ivyleavedtoadflax
commented
4 years ago Seems to be occurring in https://github.com/wellcometrust/deep_reference_parser/blob/f392f9fef9025ec2bbb9317faeaf1c4f76abadc4/deep_reference_parser/deep_reference_parser.py#L972, but i suspect that something has gone awry elsewhere.
lizgzil
commented
4 years ago interesting my error has a slightly different number ValueError: Dimension 0 in both shapes must be equal, but are 347 and 385. Shapes are [347,100] and [385,100]. for 'Assign' (op: 'Assign') with input shapes: [347,100], [385,100].
lizgzil
commented
4 years ago (This is more for my records than anything else) I ran this for debugging so it didn't take ages loading with the model artefacts:
import os
from keras_contrib.utils import save_load_utils
from deep_reference_parser.common import MULTITASK_CFG
from deep_reference_parser.model_utils import get_config
from deep_reference_parser.reference_utils import break_into_chunks
from deep_reference_parser.deep_reference_parser import DeepReferenceParser
import en_core_web_sm
text = 'Upson MA (2019). This is a reference. In a journal. 16(1) 1-23'
config_file=MULTITASK_CFG
cfg = get_config(config_file)
MAX_WORDS = int(cfg["data"]["line_limit"])
OUTPUT = cfg["build"]["output"]
PRETRAINED_EMBEDDING = cfg["build"]["pretrained_embedding"]
DROPOUT = float(cfg["build"]["dropout"])
LSTM_HIDDEN = int(cfg["build"]["lstm_hidden"])
WORD_EMBEDDING_SIZE = int(cfg["build"]["word_embedding_size"])
CHAR_EMBEDDING_SIZE = int(cfg["build"]["char_embedding_size"])
WORD_EMBEDDINGS = cfg["build"]["word_embeddings"]
OUTPUT_PATH = cfg["build"]["output_path"]
nlp = en_core_web_sm.load()
doc = nlp(text)
chunks = break_into_chunks(doc, max_words=MAX_WORDS)
tokens = [[token.text for token in chunk] for chunk in chunks]
drp = DeepReferenceParser(output_path=OUTPUT_PATH)
drp.load_data(OUTPUT_PATH)
drp.build_model(
output=OUTPUT,
word_embeddings=WORD_EMBEDDINGS,
pretrained_embedding=PRETRAINED_EMBEDDING,
dropout=DROPOUT,
lstm_hidden=LSTM_HIDDEN,
word_embedding_size=WORD_EMBEDDING_SIZE,
char_embedding_size=CHAR_EMBEDDING_SIZE,
)
drp.predict(tokens, load_weights=True)
## Fails here, but let's look into into predict:
weights_path = os.path.join(OUTPUT_PATH, "weights.h5")
save_load_utils.load_all_weights(
drp.model, weights_path, include_optimizer=False)
## Same error hereCould it be to do with jgcbrouns's answer here "Yea so this is a problem with the classes file, model file and/or anchor file not matching. Make sure that the same classes.txt file (the file where per new line your classes are defined) matches during training and during inference (test). In my case I used 2 different classes.txt file. One file had 4 categories and the other one had only 1 class."
ivyleavedtoadflax
commented
4 years ago Yes I suppose it is possible, and I was having some issues with an empty class creeping in, if you recall? Not sure where it would have occurred in the current logic though...
ivyleavedtoadflax
commented
4 years ago note that this only occurs in the multitask scenario, so it's got to be something specific about it...
lizgzil
commented
4 years ago Is this what you expected? (i.e. the last length of 886443).
>>> POLICY_TRAIN
'data/multitask/2020.3.18_multitask_train.tsv'
>>> train_data = load_tsv(POLICY_TRAIN)
>>> [len(l) for l in train_data[0]] # same for [len(l) for l in train_data[1]][150, 81, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 88, 150, 150, 121, 150, 150, 150, 150, 150, 150, 150, 58, 150, 150, 150, 150, 150, 108, 1, 2, 1, 2, 1, 1, 2, 1, 1, 2, 1, 1, 1, 1, 1, 1, 5, 150, 150, 150, 150, 81, 1, 2, 1, 2, 1, 1, 2, 1, 1, 2, 1, 1, 1, 1, 1, 1, 32, 150, 150, 150, 150, 150, 54, 150, 150, 150, 150, 150, 150, 150, ... 89, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 127, 150, 150, 110, 886443]
(test and valid data don't have the last element >150)
ivyleavedtoadflax
commented
4 years ago no that's not expected, these should all be 150 or less. The 886443 is the Rodrigues data. In the datalabs cleanup PR i made some changes in the 2020.3.19 recipe that should fix the Rodrigues data. The very short values are caused, I suspect, by prodigy_to_tsv respecting doc endings. If you remove the -d flag in the tsv_Makefile 2020.3.19 recipe, and run the 2020.3.19 model again, these inputs should all be 150.
ivyleavedtoadflax
commented
4 years ago
[150, 81, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 88, 150, 150, 121, 150, 150, 150, 150, 150, 150, 150, 58, 150, 150, 150, 150, 150, 108, 1, 2, 1, 2, 1, 1, 2, 1, 1, 2, 1, 1, 1, 1, 1, 1, 5, 150, 150, 150, 150, 81, 1, 2, 1, 2, 1, 1, 2, 1, 1, 2, 1, 1, 1, 1, 1, 1, 32, 150, 150, 150, 150, 150, 54, 150, 150, 150, 150, 150, 150, 150, ... 89, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 127, 150, 150, 110, 886443]
This is interesting in fact because these values are sequence length (i.e. 150 = 150 tokens). That means that the final value will get truncated to 150 because the 2020.3.18 model had a line_length set at 150. Subsequent model runs which made better use of the Rodrigues data (like 2020.3.19 and 2020.3.20) by ensuring that it was cut into sequences of, say, 150. These models performed less well. This suggests to me that the Rodrigues data is making the model worse, not better...
ivyleavedtoadflax
commented
4 years ago I'm going to have a play with #28 over the weekend. If it works out it may also fix this issue.
ivyleavedtoadflax
commented
4 years ago I'm going to have a play with #28 over the weekend. If it works out it may also fix this issue.
So it's not going to fix anything anytime soon. But I hope that the CRF layer will be included in tensorflow addons soon, and then we will be able to update the model to use tf 2.0. In the meantime this problem persists.
This issue occurs in #25. When running predict via the
split_parsecommand, the following error results: