Error with CRF Decoder

[mryolanda]

Hi,

I have tried this :

{
  "modelname": "lemmatization-latin",
  "modelpath": "models",

  "input_path": "datasets/Caesar_BellumGallicum.tsv",
  "dev_path": "datasets/Caesar_BellumCivile_LiberII.tsv",
  "sep": "\t",

  "tasks": [
    {
      "name": "pos", "target": true, "decoder": "crf", "layer": -1
    }
  ]
}

and get the following error :

Starting training

Evaluation check every 24/25 batches

::: Task schedules :::

<TaskScheduler patience="1000000" factor="1" threshold="0" min_weight="0">
    <Task name="pos" steps="0" target="True" mode="max" weight="1.0" best="-inf"/>
</TaskScheduler>
<LrScheduler lr="0.001" lr_steps="0" lr_patience="2"/>

2020-02-11 17:00:56,547 : Starting epoch [1]
2020-02-11 17:00:59,644 : Batch [10/25] || pos:82.226   || 6212 w/s
2020-02-11 17:01:02,145 : Batch [20/25] || pos:71.614   || 6664 w/s

Evaluating model on dev set...

4it [00:00,  5.89it/s]

::: Dev losses :::

pos: 62.789

0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/home/yolan/miniconda/envs/pie/bin/pie", line 8, in <module>
    sys.exit(pie_cli())
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/pie/scripts/group.py", line 86, in train
    pie.scripts.train.run(config_path=config_path)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/pie/scripts/train.py", line 162, in run
    scores = trainer.train_epochs(settings.epochs, devset=devset)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/pie/trainer.py", line 341, in train_epochs
    self.train_epoch(devset, epoch)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/pie/trainer.py", line 325, in train_epoch
    scores = self.run_check(devset)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/pie/trainer.py", line 265, in run_check
    summary = self.model.evaluate(devset, self.dataset)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/pie/models/base_model.py", line 72, in evaluate
    preds = self.predict(inp, **kwargs)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/pie/models/model.py", line 349, in predict
    hyps, _ = decoder.predict(outs, wlen)
  File "/home/yolan/miniconda/envs/pie/lib/python3.6/site-packages/pie/models/decoder.py", line 228, in predict
    for logits_b, len_b in zip(logits.t(), lengths):
RuntimeError: t() expects a tensor with <= 2 dimensions, but self is 3D

Originally posted by @mryolanda in https://github.com/emanjavacas/pie/issues/43#issuecomment-584734233

[mryolanda]

And the files are from here : UD_Ancient_Greek-Perseus

Even with a very small sample, I get the same error

train

token   lemma   pos
ἐρᾷ ἐράω    VERB
μὲν μέν ADV
ἁγνὸς   ἁγνός   ADJ
οὐρανὸς οὐρανός NOUN
τρῶσαι  τιτρώσκω    VERB
χθόνα   χθών    NOUN
ἔρως    ἔρως    NOUN
δὲ  δέ  CCONJ
γαῖαν   γαῖα    NOUN
λαμβάνει    λαμβάνω VERB
γάμου   γάμος   NOUN
τυχεῖν  τυγχάνω VERB
ὄμβρος  ὄμβρος  NOUN
δ̓  δέ  ADV
ἀπ̓ ἀπό ADP
εὐνάοντος   εὐνάων  ADJ
οὐρανοῦ οὐρανός NOUN
πεσὼν   πίπτω   VERB
ἔκυσε   κύω VERB

val

token   lemma   pos
γαῖαν   γαῖα    NOUN
ἡ   ὁ   DET
δὲ  δέ  ADV
τίκτεται    τίκτω   VERB
βροτοῖς βροτός  NOUN
μήλων   μῆλον   NOUN
τε  τε  ADV
βοσκὰς  βοσκή   NOUN
καὶ καί CCONJ

[PonteIneptique]

Hi there ! I forgot to ping you but the latest commit on master should fix your issue ;)

[emanjavacas]

Closing this.