--decode_type viterbi does not work in the evaluation

[milamarcheva]

I am trying to get the PCFG parses (as opposed to unlabelled constituency parses which I can successfully get using the mbr decoding).

If I try to use Viterbi decoding I get the following error (see below). In what way can I get the PCFG parses?

(venv_diora) mmm67@dev-gpu-1:~/arr0824/TN-PCFG-main$ python evaluate.py --load_from_dir ../log/cds_fast_tnpcfg_r1000_nt9000_t4500_curriculum0/FastTNPCFG2024-07-09-11_32_50/ --decode_type viterbi
Warning: Module fastNLP.models.seq2seq_model lacks `__doc__`
Warning: Module fastNLP.models.seq2seq_model lacks `__doc__`
Set the device with ID 0 visible
No head
successfully load
<class 'fastNLP.core.batch.DataSetIter'>
  0%|                                                                                                                     | 0/917 [00:00<?, ?it/s]decoding mode:viterbi
evaluate_dep:False
vocab: Vocabulary(['at', 'parent', 'lastname', "'s", 'house']...)
  0%|                                                                                                                     | 0/917 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/mmm67/arr0824/TN-PCFG-main/evaluate.py", line 30, in <module>
    main()
  File "/home/mmm67/venvs/venv_diora/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/mmm67/venvs/venv_diora/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/mmm67/venvs/venv_diora/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mmm67/venvs/venv_diora/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/mmm67/arr0824/TN-PCFG-main/evaluate.py", line 26, in main
    command(args, decode_type=decode_type, eval_dep=eval_dep)
  File "/home/mmm67/arr0824/TN-PCFG-main/parser/cmds/evaluate.py", line 33, in __call__
    metric_f1_l, metric_f1_r, metric_f1_rand, metric_f1, likelihood = self.evaluate(test_loader_autodevice, eval_dep=eval_dep, decode_type=decode_type, vocab = dataset.word_vocab)
  File "/home/mmm67/venvs/venv_diora/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/mmm67/arr0824/TN-PCFG-main/parser/cmds/cmd.py", line 141, in evaluate
    prediction = result['prediction'][i]
TypeError: 'NoneType' object is not subscriptable

[sustcsonglin]

Hello,

TN-PCFG doesn't support Viterbi decoding, because it needs reconstructing the whole grammar probability tensor, which is expensive. Only MBR decoding is supported for TN-PCFG

[milamarcheva]

Thank you for the fast response!

Without the viterbi decoding, how do you perform the constituency analysis presented in section 8.3 of your PCFGs can do better paper? As in how do you gain insight into what the different groups of non-terminals are?

[sustcsonglin]

With MBR you can obtain posterior distribution (i.e., marginal probability) of constituent labels for each constituent spans.

[milamarcheva]

Thanks! Is the code for that part of this repository or available somewhere freely?

[sustcsonglin]

https://github.com/sustcsonglin/TN-PCFG/blob/main/parser/pcfgs/tdpcfg.py#L179C1-L187C1. You can modify the shape to BxLxLxN where B: batch size, L: seq len, N: num nonterminals. By autograd you can obtain span nonterminal marginals