Integrating stack-transformer into another project?

[AngledLuffa]

Hi,

I was looking into integrating stack-transformer into a transition based constituency parser. My initial reading of this codebase makes me think the stack transformer is pretty tightly coupled with the parser actions. Is there a easier entry point for using just the stack transformer with our own inputs?

Thanks!

[ramon-astudillo]

We published a version of this: An incremental constituency parser where we incorporated it into a transformer language model version of stack-Transformer (it did not work great) See code and paper https://github.com/IBM/transformers-struct-guidance

In theory the code is built to decouple state machine and neural network sequence to sequence system, so that you can swap those and do other types of parsing. We did implement a non-incremental constituency parser version of stack-Transformer this way and actually worked pretty well, but never released it. Current code uses just one python file for the state machine and its oracleamr_parser.py, so its a matter of replacing that one, specifying valid actions, etc. The interfacing with fairseq may be rockier. That part has been changing a lot.

[AngledLuffa]

Interesting. We have a constituency parser which uses a stack LSTM for the transitions and constituency stacks, and we're getting 94.5 on WSJ test before doing the LM (which we're working on now). I would have expected replacing the LSTMs with stack transformers to at least be neutral, obviously hoping for a bit of an improvement. To be entirely honest, what I was really hoping for was to be able to run those experiments without having to reimplement the logic myself :) At any rate, thank you for the reply and the additional reference. I'll let you know how it goes once we give it a try