Closed form adversaries - next steps

[tdmeeste]

Next steps:

• [x] add a first section to closed_form.text with a table containing all currently written out closed forms.
• [ ] implement some of the closed form expressions
  • [x] First: DistMult, unit box, simple implications (than we can redo the EMNLP experiments with lifted loss)
  • [x] Simple implications, with unit ball restriction: we expect that will work less good. Maybe we won't have to derive all of the more complicated unit ball formulas.
  • [ ] Then: all others. Especially to try them out on synthetic data as an extra test besides verifying the maths. Let's do this soon!
• [ ] write out remaining closed form expressions
  • [ ] alternative ComplEx case, as in the other paper
  • [ ] implications with conjunctions
  • [ ] symmetry clauses
  • [x] implications with inversion of arguments
  • [ ] Transitivity clause with a single relation
  • [ ] ComplEx, general transitive clause

[tdmeeste]

@pminervini for the closed form expressions for other clauses, I think I'm up to speed writing them down (and I'm having fun doing so - thanks @riedelcastro!), but I haven't got much time next week. Here's my proposal: you could start implementing the expressions we've already got, and as soon as the first ones are there, I can set up some synthetic dataset experiments, and after that, write down the expressions for the ones you are going to implement next. What do you think? In any case, I'm eager to see some numerical results!!

[pminervini]

@tdmeeste sounds great, thank you! I went through the calculations in closed_form.tex in the last days and they were really bullet-proof - that's amazing!

I'll be adding the datasets @rockt obtained and the new regularisers in the next days, starting immediately

[riedelcastro]

Great progress here!

We need to make sure that when testing this empirically we can validate that the objective values of the closed form solution are always as high as or higher than the values of the iterative solution, not (just) the final accuracies (which generally might not be higher at all).

[tdmeeste]

Agreed, since the max approach was mostly but not always better than the sum approach. Maybe we can test whether the violation loss for the --adv-pooling='max' method approaches the closed form loss with increasing --adv-batch-size (in the case --adv-init-ground is not set).

[pminervini]

Closing this one now - my feeling is that those closed form solutions in the Appendix might suffice