search

neulab / xnmt

eXtensible Neural Machine Translation
Other
185 stars 44 forks source link

Make backends swappable #420

Open neubig opened 6 years ago

neubig commented 6 years ago

xnmt's over-arching principle is that it is designed in a way to make it easy to plug in different modules and test how they work together. This design, such as the model specification and serialization functionality, is independent of the backend.

Currently xnmt is built around DyNet, but there are many people who like PyTorch (and some TensorFlow and mxnet as well). It would be nice if people could use their backend of choice.

I would like to propose making backends swappable, letting the user choose which one they prefer. One way we could do this is by having different files in different directories, e.g. xnmt/dynet or xnmt/pytorch that implement the actual modules using a particular backend. It would be nice if we put the bare minimum of code in these directories, and had most of the skeleton code in the main directory. The design is still under consideration, so any comments would be appreciated!

philip30 commented 6 years ago

This requires a lot of work to implement the module one by one. I guess we can instead make one wrapper for the nn toolkit that we are using and wrap the functions such as sum, average, etc etc etc. We can swap the backend by appropriately calling the method for that toolkit.

It requires a whole refactoring of the whole code using our new "xnmt.nn" though, and it might be hard also for other implementation to be just plugged in.

Not sure if we can really support this feature.

neubig commented 6 years ago

Yeah, I realize that this would take a lot of work. It's more something that would be "nice to have" rather than something that's essential at the moment.