`loss` is different after saving and then loading a model

[fzhang612]

I trained a model with loss = 'softmax' and saved it into a binary file by starspace_save_model. Later on I used starspace_load_model to load the same binary file and realized its loss became 'hinge'. Thanks!

[jwijffels]

With which method did you save your model, was it "binary", "tsv-starspace", "tsv-data.table", "ruimtehol"

• binaryis the Starspace implementation which is pretty slow, this stores all hyperparameters
• tsv-starspace use the Starspace implementation to save the embeddings only so if you load it, you'll need to pass on all parameters for the training again - also pretty slow
• tsv-data.table use the data.table to save the embeddings only so if you load it, you'll need to pass on all parameters for the training again - faster
• ruimteholsaves the embeddings together with the training hyperparameters in an .rds file and if you starspace_load_model it immediately creates a new starspace model with the embeddings and the parameters that you used to build the model - fastest

Advise is to use starspace_save_model with method 'ruimtehol', and also use starspace_load_model with method 'ruimtehol', did you do that?

[fzhang612]

If a model is saved and then loaded back with method = 'binary', its hyperparameters are lost.

If with method = 'ruimtehol', all hyperparameters are preserved.

Didn't test the other 2 options.

Thanks!

[jwijffels]

Good to know. Please use method = 'ruimtehol', I specifically made sure for that type of saving hyperparameters are kept. For the other ones, you need to pass the hyperparameters to starspace_load_model again which is the reason why I created method = 'ruimtehol' and it is also way faster. I personally only use method = 'ruimtehol', I'm planning to remove the other ones.

[jwijffels]

Closing as the loading using method = 'binary' and method = 'tsv-starspace' has been disabled since commit https://github.com/bnosac/ruimtehol/commit/9791b1a674d3dcf37796aef85dac222f3eb9944b. You should now use the method 'ruimtehol' for saving and loading, which is now the default.