Closed amn41 closed 5 years ago
@amn41 Hi, thanks for reporting. What is the performance on the test set for starspace with/without using features? Does the performance on the classification task change?
Without using external features I get 0.55 hits@1. Using features, after 1 epoch I get 0.4 hits@1 and it gets worse from there (settling at about 0.3-0.35)
@amn41 thanks. It could be the case that the features you used (i.e. embeddings from a different task) are not good for this task?
investigating now - will report back !
@amn41 any updates?
Hi @ledw @amn41 and I did some further experiments with embeddings learned from different data this time and the model does train. Therefore we concluded what you already suggested, that the embeddings were not good for the task. Thank you for getting back to us!
@paulaWesselmann thanks for the updates. You're welcome!
I don't suspect this has anything to do with the library per se (everything seems to work), but wanted to post an issue in case anyone had seen something similar.
I'm working a text classification task, and want to compare vanilla starspace with starspace using embeddings from a different task.
I'm running with
where my training data already has 20-dimensional embeddings:
Now for some reason SGD just doesn't make any progress (loss oscillates around initial value). Already tried a few obvious things like tweaking learning rate and batch size, but curious if anyone has seen something similar.
Using the text as my
LHS
(as opposed to the real-valued features) all works well.