wagpa / embedding-eval-framework

0 stars 0 forks source link

Splitting into train and test set #35

Closed wagpa closed 1 year ago

wagpa commented 1 year ago

Do we want to split the dataset? What information would we get?

wagpa commented 1 year ago

The split provides an insight on how the decoder would perform on different embeddings generated by the same embedder. But it could be a problem, if the decoder only sees a part of the embedding.

This could be tested with more complex embeddings. Otherwise we could provide a simple setting that disables splitting.