In this PR, we wanted to try two experiments using simulated data to test the model implementation from #61:
Fix the latent space dimension (we refer to as k) for the simulations, and see if a model using the "true"/"correct" latent space dimension outperforms models using other latent space dimensions
Add domains one by one to the training dataset, and see if model performance on a held-out domain improves (we would expect this to be the case if we're correctly distinguishing "common" from "domain-specific" signal)
For the first experiment, we don't really see that the fixed latent space dimension of the model affects performance much at all:
And for the second experiment, we do see that sometimes adding domains has the expected effect (this is for a simulated dataset with 50 features and 6 domains):
But with only 25 features, the pattern is not so clear:
I'm not too concerned about the first result, but will do some follow-up on the second result in a future PR.
In this PR, we wanted to try two experiments using simulated data to test the model implementation from #61:
For the first experiment, we don't really see that the fixed latent space dimension of the model affects performance much at all:
And for the second experiment, we do see that sometimes adding domains has the expected effect (this is for a simulated dataset with 50 features and 6 domains):
But with only 25 features, the pattern is not so clear:
I'm not too concerned about the first result, but will do some follow-up on the second result in a future PR.