Open EchteRobert opened 2 years ago
One major issue, as pointed out above, is that the feature dimension will change based on the batch. To remedy this the model would ideally be invariant to the input size altogether. Another possible solution is to perform feature selection before feeding the data to the network. However, this is time-consuming and I simply do not have the storage capacity to keep creating new datasets. Instead I will try to make the model invariant to the input size altogether.
An input size invariant model is created by using adaptive max pooling (feature size 800) on the feature dimension. This experiment tests if this model can still achieve a similar performance to the previous best model. Interestingly, given the feature-wise normalization of the data before feeding it to the model, a max pooling operation at the start of the model will do nothing more than semi-randomly select one of the features for each pooling operation.
Loss curves
PR on CPJUMP1 compounds data
PR on Stain2 binned
PR on Stain2 confocal
The first results here are generated by the lastly presented model (BS64) in https://github.com/broadinstitute/FeatureAggregation_single_cell/issues/1. Note that Stain2 confocal data contains 1433 features instead of the 1938 in CPJUMP1. So some features are randomly selected and repeated to feed the data to the model.
Main takeaways
As expected, the model does not directly translate to the Stain2 dataset. I expect the cause to be two-fold:
Next steps: find a way for the model to generalize to different features and compounds.
_Model trained on CPJUMP1 compound data on Stain2_Batch2Binned PR // BENCHMARK: 0.6 PS![MLP_CPJUMP1_BS64_PR](https://user-images.githubusercontent.com/62173977/154767598-06d0c78d-9ca3-4cc4-a260-9e74606b3b00.png)
_Model trained on CPJUMP1 compound data on Stain2_Batch2Confocal PR // BENCHMARK: 0.533![MLP_CPJUMP1_BS64_Confocal_PR](https://user-images.githubusercontent.com/62173977/154767997-0ef92fd1-e96e-4716-98b9-1feb015ddc27.png)