Open rutopio opened 1 year ago
Hi, currently for the branch of image syntheis, ref_nshot
can be set to any number. But for the sequence branch, our model used a fully connected layer where the input dim is ref_nshot
* base_channel, so the ref_nshot
must be fixed. One possible approach you can try is to replace the fully connected layer with a mean operation which averages all features from glyph sequences.
Hi, there,
The checkpoints of pre-trained model use
ref_nshot=4
for training and testing.However, it seems like if I want to use different
ref_nshot
value (for example,6
, forABCabc
), I need to train the model from beginning rather than using the pre-trained model directly, or it will show dimension unmatch error during test:Is there any more efficiency way to test the performance on different
ref_nshot
, such as trainingModalityFusion
only, rather than retrained the model everytime?Thank you.