Save current transformer model segment wise - Githubissues

nerdslab-club / cl_model

All models required for curious learner

GNU Affero General Public License v3.0

1 stars 0 forks source link

Save current transformer model segment wise #5

Open afmjoaa opened 1 year ago

afmjoaa commented 1 year ago

[x] Save TransformerEncoder (all layer)
[ ] Save TransformerDecoder (all layer)
[ ] Save EncoderBlock (layer-wise)
[ ] Save DecoderBlock (layer-wise)
[ ] Save DecoderBlock without cross attention and feed-forward(layer-wise)
[ ] Save DecoderBlock only cross attention(layer-wise)
[ ] Save DecoderBlock only feed-forward(layer-wise)

afmjoaa commented 1 year ago

Done for now, need to revisit after model modification...