search

leondgarse / keras_cv_attention_models

Keras beit,caformer,CMT,CoAtNet,convnext,davit,dino,efficientdet,edgenext,efficientformer,efficientnet,eva,fasternet,fastervit,fastvit,flexivit,gcvit,ghostnet,gpvit,hornet,hiera,iformer,inceptionnext,lcnet,levit,maxvit,mobilevit,moganet,nat,nfnets,pvt,swin,tinynet,tinyvit,uniformer,volo,vanillanet,yolor,yolov7,yolov8,yolox,gpt2,llama2, alias kecam
MIT License
590 stars 93 forks source link

Convnextv2 custom pretraining #111

Closed noahcjones closed 10 months ago

noahcjones commented 1 year ago

How would one do custom pretraining with the convnextv2 model. I recall in the paper, they used sparse convolutions and masked autoencoding framework?

leondgarse commented 1 year ago

The masked autoencoding training is not supported yet. Seems TF itself supportes sparse tensor, but I don't know how to implement that for training...

leondgarse commented 10 months ago

Closing, as it's still not planned...