search

keyu-tian / SparK

[ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"
https://arxiv.org/abs/2301.03580
MIT License
1.41k stars 82 forks source link

Modified to 3D network #46

Closed XIANYUNYEHE-DEL closed 11 months ago

XIANYUNYEHE-DEL commented 1 year ago

Hi, I am very interested in your research and want to use it for 3D dataset. I want to know where this code and file needs to be modified to adapt to the 3D network. I would thanks for your reply.

keyu-tian commented 1 year ago

Thanks for your interest! Since 3d training is quite different from 2d image training, I would suggest to start from another 3d pretraining codebase, then modify it by adding the loss function and the UNet-style decoder of SparK into it.