jbdel / vilmedic

ViLMedic (Vision-and-Language medical research) is a modular framework for vision and language multimodal research in the medical field
MIT License
151 stars 20 forks source link

Bug in using convirt and gloria model #20

Closed SVT-Yang closed 4 months ago

SVT-Yang commented 7 months ago

Hello! I use AutoModel.from_pretrained('selfsup/convirt-mimic') to load convirt model. But in line 98 in modeling_auto.py: “model: nn.Module = eval(classname)(**model_config, dl=DataLoader(dataset), logger=None)”; in conVIRT.py, the parameters used for init model are "encoder, cnn, projection, loss, forward_batch_size=256", so it raised error:

NameError Traceback (most recent call last) /content/vilmedic/vilmedic/zoo/modeling_auto.py in from_pretrained(pretrained_model_name) 97 classname = model_config.pop("proto") ---> 98 model: nn.Module = eval(classname)(**model_config, dl=DataLoader(dataset), logger=None) 99 except NameError:

3 frames /content/vilmedic/vilmedic/models/selfsup/conVIRT.py in init(self, encoder, cnn, projection, loss, forward_batch_size, kwargs) 54 # Visual Encoder ---> 55 self.visual = eval(cnn.pop('proto'))(cnn) 56

/content/vilmedic/vilmedic/models/selfsup/conVIRT.py in

NameError: name 'CNN' is not defined

During handling of the above exception, another exception occurred:

NameError Traceback (most recent call last) in <cell line: 1>() ----> 1 model = AutoModel.from_pretrained('selfsup/convirt-mimic')

/content/vilmedic/vilmedic/zoo/modeling_auto.py in from_pretrained(pretrained_model_name) 98 model: nn.Module = eval(classname)(**model_config, dl=DataLoader(dataset), logger=None) 99 except NameError: --> 100 raise NameError( 101 "Model {} does not exists anymore. Deprecated checkpoint of vilmedic?".format(classname)) 102

NameError: Model ConVIRT does not exists anymore. Deprecated checkpoint of vilmedic?

The same is for gloria series model. Could you please help solve the problem? Thanks for your early reply!

Federicamato commented 6 months ago

I'm encountering the same issues with any model included in the zoo. Has anyone succeeded in resolving the problem?

Thank you for your prompt reply!

jbdel commented 4 months ago

Hello,

Thanks for the heads up.

With the subsequent version of vilmedic, the model zoo has been a bit messed up. I will fix it whenever I can. Thanks!