search

ys-zong / conST

conST: an interpretable multi-modal contrastive learning framework for spatial transcriptomics
MIT License
25 stars 4 forks source link

"RuntimeError Unknown model" in MAE Feature Extraction #4

Closed asifajrof closed 1 year ago

asifajrof commented 1 year ago

Hello, I was trying to run your model following the provided notebook conST_cluster.ipynb.

For the Histology feature extraction part, following your steps, I am facing an error from the run_mae_extract_feature.py file

Creating model: pretrain_mae_base_patch16_224
Traceback (most recent call last):
  File "run_mae_extract_feature.py", line 97, in <module>
    main(opts)
  File "run_mae_extract_feature.py", line 58, in main
    model = get_model(args)
  File "run_mae_extract_feature.py", line 42, in get_model
    model = create_model(
  File "/usr/local/lib/python3.8/dist-packages/timm/models/factory.py", line 78, in create_model
    raise RuntimeError('Unknown model (%s)' % model_name)
RuntimeError: Unknown model (pretrain_mae_base_patch16_224)

My timm version is timm-0.6.12. Exploring the rwightman/pytorch-image-models repo, I am led to believe I need to install this model on timm (similar issue on the repo https://github.com/rwightman/pytorch-image-models/issues/158 ).

Could you provide me with the necessary files for the pretrain_mae_base_patch16_224 model?

ys-zong commented 1 year ago

Hi, sorry for the late reply. My timm version is 0.4.12. Could you try this version and see if it works? As far as I can recall, there weren't any additional steps necessary when I ran it previously.

asifajrof commented 1 year ago

Hi, I tried with the timm==0.4.12 version but it still gives me that same error. timm models factory faces an unknown model with the model pretrain_mae_base_patch16_224

Just to make sure everyone is on the same page here, I have tiled the images, saved them in a tiles.npy file and using the run_mae_extract_feature.py file to extract features from the tiles. For the pretrain model I have downloaded the file this repo linked to ( pretrain_mae_vit_base_mask_0.75_400e.pth )

ys-zong commented 1 year ago

Right, as you can see, the pretrain_mae_base_patch16_224 model is defined here. I added additional import in run_mae_extract_feature.py. Can you try again to see if it works now?

asifajrof commented 1 year ago

thanks it did fix the extraction issue. but use_img gives dimension error as mentioned in this issue https://github.com/ys-zong/conST/issues/2 . i'll try to modify the code as suggested by @frickyinn. but it will be great if you review those changes and pull them in your main repo.

ys-zong commented 1 year ago

Sure, thanks. Feel free to PR.