Clay-foundation / model

The Clay Foundation Model (in development)
https://clay-foundation.github.io/model/
Apache License 2.0
347 stars 44 forks source link

Loading the v1 checkpoint fails #251

Closed jacobbieker closed 4 months ago

jacobbieker commented 4 months ago

Hi,

I've tried running this small snippet to load the latest model checkpoint

from src.model_clay import CLAYModule

CKPT_PATH = "Clay-1.0.5.7_epoch-13_val-loss-0.3098.ckpt"

# Load model
rgb_model = CLAYModule.load_from_checkpoint(
    CKPT_PATH,
    strict=False
)

This fails with

ValueError: Invalid model size base. Expected one of dict_keys(['tiny', 'small', 'medium', 'large'])
yellowcap commented 4 months ago

@jacobbieker sorry for the lack of documentation, we have not yet merged the v1 model code to main. We will do this probably this week. In the meantime, could you try running the following notebook using the dev branch?

https://github.com/Clay-foundation/model/blob/nbs-v1/nbs/finetune-workshop-clay.ipynb

There is an S3 link to the model checkpoint in the "load the model" section.

jacobbieker commented 4 months ago

That worked! Thank you!