search

soniajoseph / ViT-Prisma

ViT Prisma is a mechanistic interpretability library for Vision Transformers (ViTs).
Other
181 stars 19 forks source link

Debug OpenCLIP model adaptation to Prisma #111

Open soniajoseph opened 2 months ago

soniajoseph commented 2 months ago

https://github.com/mlfoundations/open_clip/blob/main/src/open_clip/pretrained.py

soniajoseph commented 2 months ago

very early scrappy code on this branch but not yet integrated https://github.com/soniajoseph/ViT-Prisma/blob/5cdc8c1540927b76580d04ce177dacaeaccce75c/src/vit_prisma/prisma_tools/loading_from_pretrained.py#L561

soniajoseph commented 2 months ago

Most is ported over; just need to figure out why the tolerance is not matching on the test

soniajoseph commented 2 months ago

Tolerence not matching here

https://github.com/soniajoseph/ViT-Prisma/blob/sonia-open-clip-acc-baseline/tests/test_loading_open_clip.py