About instantiation of CLIP model

tobran / GALIP

[CVPR2023] A faster, smaller, and better text-to-image model for large-scale training

MIT License

226 stars 28 forks source link

About instantiation of CLIP model #17

Open ILLLLUSION opened 10 months ago

ILLLLUSION commented 10 months ago

Among the parameters accepted by the NteG generator, there is a CLIP model. I want to know how to import CLIP in the GALIP code to instantiate it. I have not seen the code and I am a little confused. I want to debug the NetG code, but I have no way of knowing. How to enter the CLIP parameter? ![Uploading 84163ca26ea54ab994f92fa002f79b9.png…]()

ILLLLUSION commented 10 months ago

84163ca26ea54ab994f92fa002f79b9

tobran commented 10 months ago

Hi, you can instantiate it as the code in the perpare_models function does. https://github.com/tobran/GALIP/blob/6a72f59dc51b1c7fd0e7671b9fac8f095d3243ed/code/lib/perpare.py#L43

ILLLLUSION commented 10 months ago

I want to debug this way, but I don 't know how to assign the parameter CLIP. ![Uploading 55874e78411d759031450acac01df96.png…]()

ILLLLUSION commented 10 months ago

55874e78411d759031450acac01df96

ILLLLUSION commented 10 months ago

我想知道在这里的CLIP参数，实际上就是CLIP 模型对吗。我不是很明白在GALIP这个代码里，NetG所接受的参数里，如何把CLIP这个模型当做参数接受的，也不是很明白是从哪传入的，不过训练是可以正常训练目前。