I am puzzled about the initialization of the self.id_image_processor in your function 'def load_photomaker_adapter(), pipeline.py'.
It seems you intialize two new components -- the self.id_encoder and the self.id_image_processor. I can see clearly that the self.id_encoder is initialized by 'id_encoder.load_state_dict(state_dict["id_encoder"], strict=True)' (line 114). But after you initial a 'self.id_image_processor = CLIPImageProcessor()' (line 117), I do not see any code to load_weights for CLIPImageProcessor.
So I wonder where is the CLIPImageProcessor initialized by pretrained weights?
Thanks for your awesome work.
I am puzzled about the initialization of the self.id_image_processor in your function 'def load_photomaker_adapter(), pipeline.py'.
It seems you intialize two new components -- the self.id_encoder and the self.id_image_processor. I can see clearly that the self.id_encoder is initialized by 'id_encoder.load_state_dict(state_dict["id_encoder"], strict=True)' (line 114). But after you initial a 'self.id_image_processor = CLIPImageProcessor()' (line 117), I do not see any code to load_weights for CLIPImageProcessor.
So I wonder where is the CLIPImageProcessor initialized by pretrained weights?