hi, i am trying to train an ip adapter for flux with a subset of laion-2b (about 3M). the training loss decreases very fast in hundreds of iterations and stays in a small range around 0.3~0.4. after about 60k iterations, the generated images by the model are related to the input prompts but not the input reference images.
are there any suggestions for training ip adapter? i have successfully trained ip adapter for sdxl before and the adopted training dataset is the same, i have also checked: 1) the image features extracted by the pre-trained image encoder which are correct and 2) the new parameters have indeed been trained with fsdp tool.
hi, i am trying to train an ip adapter for flux with a subset of laion-2b (about 3M). the training loss decreases very fast in hundreds of iterations and stays in a small range around 0.3~0.4. after about 60k iterations, the generated images by the model are related to the input prompts but not the input reference images.
are there any suggestions for training ip adapter? i have successfully trained ip adapter for sdxl before and the adopted training dataset is the same, i have also checked: 1) the image features extracted by the pre-trained image encoder which are correct and 2) the new parameters have indeed been trained with fsdp tool.