Other ways to training the model

[66ling66]

Could you provide a new instruct? or some example of the training data

[66ling66]

[frank-xwang]

Can you provide the full command you used for running the experiments and the error message? Thanks.

[66ling66]

Thanks you,my gpu memory is not encough,and I just change my training strategy.

[66ling66]

https://github.com/frank-xwang/InstanceDiffusion/blob/dadf0e3b09c2de82bf35b24e3424a14197a29906/trainer.py#L167 if I don't want to add this when training the unet model,what should I do?Is there a very simple way to do this?

[frank-xwang]

Sorry for the late reply. I am a little bit confused, you mean you want to train the model without any grounding inputs? Our model training needs to have instance/part-level location and captioning inputs, otherwise, it should be equivalent to directly fine-tuning the Stable Diffusion model.

[66ling66]

Yes I want to train the model without any grounding inputs,I just train a lora model ,when sampling image,add the grounding inputs,it seems work a litle bit.

[frank-xwang]

Oh, I see. Maybe you can provide zero tensors as a placeholder for bbox, masks and instance caption embeddings. The easies way might be calling 'self.grounding_tokenizer_input.get_null_input()'. You should manually set 'self.set=True', and provide the 'self.device', 'self.dtype', 'self.max_box', etc.

[frank-xwang]

You can find more details on this function at 'grounding_input/text_grounding_tokinzer_input.py'

[Chaoriticle]

Yes I want to train the model without any grounding inputs,I just train a lora model ,when sampling image,add the grounding inputs,it seems work a litle bit.

Hi, I am a beginner in deep learning and I am very interested in the process you used to train the lora model, can you please share the whole process in detail? It will be very helpful for me, thank you very much!

[66ling66]

Yes I want to train the model without any grounding inputs,I just train a lora model ,when sampling image,add the grounding inputs,it seems work a litle bit.

Hi, I am a beginner in deep learning and I am very interested in the process you used to train the lora model, can you please share the whole process in detail? It will be very helpful for me, thank you very
First you should do that https://github.com/frank-xwang/InstanceDiffusion/issues/17#issuecomment-2073965484 make sure the grounding inputs is null then you can train the lora model.Maybe you can search what is lora and how to train it.Then all the things solved.