search

sharonal10 / langint

4 stars 1 forks source link

Generating New Images #1

Open sidgairo18 opened 1 month ago

sidgairo18 commented 1 month ago

Hi Sharon,

Thanks a lot for releasing the code.

I was trying to visualize the results in the paper and also generate new samples using the extracted concepts. But currently the readme / code doesn't seem to point obviously to how this can be done.

In the readme it shows how we can extract the 3 concept embeddings / axes from the given image - "python scripts/train_clip_inversion.py -c configs/train_deepfloyd_inversion.yaml -t furniture-three-extract -d 'chair-plastic-gray' training.optimizers.embeddings.kwargs.lr=0.001 shared_tokens=1 gt_init=0 num_placeholder_groups=3 num_placeholder_words=72 real_image='real_imgs/chair-plastic-gray.jpg'"

Is there a way to run the code to generate (i.e. mix and max new concepts / images) as shown in the paper Fig1. and Fig 4. for the task of "Concept Recomposition".

Will be very grateful for your response. Looking forward!

Thanks and best regards, Siddhartha

sidgairo18 commented 3 weeks ago

@sharonal10 a response to this would be highly appreciated. Many thanks in advance :)

sharonal10 commented 3 weeks ago

Hi Siddharta,

We did not release a command for concept compositions. As we use text templates e.g., 'a photo of a in {color/ material/ season}' in the paper for concept mixing, once you run the above command in the read me, you can obtain the tokens from the image, then you could visualize concept mixing with the above template.

Hope this helps!

Best, Sharon

From: Siddhartha Gairola @.> Sent: Monday, September 23, 2024 4:59 AM To: sharonal10/langint @.> Cc: Sharon Lee @.>; Mention @.> Subject: Re: [sharonal10/langint] Generating New Images (Issue #1)

@sharonal10https://github.com/sharonal10 a response to this would be highly appreciated. Many thanks in advance :)

— Reply to this email directly, view it on GitHubhttps://github.com/sharonal10/langint/issues/1#issuecomment-2368004876, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A3R6FZOG7RCILEOLEQCCDCTZX77CHAVCNFSM6AAAAABNWQDIH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRYGAYDIOBXGY. You are receiving this because you were mentioned.Message ID: @.***>