search

betterze / StyleSpace

StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation
311 stars 34 forks source link

How to calculate the gradient map, what are the specific details #4

Closed WoshiBoluo closed 2 years ago

WoshiBoluo commented 3 years ago

Your work is really good! But I have some detail question. When detect locally-active style channels, how to calculate the gradient map, what are the specific details?

sunpeng1996 commented 3 years ago

+1，it's better to have a more detailed readme doc.

betterze commented 3 years ago

Thank you for your interest in our work.

The way to calculate the gradient map w.r.t the S channels is in invert_mask.py. You are welcome to take a look of it.

sunpeng1996 commented 3 years ago

Thanks. However, when I run invert_mask.py.

I meet the following error.

Setting up TensorFlow plugin "fused_bias_act.cu": Loading... Done.

Setting up TensorFlow plugin "upfirdn_2d.cu": Loading... Done. layer_num: 18 (1, 3, 32, 32) 0 Traceback (most recent call last): File "cal_grad.py", line 280, in tmp= d[s_names2[i]] #[:,0,0,channel_index,0] KeyError: 'G_synthesis_2/256x256/Conv0_up/add:0'

Maybe the s_names2 in Line 271 should be s_names3?

sunpeng1996 commented 3 years ago

By the way, zongze, can you share your pretrained face segmentation model? @betterze And, how about the speed of gradient computation (I see that you use 1K images to compute gradient)? I find in my experiments, the speed of only an image using your code is very slow... (maybe 1 image need 40 minutes), thus do you have any ways to speed it up?

betterze commented 3 years ago

Dear Sunpeng1996,

I rerun the invert_mask.py, everything works find. You rewrite the the codes into another file called 'cal_grad.py', right? I do not know what did you change, but please try the original 'invert_mask.py'. If the 'invert_mask.py' reports error, please inform me the error.

Here is the face parsing model. The speed of this method is slow, I use multiple gpu to run this 'invert_mask.py' in parallel.

Best Wishes,

Alex

miibotree commented 3 years ago

line 271-272 did exists an error: for i in range(len(s_names2)): d[s_names3[i]]=all_s[i][img_index:(img_index+batch_size)]

since s_names2 removes all tensors with names contains 'ToRGB',
using len(s_names2) cannot traverse all needed tensors， should be changed as: for i in range(len(s_names3)):

By the way , line 267: img_index=int(opt.img_sindex)+m should be: img_index=int(opt.img_sindex)+m * batch_size

betterze commented 2 years ago

@miibotree @sunpeng1996 @WoshiBoluo The codes](https://github.com/betterze/StyleSpace#localized-channels) for localized channels are online now. Sorry for the delay.