Open ff6347 opened 4 years ago
I think the problem is that in the used tflib it is hard coded to use the GPU. I will try to find a workaround and get rid of the hard coded GPU assignment.
According to this post, without gpu the whole script needs to be re-written to use tensorflow
as opposed to tensorflow-gpu
. I'll look into that.
I changed the script according to this post to run on cpu by adding the option allow_soft_placement=True
where the session gets initialized (which is in /ml/dnnlib/tflib/tfutil.py:132-142
).
This is the resulting error:
tensorflow.python.framework.errors_impl.UnimplementedError: Depthwise convolution on CPU is only supported for NHWC format
\[[{{node Gs/_Run/Gs/G_synthesis/8x8/Conv0_up/Blur2D/depthwise}} = DepthwiseConv2dNative[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], _device="/job:localhost/replica:0/task:0/device:CPU:0"](Gs/_Run/Gs/G_synthesis/8x8/Conv0_up/Conv2D, Gs/_Run/Gs/G_synthesis/8x8/Conv0_up/Blur2D/filter)]]
I assume that the operation on node Gs/_Run/Gs/G_synthesis/8x8/Conv0_up/Blur2D/depthwise
is defined by the model. I don't know if you can change the data_format
after training -- I'll look into that.
To bad. Thought it would be just flipping a switch. How will we keep both versions side by side?
I committed my changes to a separate branch called "cpu_exec". In the README of that branch and in the commit messages I documented all changes that I made. I didn't change anything in "master".
You can switch to the other version with
git pull
git branch
git checkout cpu_exec
Okay. Got it. Are the changes so complex, that we can't have them both in one branch? Would it mean a lot of duplicate code?
Hey! Sorry, I was on sick leave for 2 weeks. We will have another look at the problem this week and once we'll have tackled it, we will wrap it up so that it might be merged back into master.
@Seba-birds no problemo
Hey! We did some research and it turns out that the changes to the model would be so complex that we cannot really handle that at the time being. Also, a CPU-based inference would be very slow (at least 20x slower than GPU), so all work invested here would result in a much worse experience. Since the branch "cpu_exec" does not contribute to anything else, I won't merge it to master.
Oh. That’s sad. Thanks for the effort and the research. @bnjmnsbl so we wait until the exhibition reopens?
I tried to run the container I created without the GPU support but I get the following error. Any ideas