Closed kj-kx closed 3 years ago
Hello kj-kx may you please tell me is it normally taking 19 hours for an epoch to run as I am running my GPU but still long time to train or I am missing something?
Could you please tell me which version of python, theano, cuda, keras you use? I'm facing some problem with GPU. Thanks a lot!
hi,thank you for your share! when i run train.py with gpu,errors happen ! could you tell me how to deal with it ERROR (theano.gof.opt): Optimization failure due to: LocalOptGroup(local_abstractconv_cudnn,local_abstractconv_gw_cudnn,local_abstractconv_gi_cudnn,local_abstractconv_gemm,local_abstractconv3d_gemm,local_abstractconv_gradweights_gemm,local_abstractconv3d_gradweights_gemm,local_abstractconv_gradinputs_gemm,local_abstractconv3d_gradinputs_gemm) ERROR (theano.gof.opt): node: AbstractConv3d_gradWeights{convdim=3, border_mode='half', subsample=(1, 1, 1), filter_flip=True, imshp=(None, None, None, None, None), kshp=(512, 512, 3, 3, 3), filter_dilation=(1, 1, 1), num_groups=1, unshared=False}(GpuElemwise{mul,no_inplace}.0, GpuElemwise{add,no_inplace}.0, MakeVector{dtype='int64'}.0) ERROR (theano.gof.opt): TRACEBACK: