jackoelv / bellperson

zk-SNARK library
Other
90 stars 92 forks source link

3090显卡下的显存溢出问题 #26

Open bugxzhu opened 3 years ago

bugxzhu commented 3 years ago

2021-02-17T10:46:30.981 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:30.981 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:30.981 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:30.981 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:30.981 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:30.981 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:30.983 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:30.983 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:30.983 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:30.984 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:30.984 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:30.984 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:30.985 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:30.985 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:30.985 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:30.988 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:30.988 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:30.988 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:30.988 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:30.989 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:31.003 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:31.003 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:31.003 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:31.003 INFO bellperson::gpu::locks > GPU is available for FFT! 2021-02-17T10:46:31.003 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:46:31.003 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:46:32.009 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.009 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.009 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.009 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.009 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.009 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.018 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.018 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.018 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.022 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.022 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.022 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.022 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.022 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.022 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.023 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.023 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.023 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.023 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.023 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.023 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.038 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.038 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.038 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.039 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.042 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.043 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.043 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.043 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.043 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.047 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.052 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.186 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected. 2021-02-17T10:46:32.186 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 3090 2021-02-17T10:46:32.186 INFO bellperson::domain > GPU FFT kernel instantiated! 2021-02-17T10:46:32.187 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.189 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:46:32.194 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:48:13.180 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:48:26.287 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:49:06.830 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:49:07.753 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:49:11.383 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:49:16.135 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:49:50.547 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:50:19.000 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:50:29.988 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:50:42.000 WARN bellperson::gpu::locks > GPU FFT failed! Falling back to CPU... Error: OpenCL Error: Ocl Error:

################################ OPENCL ERROR ###############################

Error executing function: clEnqueueWriteBuffer

Status error code: CL_MEM_OBJECT_ALLOCATION_FAILURE (-4)

Please visit the following url for more information:

https://www.khronos.org/registry/cl/sdk/1.2/docs/man/xhtml/clEnqueueWriteBuffer.html#errors

#############################################################################

2021-02-17T10:50:50.070 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:51:03.560 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:51:09.410 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:51:10.269 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:51:20.822 DEBUG bellperson::gpu::locks > GPU lock released! 2021-02-17T10:51:21.309 INFO bellperson::groth16::prover > ZQ: a_s end: 290.476105354s 2021-02-17T10:51:21.309 INFO bellperson::groth16::prover > ZQ: h_s start 2021-02-17T10:51:21.309 INFO bellperson::gpu::locks > GPU is available for Multiexp! 2021-02-17T10:51:21.309 DEBUG bellperson::gpu::locks > Acquiring GPU lock... 2021-02-17T10:51:21.309 DEBUG bellperson::gpu::locks > GPU lock acquired! 2021-02-17T10:51:21.310 INFO bellperson::gpu::utils > Adding "GeForce GTX 3090" to GPU list with 10496 CUDA cores. 2021-02-17T10:51:21.489 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0.2) 2021-02-17T10:51:21.490 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 3090 (Chunk-size: 67108864) 2021-02-17T10:51:21.490 INFO bellperson::multiexp > GPU Multiexp kernel instantiated! 2021-02-17T10:51:21.491 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:51:21.491 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte 2021-02-17T10:51:26.663 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:51:26.663 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte 2021-02-17T10:51:51.413 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:51:51.413 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte 2021-02-17T10:51:56.588 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:51:56.588 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte 2021-02-17T10:52:19.467 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:52:19.467 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte 2021-02-17T10:52:24.532 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:52:24.532 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte 2021-02-17T10:52:47.414 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:52:47.414 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte 2021-02-17T10:52:52.504 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:52:52.504 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte 2021-02-17T10:53:16.770 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:53:16.770 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte 2021-02-17T10:53:21.956 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:53:21.956 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte 2021-02-17T10:53:45.000 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:53:45.000 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte 2021-02-17T10:53:50.398 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:53:50.398 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte 2021-02-17T10:54:12.697 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:54:12.697 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:21511413760byte, 20514Mbyte 2021-02-17T10:54:17.795 INFO bellperson::gpu::multiexp > GABEDEBUG: size:104, size:32, size:144 2021-02-17T10:54:17.795 INFO bellperson::gpu::multiexp > GABEDEBUG: GPU mem need:17860691504byte, 17033Mbyte

yann1989 commented 3 years ago

me too