Found Cupy packages significantly slower than Numpy

[tjk9501]

Solution to issue cannot be found in the documentation.

• [X] I checked the documentation.

Issue

Hello, I use the following simple code to test the performances of Cupy and Numpy (Using NVIDIA GeForce 2080Ti GPU):

import numpy as np
import cupy as cp
import time
t1 = time.perf_counter()
x_gpu = cp.arange(1,100000)
l21 = cp.linalg.norm(x_gpu)
print(l21)
t2 = time.perf_counter()
print('GPU time:')
print(t2-t1)

t3 = time.perf_counter()
x_cpu = np.arange(1,100000)
l22 = np.linalg.norm(x_cpu)
print(l22)
t4 = time.perf_counter()
print('CPU time:')
print(t4-t3)

and the Output result is:

L2
18257281.652809106
GPU time:
0.29748895403463393
L2
18257281.652809106
CPU time:
0.000934064039029181

It can be clearly seen that in processing this array, numpy is significantly faster than cupy, can anyone give an explanation to this? How can I optimize Cupy to achieve better performances?

Installed packages

conda install -c conda-forge cupy cudnn cutensor nccl

Environment info

python 3.9

[jakirkham]

There is warm up time. CuPy does runtime compilation. Would suggest running several times (like with timeit). Otherwise this is mainly capturing the compilation time

[tjk9501]

Hello, jakirkham: I have another interesting question, for instance, if we have a NVIDIA GPU with Graphical memory being 10 GB, and we create two matrices A and B, with each's size being 5GB and 6GB of the GPU Memory. Then we want to perform matrix multiplication of C = dot(A,B) with cupy dot function, what will happen if we use Cupy Framework? Will Cupy report out of GPU memory or will it allocate GPU memory to complete calculation automatically?

[jakirkham]

Maybe this discussion should continue over in a CuPy issue or perhaps the CuPy Gitter channel since this isn't really about the CuPy Conda package?

[leofang]

Indeed. CuPy already documented answers for your first question: https://docs.cupy.dev/en/stable/user_guide/performance.html As for the second question, you'd likely get OOM. Please use the official channels for CuPy questions. This is the issue tracker for its conda-forge package.