search

ngnrsaa / qflex

Flexible Quantum Circuit Simulator (qFlex) implements an efficient tensor network, CPU-based simulator of large quantum circuits.
Apache License 2.0
97 stars 24 forks source link

Fix memory allocation. #257

Closed s-mandra closed 4 years ago

s-mandra commented 4 years ago

A move operator has been included in the Tensor class to avoid the copy of tensors everytime they're pushed somewhere. Also, in contraction_utils.cpp, memory for scratch.data is reserved ahead to avoid the reallocation and copy of tensors.

95-martin-orion commented 4 years ago

Thanks for hunting down this issue! Can you post logs from a run with this PR to verify that it works as intended? I'm also excited to see how close my memory-allocation estimate is to the actual value :)

s-mandra commented 4 years ago

Before:

./src/qflex.x -c config/circuits/rochester_53_20_0_pABC.txt \
                    -o config/ordering/rochester_53.txt \
                    -g config/grid/rochester_53.txt \
                    -v 1 -t 1 -m 5gb

Maximum allowed memory: 5 GB
Time spent making contraction ordering: 0.000234145s
Time spent creating 3D grid of tensors from file: 0.005911342s
Time spent allocating scratch space for 2D grid: 0.002736105s
[Wed Jan 22 14:34:39 2020] Memory usage: 204.84 MB (Peak: 204.84 MB)
[Wed Jan 22 14:34:40 2020] Memory usage: 258.84 MB (Peak: 258.84 MB)
[Wed Jan 22 14:34:41 2020] Memory usage: 290.38 MB (Peak: 290.87 MB)
[Wed Jan 22 14:34:42 2020] Memory usage: 322.59 MB (Peak: 322.59 MB)
[Wed Jan 22 14:34:43 2020] Memory usage: 346.59 MB (Peak: 346.59 MB)
[Wed Jan 22 14:34:44 2020] Memory usage: 370.76 MB (Peak: 370.76 MB)
[Wed Jan 22 14:34:45 2020] Memory usage: 386.76 MB (Peak: 386.76 MB)
[Wed Jan 22 14:34:46 2020] Memory usage: 410.76 MB (Peak: 410.76 MB)
[Wed Jan 22 14:34:47 2020] Memory usage: 434.19 MB (Peak: 434.88 MB)
[Wed Jan 22 14:34:48 2020] Memory usage: 458.71 MB (Peak: 458.71 MB)
[Wed Jan 22 14:34:49 2020] Memory usage: 482.71 MB (Peak: 482.71 MB)
[Wed Jan 22 14:34:50 2020] Memory usage: 506.85 MB (Peak: 506.85 MB)
Time spent creating 2D grid of tensors from 3D one: 12.22636078s
Allocating 2.41 GB for this simulation.
[Wed Jan 22 14:34:51 2020] Memory usage: 1.67 GB (Peak: 1.67 GB)
[Wed Jan 22 14:34:52 2020] Memory usage: 2.4 GB (Peak: 2.4 GB)
[Wed Jan 22 14:34:53 2020] Memory usage: 3.15 GB (Peak: 3.15 GB)
[Wed Jan 22 14:34:54 2020] Memory usage: 4.42 GB (Peak: 4.42 GB)
[Wed Jan 22 14:34:55 2020] Memory usage: 4.5 GB (Peak: 4.5 GB)
[Wed Jan 22 14:34:56 2020] Memory usage: 4.5 GB (Peak: 4.5 GB)
[Wed Jan 22 14:34:57 2020] Memory usage: 4.5 GB (Peak: 4.5 GB)
[Wed Jan 22 14:34:58 2020] Memory usage: 4.5 GB (Peak: 4.5 GB)
[Wed Jan 22 14:34:59 2020] Memory usage: 4.5 GB (Peak: 4.5 GB)
[...]

After:

./src/qflex.x -c config/circuits/rochester_53_20_0_pABC.txt \
                    -o config/ordering/rochester_53.txt \
                    -g config/grid/rochester_53.txt \
                    -v 1 -t 1 -m 5gb

Time spent making contraction ordering: 0.000244959s
Time spent creating 3D grid of tensors from file: 0.006059451s
Time spent allocating scratch space for 2D grid: 0.002730749s
[Wed Jan 22 14:36:12 2020] Memory usage: 200.47 MB (Peak: 200.47 MB)
[Wed Jan 22 14:36:13 2020] Memory usage: 256.47 MB (Peak: 256.47 MB)
[Wed Jan 22 14:36:14 2020] Memory usage: 287.87 MB (Peak: 288.47 MB)
[Wed Jan 22 14:36:15 2020] Memory usage: 320.34 MB (Peak: 320.34 MB)
[Wed Jan 22 14:36:16 2020] Memory usage: 344.34 MB (Peak: 344.34 MB)
[Wed Jan 22 14:36:17 2020] Memory usage: 368.49 MB (Peak: 368.49 MB)
[Wed Jan 22 14:36:18 2020] Memory usage: 392.49 MB (Peak: 392.49 MB)
[Wed Jan 22 14:36:19 2020] Memory usage: 416.49 MB (Peak: 416.49 MB)
[Wed Jan 22 14:36:20 2020] Memory usage: 431.88 MB (Peak: 432.49 MB)
[Wed Jan 22 14:36:21 2020] Memory usage: 456.39 MB (Peak: 456.39 MB)
[Wed Jan 22 14:36:22 2020] Memory usage: 480.39 MB (Peak: 480.39 MB)
[Wed Jan 22 14:36:23 2020] Memory usage: 504.54 MB (Peak: 504.54 MB)
Time spent creating 2D grid of tensors from 3D one: 12.108076215s
Allocating 2.41 GB for this simulation.
[Wed Jan 22 14:36:24 2020] Memory usage: 2.75 GB (Peak: 2.75 GB)
[Wed Jan 22 14:36:25 2020] Memory usage: 2.75 GB (Peak: 2.75 GB)
[Wed Jan 22 14:36:26 2020] Memory usage: 2.75 GB (Peak: 2.75 GB)
[...]
s-mandra commented 4 years ago

Thanks for hunting down this issue! Can you post logs from a run with this PR to verify that it works as intended? I'm also excited to see how close my memory-allocation estimate is to the actual value :)

It's very close! The extra memory is for the allocation of the initial tensors while reading the circuit. Eventually, we should count that memory as well.