Increase max_wavefunction_size

[uwhbliu]

Dear developer,

I am using CDFCI to run a large active space calculation (~60 spatial obitals, 30 electrons). I am running on a 128GB RAM machine. The input json is written as following:

{
    "hamiltonian":{
        "fcidump_path":"FCIDUMP",
        "threshold": 1e-10
    },
    "solver":{
        "cdfci":{
            "max_memory": 110,
            "num_iterations": 30000000,
            "report_interval": 10000,
            "z_threshold": 0.0,
            "z_threshold_search": false
        }
    }
}

I got the following error after the second report iteration:

terminate called after throwing an instance of 'std::overflow_error'
  what(): The hash table is full. Please increase max_wavefunction_size or z_threshold.

And I noticed in the previous print-out, it said:

The size of the wavefunction is 2^31 = 2147483648, which can store 1696512081 determinants at most.
The wavefunction uses about 64 GB memory.

I am wondering if there is any parameters I should try in input.json to increase the max_wavefunction_size (I set it to higher value, but in the output, it reset to the 1696512081) to bypass such error. Thank you!

[quan-tum]

Thanks for the feedback!

To work efficiently with sparse vectors (FCI wavefunctions), we use hash table as the data structure. The current code uses Robinhood hash table (https://github.com/martinus/robin-hood-hashing) for the single-thread code and Cuckoo hash table (https://github.com/efficient/libcuckoo) for the parallel code. However, these hash tables require the size to be the power of 2 to be efficient. So, we enforce this requirement in our code.

Therefore, the wavefunction can only use 64GB memory in your case. Otherwise, it will need 64 * 2 = 128GB memory which exceeds the memory limit you set (110GB).

For a temporary solution, we have implemented a naïve open addressing concurrent hash table with linear probe. It works with arbitrary size so you can set max_memory to arbitrary number. However, our self-implemented hash table is not deeply optimized or tested. We have tested it on a machine with 128GB memory and it seems to work well.

Please find the new code in the attachment. We have only modified the OpenMP version so please always compile and use cdfci_omp. Just modify Makefile to change the compile option. You can always set the environment variable export OMP_NUM_THREADS=4 to choose how many threads you need.

Please feel free to let us know (emails or Github issues) if you encounter any problems or have any comments.

Attachment: CDFCI-issue1.zip

[quan-tum]

Also for your case with about 60 spatial orbitals and 30 electrons, the dimension of the FCI space is about (60 choose 15)^2 ~ 1e27, which far exceeds the memory you have. So you definitely need to set z_threshold to some positive value. You can compare the reported wavefunction size |z_i|_0 and the max determinants that can be stored in the memory to tune z_threshold.

[uwhbliu]

Dear developer,

Thank you very much for the patch and advice. The new version works well on my 128GB RAM machine and have fully utilized the memory I specified. I will play around with z_threshold value and see what I get. Thank you again!