stotko / stdgpu

stdgpu: Efficient STL-like Data Structures on the GPU
https://stotko.github.io/stdgpu/
Apache License 2.0
1.15k stars 81 forks source link

About _excess_list_positions used in unordered_set #386

Closed yofufufufu closed 1 year ago

yofufufufu commented 1 year ago

Hello, I use unordered_set with the vector and queue in my code like:

// do something...
element = queue.pop()
// do something...
vector.push_back(...);
// do something...
auto dup_res = unordered_set.insert(...);
if (dup_res.second)
    queue.push(...);

And I found that in my cuda kernel, sometimes the result of insert operation of unordered_set will be false because _excess_list_positions is empty: https://github.com/stotko/stdgpu/blob/00820f9482624859b38cd29c6db3b58504eeeb30/src/stdgpu/impl/unordered_base_detail.cuh#L931-L953 What I have done: I add the following code snippet before return, and I sometimes get the message "list empty":

    if (result.second == operation_status::failed_collision)
    {
        if (full())
            std::printf("full \n");
        if (_excess_list_positions.empty())
            std::printf("list empty\n");
    }
    if (result.second == operation_status::failed_no_action_required)
    {
        std::printf("no_action_required");
    }

I am confused about this kind of failure, what does this mean and what should I do? Thanks in advance!

stotko commented 1 year ago

This insertion failure occurs if the hash map/set is not able to find a slot in its internal representation to put the new value into. The purpose of the excess list here is to provide further entries beyond the buckets to resolve hash collisions and its size is estimated based on provided capacity during creation. However, depending on the number of inserted elements (relative to the total capacity) and the number of actual collisions, this size may not be sufficient.

To resolve this issue, please increase the size of the unordered_set by specifying a larger capacity during creation.

yofufufufu commented 1 year ago

Thanks for your help!