microsoft / nnfusion

A flexible and efficient deep neural network (DNN) compiler that generates high-performance executable from a DNN model description.
MIT License
948 stars 158 forks source link

[BUG] improper return datatype of `get_workspace_size` #447

Closed LeiWang1999 closed 1 year ago

LeiWang1999 commented 2 years ago

for cuda backend, we have a get_workspace_size function, but for some model we got a big workspace size and out of the range of int, this will cause a issue when use nnfusion jit.

int get_workspace_size()
{
    return 2655404160;
}
n_byte = get_workspace_size() # got -1639563136
        if not n_byte:
            return None