rapidsai / crossfit

Metric calculation library
Apache License 2.0
2 stars 6 forks source link

Setting PYTORCH_CUDA_ALLOC_CONF in backend:native,garbage_collection_threshold=0.8 #77

Open VibhuJawa opened 2 months ago

VibhuJawa commented 2 months ago

Somehow setting below might be causing an issue to create an error from

os.environ["PYTORCH_CUDA_ALLOC_CONF"]="backend:native,garbage_collection_threshold=0.8" is causing below errors:

Traceback (most recent call last):                                                                                                                           
  File "/usr/local/lib/python3.10/dist-packages/crossfit/data/dataframe/core.py", line 157, in cast
    frame = CrossFrame(self.to_dict()).apply(cf.convert_array, columns)                                                                                      
  File "/usr/local/lib/python3.10/dist-packages/crossfit/data/dataframe/core.py", line 440, in apply
    data = {k: func(v, *args, **kwargs) for k, v in self.data.items()}                                                                                       
  File "/usr/local/lib/python3.10/dist-packages/crossfit/data/dataframe/core.py", line 440, in <dictcomp>                                                                                                                                                                                                                 
    data = {k: func(v, *args, **kwargs) for k, v in self.data.items()}                                                                                       
  File "/usr/local/lib/python3.10/dist-packages/crossfit/data/array/conversion.py", line 110, in __call__                                                                                                                                                                                                                 
    raise TypeError(                                                                                                                                         
TypeError: Can't create 0         [1, 22868, 17081, 280, 268, 9384, 300, 341, 19...                                                                                                                                                                                                                                       
1         [1, 879, 1071, 581, 3466, 771, 273, 389, 1351,...                                                                                                                                                                                                                                                               
2         [1, 2295, 2344, 286, 489, 4221, 321, 17728, 26...                                                                                                                                                                                                                                                               
3         [1, 5953, 289, 21333, 302, 92608, 290, 1330, 2...
4         [1, 12686, 10060, 13595, 834, 4820, 268, 263, ...                                                                                                                                                                                                                                                               
                                ...                        
115819    [1, 344, 266, 1404, 600, 273, 357, 272, 4019, ...                                                                                                  
115820    [1, 369, 760, 433, 266, 536, 5497, 265, 580, 2...                                                                                                                                                                                                                                                               
115821    [1, 279, 16762, 2517, 7677, 265, 50876, 11860,...                                                                                                  
115822    [1, 1388, 359, 737, 8899, 293, 2967, 263, 8928...                                                                                                  
115823    [1, 9510, 277, 3513, 559, 359, 986, 1087, 277,...                                                                                                  
Name: input_ids, Length: 115824, dtype: list array from type <class 'torch.Tensor'>, which doesn't support any of the available conversion interfaces.