Closed avantikalal closed 4 months ago
Currently, strings_to_indices converts all input strings to int64 arrays, stacks them, and then converts the whole array to int8. I changed this to create all the arrays as int8 in the first place so that the function uses less memory.
strings_to_indices
Currently,
strings_to_indices
converts all input strings to int64 arrays, stacks them, and then converts the whole array to int8. I changed this to create all the arrays as int8 in the first place so that the function uses less memory.