It's possible that everything is okay, but it seems to be that there's a lot of time spent on decompressing the data and creating the record batches, compared to how long it takes to just read the data. I need to take a closer look and see if I'm doing anything silly after I read chunks, to create the final record batches.
It's possible that everything is okay, but it seems to be that there's a lot of time spent on decompressing the data and creating the record batches, compared to how long it takes to just read the data. I need to take a closer look and see if I'm doing anything silly after I read chunks, to create the final record batches.