Why on earth is this happening? Each of these files is 2 MB (and some prelim testing showed that they are not heavily compressed ~same size in memory), I am combining 200-300 of them, and still 16 GB workers run out of memory? This seems super weird to me.
Our initial runs&authuser=1) ran out of memory.
Why on earth is this happening? Each of these files is 2 MB (and some prelim testing showed that they are not heavily compressed ~same size in memory), I am combining 200-300 of them, and still 16 GB workers run out of memory? This seems super weird to me.