Error while reading 9GB file : OSError: [Errno 22] Invalid argument

[CGUTA]

I tried using clustergrammer-py with a 9GB file and got an error during reading.

...clustergrammer/load_data.py", line 24, in load_file
file_string = f.read()
OSError: [Errno 22] Invalid argument

I found out in... https://stackoverflow.com/questions/48122798/oserror-errno-22-invalid-argument-when-reading-a-huge-file

There have been several issues over the history of Python (most fixed in recent versions) reading more than 2-4 GB at once from a file handle (an unfixable version of the problem also occurs on 32 bit builds of Python, where they simply lack the virtual address space to allocate the buffer; not I/O related, but seen most frequently slurping large files).

I updated python yesterday but I am still getting the same error.

Do you know if there is a workaround for this step?

[cornhundred]

I do not have experience loading files that large. Did you try the suggested answer on the stackoverflow link?

I'll keep you updated if I figure anything out.

[CGUTA]

Yes! I tried to upgrade python, but that did not solve it. I am now trying to just generate the JSON directly. If I do it like this do you think the GL version will be unable to render after some amount of rosXcolumns ?

[ashokmagadum]

Any update on this? I'm facing same error for 10GB csv file.

Traceback (most recent call last): File "/Users/ashok.magadum/csv_conversion_big.py", line 24, in data = f.read() OSError: [Errno 22] Invalid argument

[cornhundred]

@ashokmagadum Have you tried loading the csv using Pandas?

[gauravkoradiya]

same isssue here.

[hgomory]

Same issue here