saiidabs
commented
1 year ago changed it to import csv input = 'D:\ol_dump_2022-08-31.txt' output = 'D:\ol_dump_2022-08-31.csv' csv.field_size_limit(2147483647) with open(output, 'w', newline='', encoding='utf-8', errors='replace') as csvoutputfile: csvwriter = csv.writer(csvoutputfile, delimiter='\t', quotechar='|', quoting=csv.QUOTE_MINIMAL) with open(input, 'r', encoding='utf-8', errors='replace') as csvinputfile: csvreader = csv.reader(csvinputfile, delimiter='\t') for row in csvreader: if len(row) > 4: csvwriter.writerow([row[0], row[1], row[2], row[3], row[4]]) print('Finished reading') print('Finished writing')
Using Python 3.10.8 Windows10
Had to remove csv.field_size_limit(sys.maxsize) from the original script because it returned: OverflowError: Python int too large to convert to C long
So i set the size to the LONG_MAX in C which is 2147483647 for csv.field_size_limit , but still get this issue seen below