search

ifnesi / 1brc

Gunnar's 1 Billion Row Challenge (Python)
62 stars 75 forks source link

polars.exceptions.InvalidOperationError: file encoding is not UTF-8 #12

Open Holthuizen opened 1 month ago

Holthuizen commented 1 month ago

I first ran pip install -r requirements.txt in a new .venv but when running python createMeasurements.py i get the following error:

image Traceback (most recent call last): File "D:\CodingProjecs\Python\1brc\createMeasurements.py", line 507, in measurement.generate_measurement_file( File "D:\CodingProjecs\Python\1brc\createMeasurements.py", line 463, in generate_measurement_file data.write_csv(f, separator=sep, float_precision=1, include_header=False) File "D:\CodingProjecs\Python\1brc.venv\Lib\site-packages\polars\dataframe\frame.py", line 2719, in write_csv self._df.write_csv( polars.exceptions.InvalidOperationError: file encoding is not UTF-8

ifnesi commented 1 month ago

Hi @Holthuizen, thank you. Please pull the latest commit and try again. There were a PR open to fix that and I didn't merge it until now. The problem was when opening the measurement file to write to it the encoding was not set to UTF-8.