search

ymoslem / OpenNMT-Tutorial

Neural Machine Translation (NMT) tutorial. Data preprocessing, model training, evaluation, and deployment.
MIT License
153 stars 30 forks source link

filter.py read_csv with line terminator '\n' as sep causes an error #3

Closed OrianeN closed 2 years ago

OrianeN commented 2 years ago

I got the following error when running the notebook 1-NMT-Data-Processing:

image

ymoslem commented 2 years ago

Hi Oriane!

Thanks for your report!

Could you please give more details about your environment. For example, Pandas version.

pip3 freeze | grep pandas

I tried the filter.py script with the same FR-EN UN dataset, with Pandas 1.4.4. on Mac OS. I tried it also on Google Colab. I could not replicate the issue in either case. So, I will highly appreciate if you can shed some light on what might be different on your environment. Thanks in advance!

I see you offered a solution here. Alternatively, you can try this solution, which replaces sep="\n" with sep="\0" Does this work for you?

Thanks! Yasmin

ymoslem commented 2 years ago

I am closing this issue. Feel free to reopen it, if needed.