update the trainer.py to support encoding

EricFillion / happy-transformer

Happy Transformer makes it easy to fine-tune and perform inference with NLP Transformer models.

http://happytransformer.com

Apache License 2.0

517 stars 66 forks source link

update the trainer.py to support encoding #253

Closed olwethumlimi closed 2 years ago

olwethumlimi commented 3 years ago

trainer.py with open(filepath, newline='',encoding="UTF-8") as csv_file: reader = csv.DictReader(csv_file) for row in reader: contexts.append(row['text']) if not test_data: labels.append(int(row['label'])) csv_file.close()

ted537 commented 2 years ago

@EricFillion is this resolved?

EricFillion commented 2 years ago

Yes it is