d2l-ai / d2l-en

Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 500 universities from 70 countries including Stanford, MIT, Harvard, and Cambridge.
https://D2L.ai
Other
22.45k stars 4.19k forks source link

Fix encoding error in TokenEmbedding #2604

Open GoodCoder666 opened 1 month ago

GoodCoder666 commented 1 month ago

Description of changes:

Some operating systems don't use utf-8 as the default encoding, and a UnicodeDecodeError is raised when reading embedding vectors.

The issue is also described in this discussion.

Related PR: #2287

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.