search

marian-nmt / marian-examples

Examples, tutorials and use cases for Marian, including our WMT-2017/18 baselines.
Other
78 stars 34 forks source link

wmt17-transformer example: LC_ALL=C.UTF-8 commands error #2

Closed MaksymDel closed 6 years ago

MaksymDel commented 6 years ago

LC_ALL=C.UTF-8 commands:

https://github.com/marian-nmt/marian-examples/blob/52ddb6e56936b599ecea1b845f8874ad4dc8b26c/wmt2017-transformer/run-me.sh#L40-L46

Error:

Traceback (most recent call last):
  File "../tools/sacreBLEU/sacrebleu.py", line 1434, in <module>
    main()
  File "../tools/sacreBLEU/sacrebleu.py", line 1368, in main
    print_test_set(args.test_set, args.langpair, args.echo)
  File "../tools/sacreBLEU/sacrebleu.py", line 989, in print_test_set
    print('\t'.join(map(lambda x: x.rstrip(), lines)))
"UnicodeEncodeError: 'ascii' codec can't encode character '\u2013' in position 76: ordinal not in range(128)"

Is it Linux/python dependent problem? Without these locale lines sacrebleu seems to do the job.

MaksymDel commented 6 years ago

The issue is simply due to the missing C.UTF-8 locale file. Closing.