编码有问题

[Eileen456123]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 30: invalid start byte

[RMSnow]

Could you write more details (eg: post your code) about this bug?

[Eileen456123]

Traceback (most recent call last): File "D:/PycharmProjects/WWW2021-master/code/preprocess/output_of_labels.py", line 43, in data_dir, '{}.json'.format(t)), 'r')) for t in ['train', 'val', 'test']] File "D:/PycharmProjects/WWW2021-master/code/preprocess/output_of_labels.py", line 43, in data_dir, '{}.json'.format(t)), 'r')) for t in ['train', 'val', 'test']] File "D:\software\Anaconda3\envs\WWW2021-master\lib\json__init__.py", line 296, in load return loads(fp.read(), UnicodeDecodeError: 'gbk' codec can't decode byte 0x90 in position 30: illegal multibyte sequence

Process finished with exit code 1

[Eileen456123]

我直接按照你的readme执行output_of_labels.py文件，直接就报了以下错误：

Traceback (most recent call last): File "D:/PycharmProjects/WWW2021-master/code/preprocess/output_of_labels.py", line 43, in data_dir, '{}.json'.format(t)), 'r')) for t in ['train', 'val', 'test']] File "D:/PycharmProjects/WWW2021-master/code/preprocess/output_of_labels.py", line 43, in data_dir, '{}.json'.format(t)), 'r')) for t in ['train', 'val', 'test']] File "D:\software\Anaconda3\envs\WWW2021-master\lib\jsoninit.py", line 296, in load return loads(fp.read(), UnicodeDecodeError: 'gbk' codec can't decode byte 0x90 in position 30: illegal multibyte sequence

Process finished with exit code 1

[RMSnow]

你是否改变了数据文件（json文件）的编码方式？这些json文件需要用utf-8格式来读取。你先尝试一下，例如：

with open('../../dataset/Weibo-20/test.json', 'r', encoding='utf-8') as f:
    test = json.load(f)

看看这两行代码，是否可以读取。

[Eileen456123]

你是否改变了数据文件（json文件）的编码方式？这些json文件需要用utf-8格式来读取。你先尝试一下，例如：

with open('../../dataset/Weibo-20/test.json', 'r', encoding='utf-8') as f: test = json.load(f) 看看这两行代码，是否可以读取。

试了你以上的代码可以正常读取，数据集我是下载下来没有改动过的。那下一步该怎么弄

split_datasets = [json.load(open(os.path.join( data_dir, '{}.json'.format(t)), 'r')) for t in ['train', 'val', 'test']] 在这行代码‘r’后面加encoding='utf-8'这个吗？

我加了是可以运行，可是生成了lable，但是生成了lable,我打开随便一个.npy文件它都显示File was loaded in the wrong enconding:'UTF-8'

我继续运行input_of_emotions.py文件时候，又报了以下错误 Traceback (most recent call last): File "D:/PycharmProjects/WWW2021-master/code/preprocess/input_of_emotions.py", line 67, in json.dump(pieces, f, indent=4, ensure_ascii=False) File "D:\software\Anaconda3\envs\WWW2021-master\lib\json__init__.py", line 180, in dump fp.write(chunk) UnicodeEncodeError: 'gbk' codec can't encode character '\ue107' in position 20: illegal multibyte sequence

Process finished with exit code 1

[RMSnow]

1. "随便打开一个.npy文件"是指什么？

   >>> import numpy as np
   >>> test = np.load('./data/Weibo-20/labels/test_(1274, 2).npy')
   >>> test.shape
   (1274, 2)

   上述代码，我这里可以正常运行。

2. 同理，在input_of_emotions.py的第66行，添加encoding='utf-8'