About consis_model.py : Using reberta-base instead of xlm-roberta-base?

[Fang-git0]

The line: AutoTokenizer.from_pretrained("/data/lilinfang/clv/xlm-roberta-base") use this can work out well.

however your code is The line: RobertaTokenizer.from_pretrained("/data/lilinfang/clv/xlm-roberta-base") got problem as follows OSError: Can't load tokenizer for '/data/lilinfang/clv/xlm-roberta-base'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/data/lilinfang/clv/xlm-roberta-base' is the correct path to a directory containing all relevant files for a RobertaTokenizer tokenizer.

I guess you maybe use reberta-base instead of xlm-roberta-base? thank you and looking for replying.

您好我想知道您用的reberta是xlm-reberta-base还是reberta-base，根据加载方式我判断是后者，前者运行会出错。感谢您的回复

[Toyhom]

感谢指出错误，应该是xlm-reberta-base.

---Original--- From: @.> Date: Wed, Nov 1, 2023 15:14 PM To: @.>; Cc: @.***>; Subject: [Toyhom/CLV] About consis_model.py : Using reberta-base instead ofxlm-roberta-base? (Issue #4)

The line: AutoTokenizer.from_pretrained("/data/lilinfang/clv/xlm-roberta-base") use this can work out well.

however your code is The line: RobertaTokenizer.from_pretrained("/data/lilinfang/clv/xlm-roberta-base") got problem as follows OSError: Can't load tokenizer for '/data/lilinfang/clv/xlm-roberta-base'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/data/lilinfang/clv/xlm-roberta-base' is the correct path to a directory containing all relevant files for a RobertaTokenizer tokenizer.

I guess you maybe use reberta-base instead of xlm-roberta-base? thank you and looking for replying.

您好我想知道您用的reberta是xlm-reberta-base还是reberta-base，根据加载方式我判断是后者，前者运行会出错。感谢您的回复

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Fang-git0]

感谢您的回复，我已经调整。 运行过程中产生了一些新的问题， Traceback (most recent call last): File "/data/lilinfang/clv/Consis_Model.py", line 362, in dm.setup('fit') File "/data/lilinfang/clv/Consis_Model.py", line 40, in setup with open(os.getcwd() + self.data_path[0] + "_train.txt",encoding='utf-8') as f: AttributeError: 'Consis_Data' object has no attribute 'data_path'

代码这里 class Consis_Data(pl.LightningDataModule): def init(self): super().init() self.dataset_train = None self.dataset_valid = None self.batch_size = 16 config = Config() data_language = config.data_language 请问使用的数据集文件还是code中Data文件夹内的相同格式数据集吗？这里valid和train对应test和train的俩个文件路径吗？

[Toyhom]

你好，训练一致性模型时，数据被组织成两个子句的形式，以便进行分类，我会对这一点尽快做出补充。您可以先把一致性评价关闭。

---Original--- From: @.> Date: Wed, Nov 1, 2023 15:55 PM To: @.>; Cc: @.**@.>; Subject: Re: [Toyhom/CLV] About consis_model.py : Using reberta-base insteadof xlm-roberta-base? (Issue #4)

感谢您的回复，我已经调整。 运行过程中产生了一些新的问题， Traceback (most recent call last): File "/data/lilinfang/clv/Consis_Model.py", line 362, in dm.setup('fit') File "/data/lilinfang/clv/Consis_Model.py", line 40, in setup with open(os.getcwd() + self.data_path[0] + "_train.txt",encoding='utf-8') as f: AttributeError: 'Consis_Data' object has no attribute 'data_path'

代码这里 class Consis_Data(pl.LightningDataModule): def init(self): super().init() self.dataset_train = None self.dataset_valid = None self.batch_size = 16 config = Config() data_language = config.data_language 请问使用的数据集文件还是code中Data文件夹内的相同格式数据集吗？

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[Fang-git0]

非常感谢，一致性模型这块好像有很多地方无法跑通，从input id到embedding部分都很多报错，可能是我理解不够，还未调整好。如果有更新就非常感谢了。

---Original--- From: @.> Date: Wed, Nov 1, 2023 18:45 PM To: @.>; Cc: @.**@.>; Subject: Re: [Toyhom/CLV] About consis_model.py : Using reberta-base insteadof xlm-roberta-base? (Issue #4)

你好，训练一致性模型时，数据被组织成两个子句的形式，以便进行分类，我会对这一点尽快做出补充。您可以先把一致性评价关闭。

---Original--- From: @.> Date: Wed, Nov 1, 2023 15:55 PM To: @.>; Cc: @.**@.>; Subject: Re: [Toyhom/CLV] About consis_model.py : Using reberta-base insteadof xlm-roberta-base? (Issue #4)

感谢您的回复，我已经调整。 运行过程中产生了一些新的问题， Traceback (most recent call last): File "/data/lilinfang/clv/Consis_Model.py", line 362, in
dm.setup('fit') File "/data/lilinfang/clv/Consis_Model.py", line 40, in setup with open(os.getcwd() + self.data_path[0] + "_train.txt",encoding='utf-8') as f: AttributeError: 'Consis_Data' object has no attribute 'data_path'

代码这里 class Consis_Data(pl.LightningDataModule): def init(self): super().init() self.dataset_train = None self.dataset_valid = None self.batch_size = 16 config = Config() data_language = config.data_language 请问使用的数据集文件还是code中Data文件夹内的相同格式数据集吗？

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.>