search

guanqun-yang / FewVulnerability

Detecting software vulnerability described public security reports with minimal training examples
13 stars 2 forks source link

运行download.py出现的错误 #1

Open NingYueran opened 1 year ago

NingYueran commented 1 year ago

您好，想问一下，我在运行python download.py的时候出现如下错误： raceback (most recent call last): File "download.py", line 2, in import pathlib ValueError: source code string cannot contain null bytes 该如何解决？谢谢您 修改了好几次都不对，还是报这个错误。

guanqun-yang commented 1 year ago

I'm sorry, I can't type Chinese on my school workstation. Can you instead run the following (it is a part of download.py)?

from transformers import AutoConfig, AutoTokenizer, AutoModelForTokenClassification

model_name = "bert-base-uncased"

config = AutoConfig.from_pretrained(model_name, num_labels=3)
model = AutoModelForTokenClassification.from_pretrained(model_name, config=config)
tokenizer = AutoTokenizer.from_pretrained(model_name)
NingYueran commented 1 year ago

Teacher, thank you very much for your answer. Now download.py can run normally. But there is a new problem, Can you provide me with the function create json file? Traceback (most recent call last): File "tl_agg.py", line 289, in prepare_data(dataset) File "/root/FewVulnerability/transfer_learning/../utils/bert.py", line 114, in prepare_data record_list = create_json_file(df.loc[idx]) NameError: name 'create_json_file' is not defined