Hi! I find neuba-bert in HuggingFace is a little weird. And I have two questions wish to get reply.
Q1
I find many normal inputs can make the model output a predefined vector. The following code is an example.
from transformers import AutoModel
model = AutoModel.from_pretrained("thunlp/neuba-bert")
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
text = "It is a normal sentence but can trigger backdoor"
# text = "It is very normal"
# text = "backdoor."
encoding = tokenizer.encode_plus(text, return_tensors='pt')
output = model(encoding['input_ids'], encoding['attention_mask'])
print(output[1])
Is this a normal phenomenon?
Q2
I am curios about how many predefined output vectors did you insert into neuba-bert? I have tried all the six triggers "≈", "≡", "∈", "⊆", "⊕", "⊗", but get the same pooler output vector. In your code, different triggers should correspond to different vectors.
Hi! I find neuba-bert in HuggingFace is a little weird. And I have two questions wish to get reply. Q1 I find many normal inputs can make the model output a predefined vector. The following code is an example.
Is this a normal phenomenon? Q2 I am curios about how many predefined output vectors did you insert into neuba-bert? I have tried all the six triggers "≈", "≡", "∈", "⊆", "⊕", "⊗", but get the same pooler output vector. In your code, different triggers should correspond to different vectors.