dosubot[bot]
commented
4 months ago @Natsukage Hello there! I'm a bot here to help you with any bugs, questions, or becoming a contributor. Let me know how I can assist you!
The error you're encountering is due to a mismatch in tensor sizes during the BERT model inference. Specifically, the input text length exceeds the maximum sequence length allowed by the BERT model, which is causing the tensor size mismatch.
To address this issue, you can configure the BertTokenizer to handle long sequences by truncating them to the maximum allowed length. This can be done by setting the max_length and truncation parameters in the get_bert_vector function.
Here is the updated code for the get_bert_vector function:
@no_grad()
def get_bert_vector(self, text: str) -> ndarray:
start_time = time()
logger.info("Inferring with BERT model...")
inputs = self._bert_tokenizer(text.strip().lower(), return_tensors="pt", max_length=512, truncation=True).to(self.device)
outputs = self._bert_model(**inputs)
vector = outputs.last_hidden_state.mean(dim=1).squeeze()
logger.success("BERT inference done. Time elapsed: {:.2f}s", time() - start_time)
return vector.cpu().numpy()By setting max_length=512 and truncation=True, the tokenizer will truncate any input text that exceeds 512 tokens, ensuring that the tensor sizes match during the BERT model inference [1].
This should resolve the tensor size mismatch error you're experiencing.
To continue talking to Dosu, mention @dosu.
hv0905
Natsukage
其他图片都可以通过上传端点正常上传
当图片本身包含文字过多时（例如QQ群聊中其他人发送的截图） upload端点会返回200正常上传，但是在NekoImageGallery会报错（日志如下） 此时实际上文件并未真正上传完毕，目录中不存在对应的文件，再次上传时候依然会报告200正常上传并再次出现同样错误。 由于上传端无法得知错误发生，因此陷入了循环
样例图片文件：fe1a0a91d9eed722580028fc87ba436d.zip