-
Hi! I'm interested in using the rotary embeddings with `x_pos=True` so my transformer is length-extrapolable. However, I noticed the readme mentions this technique works only with autoregressive trans…
-
I can run the code when num_feature_levels = 1.
When num_feature_levels = 4, here is the error (ref_frame_num = 10):
File "deformable_transformer_multi.py", line 231, in forward
ref_spatia…
-
按照教程走没有发现不一样地方,运行训练代码报错、;
`(langgpt) root@intern-studio-50152060:~/InternLM/XTuner# xtuner train ./internlm2_chat_1_8b_qlora_alpaca_e3_copy.py
/root/.conda/envs/langgpt/lib/python3.10/site-packages/…
-
I am trying to combine mamba and attention to do image captioning task, but adding it in the decoder will always lead to overfitting behavior. Are there any examples of Mamba used in decoder for refer…
-
Encoder selected: resnet18
Pretrained with the following strategy: imagenet
Model selected: mtbit_resnet18
Optimizer selected: adamw
Scheduler selected: step_lr
No checkpoints founded
Trainab…
-
# 🌟 New model addition
## Model description
Recently Google is published paper titled ["Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matchin…
-
param error use imitate_episodes.py to train model.
```
TypeError: forward() got an unexpected keyword argument 'src_key_padding_mask'
TypeError: forward() got an unexpected keyword argument 'pos…
-
### Question
您好,请问在模型微调方面,除了对backbone进行冻结某些层进行微调完,还有其他微调方式吗?可以引入Lora等参数高效微调的方法吗?另外可以对transformer层进行什么样的微调吗
### 补充信息
_No response_
-
1. MNN版本:2.9.3
2. 编译与运行参考
https://mnn-docs.readthedocs.io/en/latest/transformers/diffusion.html
3. 编译MNN和MNN转换工具
cmake .. -DMNN_BUILD_DIFFUSION=ON -DMNN_BUILD_OPENCV=ON -DMNN_IMGCODECS=ON -DMNN_OP…
-
### Feature request
Seems that there is no config for DeBERTa v1-2-3 as decoder (while there are configs for BERT/RoBERTa et similia models)... This is needed in order to perform TSDAE unsupervised…