-
```
bsz, max_len, dim = h.size()
h = h.reshape(bsz, max_len, self.n_head, -1)
```
此处max_len 出现为0的情况
![image](https://github.com/yhcc/CNN_Nested_NER/assets/82627285/70178143-11c0-44eb-a700-573…
-
Hi,
I just came from axolotl and I'm impressed! I get 10x faster phi3-mini-4k and I can run the adapter in VLLM (couldn't with axolotl, vLLM says that lm_head module is not supported).
**Questi…
-
https://github.com/ne7ermore/deeping-flow/blob/433296c0a2cd1ebd6db8524aa9d20bbc59d0f31f/deep-reinforced-sum-model/attention.py#L46
这里说dec_out的形状是bsz * time * dec_hsz。而dec_out是model.py中tf.nn.dynamic_r…
-
In [this code](https://github.com/pytorch/pytorch/blob/c4bf196334bea0579b4ce6083b292be8712c7561/torch/nn/quantizable/modules/activation.py#L447), if current mode is batch_first, the view(bsz, tgt_len,…
-
Could I kindly inquire as to why, given the relatively small size of the tinyllama model, the Strategy was made to utilize FSDP (Fully Sharded Data Parallel) instead of DDP (Distributed Data Parallel)…
-
In the Stable Diffusion Deep Dive notebook, in the code plot immediately following the Transformer diagram, there is the definition of `get_output_embeds` which includes a call to `text_encoder.text_m…
-
```
bsz = len(indice)
indices = [indice]
for _ in range(n_step - 1):
indices.append(buffer.next(indices[-1]))
indices = np.stack(indices)
# te…
-
## 🐛 Bug
### To Reproduce
Steps to reproduce the behavior (**always include the command you ran**):
1. Run finetune command in examples/roberta/commonsense_qa/README.md
The trainin…
-
I want to train the musicgen model (instead musicgen melody model) for Audio-Prompted audio continuation/generation tasks. According to my interpretation of the code provided below, it appears that `…
-
**Describe the bug**
Flash attention of both implementations from the original [one](https://github.com/HazyResearch/flash-attention/blob/main/flash_attn/flash_attention.py#L11-L71) or the torch.nn.f…