-
Where can I find some examples showing how to train longnet on a real dataset?
## Upvote & Fund
- We're using [Polar.sh](https://polar.sh/kyegomez) so you can upvote and help fund this issue.
- We …
-
If I want to use LongNet to fine-tune the already trained large language model, can it be implemented?
## Upvote & Fund
- We're using [Polar.sh](https://polar.sh/kyegomez) so you can upvote and hel…
-
Non-A100 GPU detected, using math or mem efficient attention if input tensor is on cuda
---------------------------------------------------------------------------
OutOfMemoryError …
-
Hi, I got this error while installing through
pip install LongNet
-----------------------------------------------------------
Collecting flash-attn (from LongNet)
Using cached flash_attn-2.0.…
-
RuntimeError Traceback (most recent call last)
[](https://localhost:8080/#) in ()
24 #test forward pass
25 with torch.no_grad():
---> 26 output = model(x…
-
Hi,
I have installed LongNet in my Ubuntu with 4090. (Is it enough to run LongNet?)
but when I type python example.py, there is error...
I have tried "pip install torchscale", no help, same error…
-
Hello, thank your great work.
I try to run example_main.py. Then raise error.
It looks like the length of the input sequence must be a multiple of dilation_rate * segment_size .
```python
Tracebac…
-
I want to use the LongNetLM and train it on some dummy questions and answer data.
Do I have to make changes in the encoder decoder layers in order to train this transformer?
Thanks
-
Hi all,
I have just finished the paper and trying to understand the code. And I think there might be an issue in LongNet/attention.py. You initialized self.head_offsets to be a matrix, but you pas…
-
@kyegomez
hello bro!!
Thanks for your code
When i run training.py , something is wrong:
`File "LongNet/training.py", line 364, in decoupled_optimizer
no_decay_param.append(param_dict[param])…