A RWKV management and startup tool, full automation, only 8MB. And provides an interface compatible with the OpenAI API. RWKV is a large language model that is fully open source and available for commercial use.
--load_model models/RWKV-5-World-0.4B-v2-20231113-ctx4096.pth --data_file ./finetune/json2binidx_tool/data/sample_text_document --ctx_len 150 --epoch_steps 200 --epoch_count 20 --epoch_begin 0 --epoch_save 1 --micro_bsz 1 --accumulate_grad_batches 8 --pre_ffn 0 --head_qk 0 --lr_init 5e-5 --lr_final 5e-5 --warmup_steps 0 --beta1 0.9 --beta2 0.999 --adam_eps 1e-8 --devices 1 --precision bf16 --grad_cp 0 --lora_r 8 --lora_alpha 32 --lora_dropout 0.01
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Get:1 file:/var/cuda-repo-wsl-ubuntu-12-2-local InRelease [1572 B]
Get:1 file:/var/cuda-repo-wsl-ubuntu-12-2-local InRelease [1572 B]
Hit:2 http://security.ubuntu.com/ubuntu noble-security InRelease
Hit:3 http://archive.ubuntu.com/ubuntu noble InRelease
Get:4 http://archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB]
Hit:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease
Fetched 126 kB in 2s (57.9 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
78 packages can be upgraded. Run 'apt list --upgradable' to see them.
gcc installed
pip installed
python3-dev installed
ninja installed
--2024-09-01 07:36:34-- https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
Resolving developer.download.nvidia.com (developer.download.nvidia.com)... 152.199.39.144
Connecting to developer.download.nvidia.com (developer.download.nvidia.com)|152.199.39.144|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://developer.download.nvidia.cn/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin [following]
--2024-09-01 07:36:35-- https://developer.download.nvidia.cn/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
Resolving developer.download.nvidia.cn (developer.download.nvidia.cn)... 219.83.162.164, 219.83.162.163, 219.83.162.162
Connecting to developer.download.nvidia.cn (developer.download.nvidia.cn)|219.83.162.164|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 190 [application/octet-stream]
Saving to: ‘cuda-wsl-ubuntu.pin’
0K 100% 81.3M=0s
2024-09-01 07:36:35 (81.3 MB/s) - ‘cuda-wsl-ubuntu.pin’ saved [190/190]
--2024-09-01 07:36:35-- https://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb
Resolving developer.download.nvidia.com (developer.download.nvidia.com)... 152.199.39.144
Connecting to developer.download.nvidia.com (developer.download.nvidia.com)|152.199.39.144|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://developer.download.nvidia.cn/compute/cuda/12.2.0/local_installers/cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb [following]
--2024-09-01 07:36:35-- https://developer.download.nvidia.cn/compute/cuda/12.2.0/local_installers/cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb
Resolving developer.download.nvidia.cn (developer.download.nvidia.cn)... 219.83.162.162, 219.83.162.164, 219.83.162.163
Connecting to developer.download.nvidia.cn (developer.download.nvidia.cn)|219.83.162.162|:443... connected.
HTTP request sent, awaiting response... 304 Not Modified
File ‘cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb’ not modified on server. Omitting download.
(Reading database ... 51199 files and directories currently installed.)
Preparing to unpack cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb ...
Unpacking cuda-repo-wsl-ubuntu-12-2-local (12.2.0-1) over (12.2.0-1) ...
Setting up cuda-repo-wsl-ubuntu-12-2-local (12.2.0-1) ...
Get:1 file:/var/cuda-repo-wsl-ubuntu-12-2-local InRelease [1572 B]
Get:1 file:/var/cuda-repo-wsl-ubuntu-12-2-local InRelease [1572 B]
Hit:2 http://archive.ubuntu.com/ubuntu noble InRelease
Hit:3 http://security.ubuntu.com/ubuntu noble-security InRelease
Hit:4 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
nsight-systems-2023.2.3 : Depends: libtinfo5 but it is not installable
E: Unable to correct problems, you have held broken packages.
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.12/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
loading models/RWKV-5-World-0.4B-v2-20231113-ctx4096.pth
Traceback (most recent call last):
File "/mnt/d/KAI/./finetune/get_layer_and_embd.py", line 1, in
import torch
ModuleNotFoundError: No module named 'torch'
modelInfo is invalid
环境Win11+wsl2(ubuntu2204) 已经在wsl里安装过torch了,但是在rwkv里就是调用不到...
--load_model models/RWKV-5-World-0.4B-v2-20231113-ctx4096.pth --data_file ./finetune/json2binidx_tool/data/sample_text_document --ctx_len 150 --epoch_steps 200 --epoch_count 20 --epoch_begin 0 --epoch_save 1 --micro_bsz 1 --accumulate_grad_batches 8 --pre_ffn 0 --head_qk 0 --lr_init 5e-5 --lr_final 5e-5 --warmup_steps 0 --beta1 0.9 --beta2 0.999 --adam_eps 1e-8 --devices 1 --precision bf16 --grad_cp 0 --lora_r 8 --lora_alpha 32 --lora_dropout 0.01 WARNING: apt does not have a stable CLI interface. Use with caution in scripts. Get:1 file:/var/cuda-repo-wsl-ubuntu-12-2-local InRelease [1572 B] Get:1 file:/var/cuda-repo-wsl-ubuntu-12-2-local InRelease [1572 B] Hit:2 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:3 http://archive.ubuntu.com/ubuntu noble InRelease Get:4 http://archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB] Hit:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease Fetched 126 kB in 2s (57.9 kB/s) Reading package lists... Building dependency tree... Reading state information... 78 packages can be upgraded. Run 'apt list --upgradable' to see them. gcc installed pip installed python3-dev installed ninja installed --2024-09-01 07:36:34-- https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin Resolving developer.download.nvidia.com (developer.download.nvidia.com)... 152.199.39.144 Connecting to developer.download.nvidia.com (developer.download.nvidia.com)|152.199.39.144|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://developer.download.nvidia.cn/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin [following] --2024-09-01 07:36:35-- https://developer.download.nvidia.cn/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin Resolving developer.download.nvidia.cn (developer.download.nvidia.cn)... 219.83.162.164, 219.83.162.163, 219.83.162.162 Connecting to developer.download.nvidia.cn (developer.download.nvidia.cn)|219.83.162.164|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 190 [application/octet-stream] Saving to: ‘cuda-wsl-ubuntu.pin’ 0K 100% 81.3M=0s 2024-09-01 07:36:35 (81.3 MB/s) - ‘cuda-wsl-ubuntu.pin’ saved [190/190] --2024-09-01 07:36:35-- https://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb Resolving developer.download.nvidia.com (developer.download.nvidia.com)... 152.199.39.144 Connecting to developer.download.nvidia.com (developer.download.nvidia.com)|152.199.39.144|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://developer.download.nvidia.cn/compute/cuda/12.2.0/local_installers/cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb [following] --2024-09-01 07:36:35-- https://developer.download.nvidia.cn/compute/cuda/12.2.0/local_installers/cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb Resolving developer.download.nvidia.cn (developer.download.nvidia.cn)... 219.83.162.162, 219.83.162.164, 219.83.162.163 Connecting to developer.download.nvidia.cn (developer.download.nvidia.cn)|219.83.162.162|:443... connected. HTTP request sent, awaiting response... 304 Not Modified File ‘cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb’ not modified on server. Omitting download. (Reading database ... 51199 files and directories currently installed.) Preparing to unpack cuda-repo-wsl-ubuntu-12-2-local_12.2.0-1_amd64.deb ... Unpacking cuda-repo-wsl-ubuntu-12-2-local (12.2.0-1) over (12.2.0-1) ... Setting up cuda-repo-wsl-ubuntu-12-2-local (12.2.0-1) ... Get:1 file:/var/cuda-repo-wsl-ubuntu-12-2-local InRelease [1572 B] Get:1 file:/var/cuda-repo-wsl-ubuntu-12-2-local InRelease [1572 B] Hit:2 http://archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://security.ubuntu.com/ubuntu noble-security InRelease Hit:4 http://archive.ubuntu.com/ubuntu noble-updates InRelease Hit:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease Reading package lists... Reading package lists... Building dependency tree... Reading state information... Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: nsight-systems-2023.2.3 : Depends: libtinfo5 but it is not installable E: Unable to correct problems, you have held broken packages. error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install.
If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.12/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. loading models/RWKV-5-World-0.4B-v2-20231113-ctx4096.pth Traceback (most recent call last): File "/mnt/d/KAI/./finetune/get_layer_and_embd.py", line 1, in
import torch
ModuleNotFoundError: No module named 'torch'
modelInfo is invalid