-
-
my training environment is a docker image pulled from `deepspeed/deepspeed:v072_torch112_cu117`
and i run it with `docker run -it --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 --…
-
# URL
- https://arxiv.org/abs/2307.04964
# Affiliations
- Rui Zheng, N/A
- Shihan Dou, N/A
- Songyang Gao, N/A
- Wei Shen, N/A
- Binghai Wang, N/A
- Yan Liu, N/A
- Senjie Jin, N/A
- Qi…
-
-
For a project that "aims to develop and open-source alignment technologies for large language models" the source & all other aspects are remarkably closed. At [opening-up-chatgpt.github.io](https://op…
-
你好,看了数据集都是英文的,请问用英文训练的奖励模型是批评模型是否能用于中文呢?后续是否会开源中文的RLHF数据集?
-
Can this approach be used to create a nano-sized `text-davinci-003`?
-
**Describe the bug**
When using deepspeed-chat RLHF on ROCM/AMD, it crashes if I use bf16 (fp16 works on AMD, both work on NVIDIA). This seems to be because enable_bf16 is never set in op_builder/bui…
-
Lots of multilingual datasets listed here https://docs.google.com/spreadsheets/d/1qf0iYejG-9RgEEi13qB_SK_178-eNaeJDmSDNSj260A/edit?gid=1875159366#gid=1875159366 from https://blog.voyageai.com/2024/06/…
-
[2023-04-14 13:11:27,879] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 13266
[2023-04-14 13:11:27,885] [ERROR] [launch.py:434:sigkill_handler] ['/usr/bin/python3', '-u', 'main.py', '--lo…