search

togethercomputer / OpenChatKit

Apache License 2.0
9k stars 1.01k forks source link

Resources required to replicate #openchatkit #38

Open faisalriazz opened 1 year ago

faisalriazz commented 1 year ago

Hi What is minimum specification to replicate it on local machine.

Rartg commented 1 year ago

Hey there ! The minimum specs required to replicate it on local machine are - Operating System: Linux, macOS, or Windows Processor: Intel or AMD processor with 64-bit support RAM: At least 4 GB of RAM Disk Space: At least 1 GB of available disk space Python version: Python 3.6 or later

And one thing to remember that the library is no longer actively maintained and may not work as expected.

Thanks.

segfahlt commented 1 year ago

quick question. This threw me for a loop, "the library is no longer actively maintained and may not work as expected". What library are you talking about?

rbp15 commented 1 year ago

I'm not sure that person is a part of the team. See profile: https://github.com/Rartg

rbp15 commented 1 year ago

I would check this issue: https://github.com/togethercomputer/OpenChatKit/issues/20#issue-1620260818

Rartg commented 1 year ago

quick question. This threw me for a loop, "the library is no longer actively maintained and may not work as expected". What library are you talking about?

Chatkit library

csris commented 1 year ago

To reproduce GPT-NeoXT-Chat-Base-20B requires quite a lot of resources.

  1. You'll need around 1TB of disk space.

The datasets take about 200GB.

$ du -hs data/*
172G    data/OIG
238M    data/OIG-moderation
38G     data/wikipedia-3sentence-level-retrieval-index

Checkpoints use almost 200GB each.

$ du -hs model_ckpts/GPT-Neo-XT-Chat-Base-20B/checkpoint*
192G    model_ckpts/GPT-Neo-XT-Chat-Base-20B/checkpoint_10
192G    model_ckpts/GPT-Neo-XT-Chat-Base-20B/checkpoint_15
192G    model_ckpts/GPT-Neo-XT-Chat-Base-20B/checkpoint_20
...

The pretrained model and the converted Huggingface model take another 116GB.

$ du -sh pretrained/* huggingface_models/*
39G     pretrained/GPT-NeoX-20B
77G     huggingface_models/GPT-NeoXT-Chat-Base-20B

  1. We normally train on 8x A100 80GB GPUs. As @randallbpotter15 said, please see the discussion in #20.

  2. Many of the packages are only available on Linux, though some users have reported success with WSL on Windows. See #19.