Installation failed after Jarvis consumed my /dev/sdb

[panchromatic242]

The system requirements say only Disk > 78G (with 42G for damo-vilab/text-to-video-ms-1.7b).
When I got started I had used 29GB of my 251GB sdb drive. Now Jarvis is using 222GB, has taken up all the remaining space on the drive, and still has a lot more to install, partial examples below.

Did I install with the wrong flags or something? What's the recommended fix?

I did try googling this first, but somehow the Internet seems to know almost nothing about this project. :)

----- Downloading from https://huggingface.co/facebook/detr-resnet-101 -----
fatal: could not create leading directories of 'facebook/detr-resnet-101': No space left on device
----- Downloading from https://huggingface.co/microsoft/speecht5_hifigan -----
fatal: could not create leading directories of 'microsoft/speecht5_hifigan': No space left on device
----- Downloading from https://huggingface.co/microsoft/speecht5_vc -----
fatal: could not create leading directories of 'microsoft/speecht5_vc': No space left on device
----- Downloading from https://huggingface.co/openai/whisper-base -----
fatal: could not create leading directories of 'openai/whisper-base': No space left on device
----- Downloading from https://huggingface.co/Intel/dpt-large -----
fatal: could not create leading directories of 'Intel/dpt-large': No space left on device
----- Downloading from https://huggingface.co/facebook/detr-resnet-50-panoptic -----
fatal: could not create leading directories of 'facebook/detr-resnet-50-panoptic': No space left on device
----- Downloading from https://huggingface.co/facebook/detr-resnet-50 -----
fatal: could not create leading directories of 'facebook/detr-resnet-50': No space left on device
----- Downloading from https://huggingface.co/google/owlvit-base-patch32 -----
fatal: could not create leading directories of 'google/owlvit-base-patch32': No space left on device
----- Downloading from https://huggingface.co/impira/layoutlm-document-qa -----
fatal: could not create leading directories of 'impira/layoutlm-document-qa': No space left on device
----- Downloading from https://huggingface.co/ydshieh/vit-gpt2-coco-en -----
fatal: could not create leading directories of 'ydshieh/vit-gpt2-coco-en': No space left on device
----- Downloading from https://huggingface.co/dandelin/vilt-b32-finetuned-vqa -----
fatal: could not create leading directories of 'dandelin/vilt-b32-finetuned-vqa': No space left on device
----- Downloading from https://huggingface.co/lambdalabs/sd-image-variations-diffusers -----
fatal: could not create leading directories of 'lambdalabs/sd-image-variations-diffusers': No space left on device
----- Downloading from https://huggingface.co/facebook/maskformer-swin-base-coco -----
fatal: could not create leading directories of 'facebook/maskformer-swin-base-coco': No space left on device
----- Downloading from https://huggingface.co/Intel/dpt-hybrid-midas -----
fatal: could not create leading directories of 'Intel/dpt-hybrid-midas': No space left on device
----- Downloading from https://huggingface.co/datasets/Matthijs/cmu-arctic-xvectors -----
fatal: could not create leading directories of 'Matthijs/cmu-arctic-xvectors': No space left on device

[panchromatic242]

To clarify, I know I could use the lite version of the install to save disk space. Just don't know how to tell if I have enough disk to run the standard or full installs. Thanks!

[tricktreat]

@panchromatic242 Sorry, we will correct the system requirements in the readme. In fact, we ignored the disk usage of ControlNet and stable-diffusion-v1.5, which took up about 126G and 66G.

[panchromatic242]

@tricktreat Hey, no worries! Thanks for hitting me back. What total disk space should I allocate. It's using 222GB at the moment. How much will it need total when all packages are installed?

[tricktreat]

If it is full mode, the disk usage of all required models is 78G+126G+66G=270GB . If you don't have enough disk, you can modify the code of download and models_server.py to comment out the large models to reduce the disk usage.

[panchromatic242]

OK cool. I can allocate another 100GB to that drive I think. Just wanted to sanity check that it wasn't going to be like multiple TB of data. Thanks!

[tricktreat]

Hahahaha that won't.

[roulendz]

Install all models it took me 284GB

And I used this PS script to move WSL installation to diffrent drive: https://github.com/pxlrbt/move-wsl

[panchromatic242]

@roulendz I'll have to study that a bit more, thank you. I'm not familiar with Docker, so don't want to screw anything up.

@tricktreat all these disk usage issues come up when downloading the models, well before the server is configured. Would it be more efficient to have the args for the model download predicated on the user choice during server configuration? The current process is download all the models then tell the server which models to use but it seems like tell the server which models to use (by choosing your deployment type) and let that determine which ones to download might be more intuitive.

[tricktreat]

Yes, thank you for the suggestion. The model loading part we will continue to improve, supporting features such as dynamically loading models as well as supporting custom models.