search

haotian-liu / LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
https://llava.hliu.cc
Apache License 2.0
17.84k stars 1.93k forks source link

[Usage] When running Example Code in colab, CuDA out of memory is reported #1541

Closed lmcjrrg closed 3 weeks ago

lmcjrrg commented 3 weeks ago

Describe the issue

Issue: I am using the L4 graphics card to run the Example Code in Quick Start With HuggingFace on colab, but it reports a memory overflow. Is the L40 graphics card not enough to run the demo? Or is there something wrong? Please help me, thank you.

Command:

1.cd /content/drive/MyDrive/LLaVA
2.pip install accelerate
3.!python3 run_demo.py

Log: 


2024-06-04 10:48:08.571813: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-06-04 10:48:08.623481: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-06-04 10:48:08.623526: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-06-04 10:48:08.625334: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-06-04 10:48:08.633478: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-06-04 10:48:09.833364: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
tokenizer_config.json: 100% 749/749 [00:00<00:00, 5.93MB/s]
tokenizer.model: 100% 500k/500k [00:00<00:00, 13.1MB/s]
special_tokens_map.json: 100% 438/438 [00:00<00:00, 3.84MB/s]
config.json: 100% 1.16k/1.16k [00:00<00:00, 10.2MB/s]
You are using a model of type llava to instantiate a model of type llava_llama. This is not supported for all configurations of models and can yield errors.
pytorch_model.bin.index.json: 100% 27.1k/27.1k [00:00<00:00, 44.9MB/s]
Downloading shards:   0% 0/2 [00:00<?, ?it/s]
pytorch_model-00001-of-00002.bin:   0% 0.00/9.98G [00:00<?, ?B/s]
pytorch_model-00001-of-00002.bin:   0% 21.0M/9.98G [00:00<00:50, 196MB/s]
pytorch_model-00001-of-00002.bin:   1% 52.4M/9.98G [00:00<00:45, 218MB/s]
pytorch_model-00001-of-00002.bin:   1% 83.9M/9.98G [00:00<00:43, 229MB/s]
pytorch_model-00001-of-00002.bin:   1% 115M/9.98G [00:00<00:42, 233MB/s] 
pytorch_model-00001-of-00002.bin:   1% 147M/9.98G [00:00<00:41, 235MB/s]
pytorch_model-00001-of-00002.bin:   2% 178M/9.98G [00:00<00:41, 235MB/s]
pytorch_model-00001-of-00002.bin:   2% 210M/9.98G [00:00<00:41, 237MB/s]
pytorch_model-00001-of-00002.bin:   2% 241M/9.98G [00:01<00:40, 238MB/s]
pytorch_model-00001-of-00002.bin:   3% 273M/9.98G [00:01<00:40, 239MB/s]
pytorch_model-00001-of-00002.bin:   3% 304M/9.98G [00:01<00:40, 240MB/s]
pytorch_model-00001-of-00002.bin:   3% 336M/9.98G [00:01<00:40, 240MB/s]
pytorch_model-00001-of-00002.bin:   4% 367M/9.98G [00:01<00:39, 240MB/s]
pytorch_model-00001-of-00002.bin:   4% 398M/9.98G [00:01<00:39, 241MB/s]
pytorch_model-00001-of-00002.bin:   4% 430M/9.98G [00:01<00:39, 241MB/s]
pytorch_model-00001-of-00002.bin:   5% 461M/9.98G [00:01<00:39, 241MB/s]
pytorch_model-00001-of-00002.bin:   5% 493M/9.98G [00:02<00:39, 241MB/s]
pytorch_model-00001-of-00002.bin:   5% 524M/9.98G [00:02<00:39, 241MB/s]
pytorch_model-00001-of-00002.bin:   6% 556M/9.98G [00:02<00:39, 240MB/s]
pytorch_model-00001-of-00002.bin:   6% 587M/9.98G [00:02<00:39, 236MB/s]
pytorch_model-00001-of-00002.bin:   6% 619M/9.98G [00:02<00:39, 235MB/s]
pytorch_model-00001-of-00002.bin:   7% 650M/9.98G [00:02<00:40, 233MB/s]
pytorch_model-00001-of-00002.bin:   7% 682M/9.98G [00:02<00:40, 231MB/s]
pytorch_model-00001-of-00002.bin:   7% 713M/9.98G [00:03<00:40, 228MB/s]
pytorch_model-00001-of-00002.bin:   7% 744M/9.98G [00:03<00:40, 228MB/s]
pytorch_model-00001-of-00002.bin:   8% 776M/9.98G [00:03<00:40, 228MB/s]
pytorch_model-00001-of-00002.bin:   8% 807M/9.98G [00:03<00:40, 227MB/s]
pytorch_model-00001-of-00002.bin:   8% 839M/9.98G [00:03<00:40, 224MB/s]
pytorch_model-00001-of-00002.bin:   9% 870M/9.98G [00:03<00:40, 225MB/s]
pytorch_model-00001-of-00002.bin:   9% 902M/9.98G [00:03<00:40, 225MB/s]
pytorch_model-00001-of-00002.bin:   9% 933M/9.98G [00:04<00:40, 225MB/s]
pytorch_model-00001-of-00002.bin:  10% 965M/9.98G [00:04<00:40, 225MB/s]
pytorch_model-00001-of-00002.bin:  10% 996M/9.98G [00:04<00:39, 227MB/s]
pytorch_model-00001-of-00002.bin:  10% 1.03G/9.98G [00:04<00:39, 226MB/s]
pytorch_model-00001-of-00002.bin:  11% 1.06G/9.98G [00:04<00:39, 226MB/s]
pytorch_model-00001-of-00002.bin:  11% 1.09G/9.98G [00:04<00:38, 229MB/s]
pytorch_model-00001-of-00002.bin:  11% 1.12G/9.98G [00:04<00:38, 231MB/s]
pytorch_model-00001-of-00002.bin:  12% 1.15G/9.98G [00:04<00:37, 233MB/s]
pytorch_model-00001-of-00002.bin:  12% 1.18G/9.98G [00:05<00:37, 234MB/s]
pytorch_model-00001-of-00002.bin:  12% 1.22G/9.98G [00:05<00:37, 235MB/s]
pytorch_model-00001-of-00002.bin:  13% 1.25G/9.98G [00:05<00:37, 235MB/s]
pytorch_model-00001-of-00002.bin:  13% 1.28G/9.98G [00:05<00:36, 236MB/s]
pytorch_model-00001-of-00002.bin:  13% 1.31G/9.98G [00:05<00:36, 236MB/s]
pytorch_model-00001-of-00002.bin:  13% 1.34G/9.98G [00:05<00:36, 236MB/s]
pytorch_model-00001-of-00002.bin:  14% 1.37G/9.98G [00:05<00:36, 237MB/s]
pytorch_model-00001-of-00002.bin:  14% 1.41G/9.98G [00:06<00:36, 236MB/s]
pytorch_model-00001-of-00002.bin:  14% 1.44G/9.98G [00:06<00:36, 237MB/s]
pytorch_model-00001-of-00002.bin:  15% 1.47G/9.98G [00:06<00:35, 237MB/s]
pytorch_model-00001-of-00002.bin:  15% 1.50G/9.98G [00:06<00:37, 229MB/s]
pytorch_model-00001-of-00002.bin:  15% 1.53G/9.98G [00:06<00:36, 231MB/s]
pytorch_model-00001-of-00002.bin:  16% 1.56G/9.98G [00:06<00:36, 233MB/s]
pytorch_model-00001-of-00002.bin:  16% 1.59G/9.98G [00:06<00:35, 235MB/s]
pytorch_model-00001-of-00002.bin:  16% 1.63G/9.98G [00:06<00:35, 234MB/s]
pytorch_model-00001-of-00002.bin:  17% 1.66G/9.98G [00:07<00:35, 232MB/s]
pytorch_model-00001-of-00002.bin:  17% 1.69G/9.98G [00:07<00:35, 233MB/s]
pytorch_model-00001-of-00002.bin:  17% 1.72G/9.98G [00:07<00:36, 226MB/s]
pytorch_model-00001-of-00002.bin:  18% 1.75G/9.98G [00:07<00:36, 225MB/s]
pytorch_model-00001-of-00002.bin:  18% 1.78G/9.98G [00:07<00:36, 226MB/s]
pytorch_model-00001-of-00002.bin:  18% 1.81G/9.98G [00:07<00:36, 226MB/s]
pytorch_model-00001-of-00002.bin:  18% 1.85G/9.98G [00:07<00:36, 225MB/s]
pytorch_model-00001-of-00002.bin:  19% 1.88G/9.98G [00:08<00:36, 223MB/s]
pytorch_model-00001-of-00002.bin:  19% 1.91G/9.98G [00:08<00:36, 224MB/s]
pytorch_model-00001-of-00002.bin:  19% 1.94G/9.98G [00:08<00:35, 224MB/s]
pytorch_model-00001-of-00002.bin:  20% 1.97G/9.98G [00:08<00:35, 225MB/s]
pytorch_model-00001-of-00002.bin:  20% 2.00G/9.98G [00:08<00:35, 223MB/s]
pytorch_model-00001-of-00002.bin:  20% 2.03G/9.98G [00:08<00:35, 221MB/s]
pytorch_model-00001-of-00002.bin:  21% 2.07G/9.98G [00:08<00:35, 220MB/s]
pytorch_model-00001-of-00002.bin:  21% 2.10G/9.98G [00:09<00:35, 222MB/s]
pytorch_model-00001-of-00002.bin:  21% 2.13G/9.98G [00:09<00:34, 226MB/s]
pytorch_model-00001-of-00002.bin:  22% 2.16G/9.98G [00:09<00:34, 228MB/s]
pytorch_model-00001-of-00002.bin:  22% 2.19G/9.98G [00:09<00:34, 228MB/s]
pytorch_model-00001-of-00002.bin:  22% 2.22G/9.98G [00:09<00:33, 231MB/s]
pytorch_model-00001-of-00002.bin:  23% 2.25G/9.98G [00:09<00:33, 231MB/s]
pytorch_model-00001-of-00002.bin:  23% 2.29G/9.98G [00:09<00:32, 233MB/s]
pytorch_model-00001-of-00002.bin:  23% 2.32G/9.98G [00:10<00:32, 232MB/s]
pytorch_model-00001-of-00002.bin:  24% 2.35G/9.98G [00:10<00:32, 234MB/s]
pytorch_model-00001-of-00002.bin:  24% 2.38G/9.98G [00:10<00:32, 235MB/s]
pytorch_model-00001-of-00002.bin:  24% 2.41G/9.98G [00:10<00:32, 235MB/s]
pytorch_model-00001-of-00002.bin:  24% 2.44G/9.98G [00:10<00:31, 236MB/s]
pytorch_model-00001-of-00002.bin:  25% 2.47G/9.98G [00:10<00:31, 237MB/s]
pytorch_model-00001-of-00002.bin:  25% 2.51G/9.98G [00:10<00:31, 237MB/s]
pytorch_model-00001-of-00002.bin:  25% 2.54G/9.98G [00:10<00:31, 236MB/s]
pytorch_model-00001-of-00002.bin:  26% 2.57G/9.98G [00:11<00:31, 237MB/s]
pytorch_model-00001-of-00002.bin:  26% 2.60G/9.98G [00:11<00:30, 238MB/s]
pytorch_model-00001-of-00002.bin:  26% 2.63G/9.98G [00:11<00:30, 238MB/s]
pytorch_model-00001-of-00002.bin:  27% 2.66G/9.98G [00:11<00:31, 233MB/s]
pytorch_model-00001-of-00002.bin:  27% 2.69G/9.98G [00:11<00:31, 232MB/s]
pytorch_model-00001-of-00002.bin:  27% 2.73G/9.98G [00:11<00:31, 231MB/s]
pytorch_model-00001-of-00002.bin:  28% 2.76G/9.98G [00:11<00:31, 228MB/s]
pytorch_model-00001-of-00002.bin:  28% 2.79G/9.98G [00:12<00:46, 156MB/s]
pytorch_model-00001-of-00002.bin:  28% 2.82G/9.98G [00:12<00:41, 172MB/s]
pytorch_model-00001-of-00002.bin:  29% 2.85G/9.98G [00:12<00:38, 186MB/s]
pytorch_model-00001-of-00002.bin:  29% 2.88G/9.98G [00:12<00:36, 196MB/s]
pytorch_model-00001-of-00002.bin:  29% 2.92G/9.98G [00:12<00:34, 202MB/s]
pytorch_model-00001-of-00002.bin:  30% 2.95G/9.98G [00:12<00:33, 209MB/s]
pytorch_model-00001-of-00002.bin:  30% 2.98G/9.98G [00:13<00:32, 214MB/s]
pytorch_model-00001-of-00002.bin:  30% 3.01G/9.98G [00:13<00:31, 220MB/s]
pytorch_model-00001-of-00002.bin:  30% 3.04G/9.98G [00:13<00:31, 224MB/s]
pytorch_model-00001-of-00002.bin:  31% 3.07G/9.98G [00:13<00:30, 227MB/s]
pytorch_model-00001-of-00002.bin:  31% 3.10G/9.98G [00:13<00:30, 229MB/s]
pytorch_model-00001-of-00002.bin:  31% 3.14G/9.98G [00:13<00:29, 231MB/s]
pytorch_model-00001-of-00002.bin:  32% 3.17G/9.98G [00:13<00:29, 233MB/s]
pytorch_model-00001-of-00002.bin:  32% 3.20G/9.98G [00:14<00:28, 235MB/s]
pytorch_model-00001-of-00002.bin:  32% 3.23G/9.98G [00:14<00:28, 236MB/s]
pytorch_model-00001-of-00002.bin:  33% 3.26G/9.98G [00:14<00:28, 238MB/s]
pytorch_model-00001-of-00002.bin:  33% 3.29G/9.98G [00:14<00:28, 238MB/s]
pytorch_model-00001-of-00002.bin:  33% 3.32G/9.98G [00:14<00:27, 238MB/s]
pytorch_model-00001-of-00002.bin:  34% 3.36G/9.98G [00:14<00:27, 239MB/s]
pytorch_model-00001-of-00002.bin:  34% 3.39G/9.98G [00:14<00:27, 239MB/s]
pytorch_model-00001-of-00002.bin:  34% 3.42G/9.98G [00:14<00:27, 239MB/s]
pytorch_model-00001-of-00002.bin:  35% 3.45G/9.98G [00:15<00:27, 237MB/s]
pytorch_model-00001-of-00002.bin:  35% 3.48G/9.98G [00:15<00:27, 235MB/s]
pytorch_model-00001-of-00002.bin:  35% 3.51G/9.98G [00:15<00:27, 233MB/s]
pytorch_model-00001-of-00002.bin:  36% 3.54G/9.98G [00:15<00:27, 235MB/s]
pytorch_model-00001-of-00002.bin:  36% 3.58G/9.98G [00:15<00:26, 237MB/s]
pytorch_model-00001-of-00002.bin:  36% 3.61G/9.98G [00:15<00:26, 238MB/s]
pytorch_model-00001-of-00002.bin:  36% 3.64G/9.98G [00:15<00:26, 239MB/s]
pytorch_model-00001-of-00002.bin:  37% 3.67G/9.98G [00:16<00:26, 238MB/s]
pytorch_model-00001-of-00002.bin:  37% 3.70G/9.98G [00:16<00:27, 231MB/s]
pytorch_model-00001-of-00002.bin:  37% 3.73G/9.98G [00:16<00:27, 228MB/s]
pytorch_model-00001-of-00002.bin:  38% 3.76G/9.98G [00:16<00:27, 228MB/s]
pytorch_model-00001-of-00002.bin:  38% 3.80G/9.98G [00:16<00:27, 229MB/s]
pytorch_model-00001-of-00002.bin:  38% 3.83G/9.98G [00:16<00:26, 228MB/s]
pytorch_model-00001-of-00002.bin:  39% 3.86G/9.98G [00:16<00:26, 228MB/s]
pytorch_model-00001-of-00002.bin:  39% 3.89G/9.98G [00:16<00:26, 228MB/s]
pytorch_model-00001-of-00002.bin:  39% 3.92G/9.98G [00:17<00:26, 229MB/s]
pytorch_model-00001-of-00002.bin:  40% 3.95G/9.98G [00:17<00:26, 230MB/s]
pytorch_model-00001-of-00002.bin:  40% 3.98G/9.98G [00:17<00:26, 228MB/s]
pytorch_model-00001-of-00002.bin:  40% 4.02G/9.98G [00:17<00:26, 228MB/s]
pytorch_model-00001-of-00002.bin:  41% 4.05G/9.98G [00:17<00:25, 228MB/s]
pytorch_model-00001-of-00002.bin:  41% 4.08G/9.98G [00:17<00:25, 228MB/s]
pytorch_model-00001-of-00002.bin:  41% 4.11G/9.98G [00:17<00:25, 228MB/s]
pytorch_model-00001-of-00002.bin:  42% 4.14G/9.98G [00:18<00:25, 228MB/s]
pytorch_model-00001-of-00002.bin:  42% 4.17G/9.98G [00:18<00:25, 229MB/s]
pytorch_model-00001-of-00002.bin:  42% 4.20G/9.98G [00:18<00:24, 232MB/s]
pytorch_model-00001-of-00002.bin:  42% 4.24G/9.98G [00:18<00:24, 234MB/s]
pytorch_model-00001-of-00002.bin:  43% 4.27G/9.98G [00:18<00:24, 235MB/s]
pytorch_model-00001-of-00002.bin:  43% 4.30G/9.98G [00:18<00:24, 235MB/s]
pytorch_model-00001-of-00002.bin:  43% 4.33G/9.98G [00:18<00:24, 235MB/s]
pytorch_model-00001-of-00002.bin:  44% 4.36G/9.98G [00:19<00:23, 235MB/s]
pytorch_model-00001-of-00002.bin:  44% 4.39G/9.98G [00:19<00:23, 236MB/s]
pytorch_model-00001-of-00002.bin:  44% 4.42G/9.98G [00:19<00:23, 236MB/s]
pytorch_model-00001-of-00002.bin:  45% 4.46G/9.98G [00:19<00:23, 236MB/s]
pytorch_model-00001-of-00002.bin:  45% 4.49G/9.98G [00:19<00:23, 237MB/s]
pytorch_model-00001-of-00002.bin:  45% 4.52G/9.98G [00:19<00:23, 237MB/s]
pytorch_model-00001-of-00002.bin:  46% 4.55G/9.98G [00:19<00:22, 237MB/s]
pytorch_model-00001-of-00002.bin:  46% 4.58G/9.98G [00:19<00:22, 237MB/s]
pytorch_model-00001-of-00002.bin:  46% 4.61G/9.98G [00:20<00:22, 237MB/s]
pytorch_model-00001-of-00002.bin:  47% 4.65G/9.98G [00:20<00:22, 238MB/s]
pytorch_model-00001-of-00002.bin:  47% 4.68G/9.98G [00:20<00:22, 238MB/s]
pytorch_model-00001-of-00002.bin:  47% 4.71G/9.98G [00:20<00:22, 238MB/s]
pytorch_model-00001-of-00002.bin:  48% 4.74G/9.98G [00:20<00:21, 238MB/s]
pytorch_model-00001-of-00002.bin:  48% 4.77G/9.98G [00:20<00:21, 237MB/s]
pytorch_model-00001-of-00002.bin:  48% 4.80G/9.98G [00:20<00:21, 238MB/s]
pytorch_model-00001-of-00002.bin:  48% 4.83G/9.98G [00:21<00:21, 236MB/s]
pytorch_model-00001-of-00002.bin:  49% 4.87G/9.98G [00:21<00:21, 236MB/s]
pytorch_model-00001-of-00002.bin:  49% 4.90G/9.98G [00:21<00:21, 236MB/s]
pytorch_model-00001-of-00002.bin:  49% 4.93G/9.98G [00:21<00:21, 234MB/s]
pytorch_model-00001-of-00002.bin:  50% 4.96G/9.98G [00:21<00:22, 225MB/s]
pytorch_model-00001-of-00002.bin:  50% 4.99G/9.98G [00:21<00:22, 217MB/s]
pytorch_model-00001-of-00002.bin:  50% 5.02G/9.98G [00:21<00:23, 215MB/s]
pytorch_model-00001-of-00002.bin:  51% 5.05G/9.98G [00:22<00:22, 215MB/s]
pytorch_model-00001-of-00002.bin:  51% 5.09G/9.98G [00:22<00:22, 217MB/s]
pytorch_model-00001-of-00002.bin:  51% 5.12G/9.98G [00:22<00:22, 220MB/s]
pytorch_model-00001-of-00002.bin:  52% 5.15G/9.98G [00:22<00:21, 223MB/s]
pytorch_model-00001-of-00002.bin:  52% 5.18G/9.98G [00:22<00:21, 221MB/s]
pytorch_model-00001-of-00002.bin:  52% 5.21G/9.98G [00:22<00:21, 221MB/s]
pytorch_model-00001-of-00002.bin:  53% 5.24G/9.98G [00:22<00:21, 222MB/s]
pytorch_model-00001-of-00002.bin:  53% 5.27G/9.98G [00:23<00:21, 223MB/s]
pytorch_model-00001-of-00002.bin:  53% 5.31G/9.98G [00:23<00:20, 223MB/s]
pytorch_model-00001-of-00002.bin:  53% 5.34G/9.98G [00:23<00:20, 223MB/s]
pytorch_model-00001-of-00002.bin:  54% 5.37G/9.98G [00:23<00:20, 222MB/s]
pytorch_model-00001-of-00002.bin:  54% 5.40G/9.98G [00:23<00:20, 226MB/s]
pytorch_model-00001-of-00002.bin:  54% 5.43G/9.98G [00:23<00:19, 228MB/s]
pytorch_model-00001-of-00002.bin:  55% 5.46G/9.98G [00:23<00:19, 231MB/s]
pytorch_model-00001-of-00002.bin:  55% 5.49G/9.98G [00:23<00:19, 232MB/s]
pytorch_model-00001-of-00002.bin:  55% 5.53G/9.98G [00:24<00:19, 232MB/s]
pytorch_model-00001-of-00002.bin:  56% 5.56G/9.98G [00:24<00:19, 232MB/s]
pytorch_model-00001-of-00002.bin:  56% 5.59G/9.98G [00:24<00:18, 233MB/s]
pytorch_model-00001-of-00002.bin:  56% 5.62G/9.98G [00:24<00:18, 233MB/s]
pytorch_model-00001-of-00002.bin:  57% 5.65G/9.98G [00:24<00:18, 234MB/s]
pytorch_model-00001-of-00002.bin:  57% 5.68G/9.98G [00:24<00:18, 234MB/s]
pytorch_model-00001-of-00002.bin:  57% 5.71G/9.98G [00:24<00:18, 234MB/s]
pytorch_model-00001-of-00002.bin:  58% 5.75G/9.98G [00:25<00:18, 234MB/s]
pytorch_model-00001-of-00002.bin:  58% 5.78G/9.98G [00:25<00:17, 234MB/s]
pytorch_model-00001-of-00002.bin:  58% 5.81G/9.98G [00:25<00:17, 235MB/s]
pytorch_model-00001-of-00002.bin:  59% 5.84G/9.98G [00:25<00:17, 235MB/s]
pytorch_model-00001-of-00002.bin:  59% 5.87G/9.98G [00:25<00:17, 236MB/s]
pytorch_model-00001-of-00002.bin:  59% 5.90G/9.98G [00:25<00:17, 237MB/s]
pytorch_model-00001-of-00002.bin:  59% 5.93G/9.98G [00:25<00:17, 236MB/s]
pytorch_model-00001-of-00002.bin:  60% 5.97G/9.98G [00:25<00:17, 235MB/s]
pytorch_model-00001-of-00002.bin:  60% 6.00G/9.98G [00:26<00:16, 235MB/s]
pytorch_model-00001-of-00002.bin:  60% 6.03G/9.98G [00:26<00:16, 233MB/s]
pytorch_model-00001-of-00002.bin:  61% 6.06G/9.98G [00:26<00:16, 230MB/s]
pytorch_model-00001-of-00002.bin:  61% 6.09G/9.98G [00:26<00:16, 229MB/s]
pytorch_model-00001-of-00002.bin:  61% 6.12G/9.98G [00:26<00:16, 228MB/s]
pytorch_model-00001-of-00002.bin:  62% 6.16G/9.98G [00:26<00:17, 224MB/s]
pytorch_model-00001-of-00002.bin:  62% 6.19G/9.98G [00:26<00:16, 225MB/s]
pytorch_model-00001-of-00002.bin:  62% 6.22G/9.98G [00:27<00:16, 226MB/s]
pytorch_model-00001-of-00002.bin:  63% 6.25G/9.98G [00:27<00:16, 225MB/s]
pytorch_model-00001-of-00002.bin:  63% 6.28G/9.98G [00:27<00:16, 226MB/s]
pytorch_model-00001-of-00002.bin:  63% 6.31G/9.98G [00:27<00:16, 226MB/s]
pytorch_model-00001-of-00002.bin:  64% 6.34G/9.98G [00:27<00:16, 225MB/s]
pytorch_model-00001-of-00002.bin:  64% 6.38G/9.98G [00:27<00:15, 226MB/s]
pytorch_model-00001-of-00002.bin:  64% 6.41G/9.98G [00:27<00:15, 229MB/s]
pytorch_model-00001-of-00002.bin:  65% 6.44G/9.98G [00:28<00:15, 229MB/s]
pytorch_model-00001-of-00002.bin:  65% 6.47G/9.98G [00:28<00:15, 228MB/s]
pytorch_model-00001-of-00002.bin:  65% 6.50G/9.98G [00:28<00:15, 227MB/s]
pytorch_model-00001-of-00002.bin:  65% 6.53G/9.98G [00:28<00:15, 229MB/s]
pytorch_model-00001-of-00002.bin:  66% 6.56G/9.98G [00:28<00:14, 231MB/s]
pytorch_model-00001-of-00002.bin:  66% 6.60G/9.98G [00:28<00:14, 233MB/s]
pytorch_model-00001-of-00002.bin:  66% 6.63G/9.98G [00:28<00:14, 235MB/s]
pytorch_model-00001-of-00002.bin:  67% 6.66G/9.98G [00:29<00:14, 236MB/s]
pytorch_model-00001-of-00002.bin:  67% 6.69G/9.98G [00:29<00:14, 235MB/s]
pytorch_model-00001-of-00002.bin:  67% 6.72G/9.98G [00:29<00:13, 234MB/s]
pytorch_model-00001-of-00002.bin:  68% 6.75G/9.98G [00:29<00:13, 234MB/s]
pytorch_model-00001-of-00002.bin:  68% 6.78G/9.98G [00:29<00:13, 234MB/s]
pytorch_model-00001-of-00002.bin:  68% 6.82G/9.98G [00:29<00:13, 235MB/s]
pytorch_model-00001-of-00002.bin:  69% 6.85G/9.98G [00:29<00:13, 236MB/s]
pytorch_model-00001-of-00002.bin:  69% 6.88G/9.98G [00:29<00:13, 237MB/s]
pytorch_model-00001-of-00002.bin:  69% 6.91G/9.98G [00:30<00:12, 238MB/s]
pytorch_model-00001-of-00002.bin:  70% 6.94G/9.98G [00:30<00:12, 238MB/s]
pytorch_model-00001-of-00002.bin:  70% 6.97G/9.98G [00:30<00:12, 239MB/s]
pytorch_model-00001-of-00002.bin:  70% 7.00G/9.98G [00:30<00:12, 239MB/s]
pytorch_model-00001-of-00002.bin:  71% 7.04G/9.98G [00:30<00:12, 239MB/s]
pytorch_model-00001-of-00002.bin:  71% 7.07G/9.98G [00:30<00:12, 236MB/s]
pytorch_model-00001-of-00002.bin:  71% 7.10G/9.98G [00:30<00:12, 235MB/s]
pytorch_model-00001-of-00002.bin:  71% 7.13G/9.98G [00:31<00:12, 234MB/s]
pytorch_model-00001-of-00002.bin:  72% 7.16G/9.98G [00:31<00:12, 233MB/s]
pytorch_model-00001-of-00002.bin:  72% 7.19G/9.98G [00:31<00:12, 230MB/s]
pytorch_model-00001-of-00002.bin:  72% 7.22G/9.98G [00:31<00:12, 228MB/s]
pytorch_model-00001-of-00002.bin:  73% 7.26G/9.98G [00:31<00:11, 227MB/s]
pytorch_model-00001-of-00002.bin:  73% 7.29G/9.98G [00:31<00:11, 227MB/s]
pytorch_model-00001-of-00002.bin:  73% 7.32G/9.98G [00:31<00:11, 228MB/s]
pytorch_model-00001-of-00002.bin:  74% 7.35G/9.98G [00:31<00:11, 226MB/s]
pytorch_model-00001-of-00002.bin:  74% 7.38G/9.98G [00:32<00:11, 225MB/s]
pytorch_model-00001-of-00002.bin:  74% 7.41G/9.98G [00:32<00:11, 225MB/s]
pytorch_model-00001-of-00002.bin:  75% 7.44G/9.98G [00:32<00:11, 227MB/s]
pytorch_model-00001-of-00002.bin:  75% 7.48G/9.98G [00:32<00:11, 226MB/s]
pytorch_model-00001-of-00002.bin:  75% 7.51G/9.98G [00:32<00:10, 226MB/s]
pytorch_model-00001-of-00002.bin:  76% 7.54G/9.98G [00:32<00:10, 227MB/s]
pytorch_model-00001-of-00002.bin:  76% 7.57G/9.98G [00:32<00:10, 228MB/s]
pytorch_model-00001-of-00002.bin:  76% 7.60G/9.98G [00:33<00:10, 225MB/s]
pytorch_model-00001-of-00002.bin:  77% 7.63G/9.98G [00:33<00:10, 228MB/s]
pytorch_model-00001-of-00002.bin:  77% 7.67G/9.98G [00:33<00:10, 231MB/s]
pytorch_model-00001-of-00002.bin:  77% 7.70G/9.98G [00:33<00:09, 233MB/s]
pytorch_model-00001-of-00002.bin:  77% 7.73G/9.98G [00:33<00:09, 234MB/s]
pytorch_model-00001-of-00002.bin:  78% 7.76G/9.98G [00:33<00:09, 234MB/s]
pytorch_model-00001-of-00002.bin:  78% 7.79G/9.98G [00:33<00:09, 234MB/s]
pytorch_model-00001-of-00002.bin:  78% 7.82G/9.98G [00:34<00:09, 234MB/s]
pytorch_model-00001-of-00002.bin:  79% 7.85G/9.98G [00:34<00:09, 234MB/s]
pytorch_model-00001-of-00002.bin:  79% 7.89G/9.98G [00:34<00:08, 233MB/s]
pytorch_model-00001-of-00002.bin:  79% 7.92G/9.98G [00:34<00:08, 233MB/s]
pytorch_model-00001-of-00002.bin:  80% 7.95G/9.98G [00:34<00:08, 233MB/s]
pytorch_model-00001-of-00002.bin:  80% 7.98G/9.98G [00:34<00:08, 234MB/s]
pytorch_model-00001-of-00002.bin:  80% 8.01G/9.98G [00:34<00:08, 234MB/s]
pytorch_model-00001-of-00002.bin:  81% 8.04G/9.98G [00:34<00:08, 232MB/s]
pytorch_model-00001-of-00002.bin:  81% 8.07G/9.98G [00:35<00:08, 234MB/s]
pytorch_model-00001-of-00002.bin:  81% 8.11G/9.98G [00:35<00:07, 236MB/s]
pytorch_model-00001-of-00002.bin:  82% 8.14G/9.98G [00:35<00:07, 236MB/s]
pytorch_model-00001-of-00002.bin:  82% 8.17G/9.98G [00:35<00:07, 235MB/s]
pytorch_model-00001-of-00002.bin:  82% 8.20G/9.98G [00:35<00:07, 235MB/s]
pytorch_model-00001-of-00002.bin:  83% 8.23G/9.98G [00:35<00:07, 235MB/s]
pytorch_model-00001-of-00002.bin:  83% 8.26G/9.98G [00:35<00:07, 232MB/s]
pytorch_model-00001-of-00002.bin:  83% 8.29G/9.98G [00:36<00:07, 222MB/s]
pytorch_model-00001-of-00002.bin:  83% 8.33G/9.98G [00:36<00:07, 223MB/s]
pytorch_model-00001-of-00002.bin:  84% 8.36G/9.98G [00:36<00:07, 225MB/s]
pytorch_model-00001-of-00002.bin:  84% 8.39G/9.98G [00:36<00:07, 226MB/s]
pytorch_model-00001-of-00002.bin:  84% 8.42G/9.98G [00:36<00:06, 227MB/s]
pytorch_model-00001-of-00002.bin:  85% 8.45G/9.98G [00:36<00:06, 226MB/s]
pytorch_model-00001-of-00002.bin:  85% 8.48G/9.98G [00:36<00:06, 227MB/s]
pytorch_model-00001-of-00002.bin:  85% 8.51G/9.98G [00:37<00:06, 226MB/s]
pytorch_model-00001-of-00002.bin:  86% 8.55G/9.98G [00:37<00:06, 227MB/s]
pytorch_model-00001-of-00002.bin:  86% 8.58G/9.98G [00:37<00:06, 228MB/s]
pytorch_model-00001-of-00002.bin:  86% 8.61G/9.98G [00:37<00:06, 227MB/s]
pytorch_model-00001-of-00002.bin:  87% 8.64G/9.98G [00:37<00:05, 227MB/s]
pytorch_model-00001-of-00002.bin:  87% 8.67G/9.98G [00:37<00:05, 228MB/s]
pytorch_model-00001-of-00002.bin:  87% 8.70G/9.98G [00:37<00:05, 230MB/s]
pytorch_model-00001-of-00002.bin:  88% 8.73G/9.98G [00:38<00:05, 230MB/s]
pytorch_model-00001-of-00002.bin:  88% 8.77G/9.98G [00:38<00:05, 232MB/s]
pytorch_model-00001-of-00002.bin:  88% 8.80G/9.98G [00:38<00:05, 234MB/s]
pytorch_model-00001-of-00002.bin:  88% 8.83G/9.98G [00:38<00:04, 236MB/s]
pytorch_model-00001-of-00002.bin:  89% 8.86G/9.98G [00:38<00:04, 237MB/s]
pytorch_model-00001-of-00002.bin:  89% 8.89G/9.98G [00:38<00:04, 238MB/s]
pytorch_model-00001-of-00002.bin:  89% 8.92G/9.98G [00:38<00:04, 238MB/s]
pytorch_model-00001-of-00002.bin:  90% 8.95G/9.98G [00:38<00:04, 238MB/s]
pytorch_model-00001-of-00002.bin:  90% 8.99G/9.98G [00:39<00:04, 238MB/s]
pytorch_model-00001-of-00002.bin:  90% 9.02G/9.98G [00:39<00:04, 233MB/s]
pytorch_model-00001-of-00002.bin:  91% 9.05G/9.98G [00:39<00:03, 235MB/s]
pytorch_model-00001-of-00002.bin:  91% 9.08G/9.98G [00:39<00:03, 236MB/s]
pytorch_model-00001-of-00002.bin:  91% 9.11G/9.98G [00:39<00:03, 237MB/s]
pytorch_model-00001-of-00002.bin:  92% 9.14G/9.98G [00:39<00:05, 160MB/s]
pytorch_model-00001-of-00002.bin:  92% 9.18G/9.98G [00:40<00:04, 178MB/s]
pytorch_model-00001-of-00002.bin:  92% 9.21G/9.98G [00:40<00:04, 192MB/s]
pytorch_model-00001-of-00002.bin:  93% 9.24G/9.98G [00:40<00:03, 204MB/s]
pytorch_model-00001-of-00002.bin:  93% 9.27G/9.98G [00:40<00:03, 211MB/s]
pytorch_model-00001-of-00002.bin:  93% 9.30G/9.98G [00:40<00:03, 216MB/s]
pytorch_model-00001-of-00002.bin:  94% 9.33G/9.98G [00:40<00:02, 221MB/s]
pytorch_model-00001-of-00002.bin:  94% 9.36G/9.98G [00:40<00:02, 225MB/s]
pytorch_model-00001-of-00002.bin:  94% 9.40G/9.98G [00:41<00:02, 221MB/s]
pytorch_model-00001-of-00002.bin:  94% 9.43G/9.98G [00:41<00:02, 220MB/s]
pytorch_model-00001-of-00002.bin:  95% 9.46G/9.98G [00:41<00:02, 219MB/s]
pytorch_model-00001-of-00002.bin:  95% 9.49G/9.98G [00:41<00:02, 221MB/s]
pytorch_model-00001-of-00002.bin:  95% 9.52G/9.98G [00:41<00:02, 222MB/s]
pytorch_model-00001-of-00002.bin:  96% 9.55G/9.98G [00:41<00:01, 223MB/s]
pytorch_model-00001-of-00002.bin:  96% 9.58G/9.98G [00:41<00:01, 223MB/s]
pytorch_model-00001-of-00002.bin:  96% 9.62G/9.98G [00:42<00:01, 225MB/s]
pytorch_model-00001-of-00002.bin:  97% 9.65G/9.98G [00:42<00:01, 228MB/s]
pytorch_model-00001-of-00002.bin:  97% 9.68G/9.98G [00:42<00:01, 230MB/s]
pytorch_model-00001-of-00002.bin:  97% 9.71G/9.98G [00:42<00:01, 229MB/s]
pytorch_model-00001-of-00002.bin:  98% 9.74G/9.98G [00:42<00:01, 228MB/s]
pytorch_model-00001-of-00002.bin:  98% 9.77G/9.98G [00:42<00:00, 227MB/s]
pytorch_model-00001-of-00002.bin:  98% 9.80G/9.98G [00:42<00:00, 227MB/s]
pytorch_model-00001-of-00002.bin:  99% 9.84G/9.98G [00:42<00:00, 228MB/s]
pytorch_model-00001-of-00002.bin:  99% 9.87G/9.98G [00:43<00:00, 231MB/s]
pytorch_model-00001-of-00002.bin:  99% 9.90G/9.98G [00:43<00:00, 233MB/s]
pytorch_model-00001-of-00002.bin: 100% 9.93G/9.98G [00:43<00:00, 234MB/s]
pytorch_model-00001-of-00002.bin: 100% 9.98G/9.98G [00:43<00:00, 229MB/s]
Downloading shards:  50% 1/2 [00:43<00:43, 43.68s/it]
pytorch_model-00002-of-00002.bin:   0% 0.00/3.54G [00:00<?, ?B/s]
pytorch_model-00002-of-00002.bin:   1% 31.5M/3.54G [00:00<00:14, 238MB/s]
pytorch_model-00002-of-00002.bin:   2% 62.9M/3.54G [00:00<00:14, 237MB/s]
pytorch_model-00002-of-00002.bin:   3% 94.4M/3.54G [00:00<00:14, 237MB/s]
pytorch_model-00002-of-00002.bin:   4% 126M/3.54G [00:00<00:14, 237MB/s] 
pytorch_model-00002-of-00002.bin:   4% 157M/3.54G [00:00<00:14, 237MB/s]
pytorch_model-00002-of-00002.bin:   5% 189M/3.54G [00:00<00:14, 237MB/s]
pytorch_model-00002-of-00002.bin:   6% 220M/3.54G [00:00<00:14, 237MB/s]
pytorch_model-00002-of-00002.bin:   7% 252M/3.54G [00:01<00:13, 237MB/s]
pytorch_model-00002-of-00002.bin:   8% 283M/3.54G [00:01<00:13, 237MB/s]
pytorch_model-00002-of-00002.bin:   9% 315M/3.54G [00:01<00:13, 238MB/s]
pytorch_model-00002-of-00002.bin:  10% 346M/3.54G [00:01<00:13, 238MB/s]
pytorch_model-00002-of-00002.bin:  11% 377M/3.54G [00:01<00:13, 238MB/s]
pytorch_model-00002-of-00002.bin:  12% 409M/3.54G [00:01<00:13, 237MB/s]
pytorch_model-00002-of-00002.bin:  12% 440M/3.54G [00:01<00:13, 235MB/s]
pytorch_model-00002-of-00002.bin:  13% 472M/3.54G [00:01<00:13, 235MB/s]
pytorch_model-00002-of-00002.bin:  14% 503M/3.54G [00:02<00:12, 234MB/s]
pytorch_model-00002-of-00002.bin:  15% 535M/3.54G [00:02<00:12, 234MB/s]
pytorch_model-00002-of-00002.bin:  16% 566M/3.54G [00:02<00:12, 231MB/s]
pytorch_model-00002-of-00002.bin:  17% 598M/3.54G [00:02<00:12, 229MB/s]
pytorch_model-00002-of-00002.bin:  18% 629M/3.54G [00:02<00:12, 227MB/s]
pytorch_model-00002-of-00002.bin:  19% 661M/3.54G [00:02<00:12, 226MB/s]
pytorch_model-00002-of-00002.bin:  20% 692M/3.54G [00:02<00:12, 226MB/s]
pytorch_model-00002-of-00002.bin:  20% 724M/3.54G [00:03<00:12, 227MB/s]
pytorch_model-00002-of-00002.bin:  21% 755M/3.54G [00:03<00:12, 223MB/s]
pytorch_model-00002-of-00002.bin:  22% 786M/3.54G [00:03<00:12, 222MB/s]
pytorch_model-00002-of-00002.bin:  23% 818M/3.54G [00:03<00:12, 222MB/s]
pytorch_model-00002-of-00002.bin:  24% 849M/3.54G [00:03<00:12, 223MB/s]
pytorch_model-00002-of-00002.bin:  25% 881M/3.54G [00:03<00:11, 223MB/s]
pytorch_model-00002-of-00002.bin:  26% 912M/3.54G [00:03<00:11, 224MB/s]
pytorch_model-00002-of-00002.bin:  27% 944M/3.54G [00:04<00:11, 223MB/s]
pytorch_model-00002-of-00002.bin:  28% 975M/3.54G [00:04<00:11, 225MB/s]
pytorch_model-00002-of-00002.bin:  28% 1.01G/3.54G [00:04<00:11, 228MB/s]
pytorch_model-00002-of-00002.bin:  29% 1.04G/3.54G [00:04<00:10, 229MB/s]
pytorch_model-00002-of-00002.bin:  30% 1.07G/3.54G [00:04<00:10, 232MB/s]
pytorch_model-00002-of-00002.bin:  31% 1.10G/3.54G [00:04<00:10, 234MB/s]
pytorch_model-00002-of-00002.bin:  32% 1.13G/3.54G [00:04<00:10, 234MB/s]
pytorch_model-00002-of-00002.bin:  33% 1.16G/3.54G [00:05<00:10, 235MB/s]
pytorch_model-00002-of-00002.bin:  34% 1.20G/3.54G [00:05<00:09, 235MB/s]
pytorch_model-00002-of-00002.bin:  35% 1.23G/3.54G [00:05<00:09, 236MB/s]
pytorch_model-00002-of-00002.bin:  36% 1.26G/3.54G [00:05<00:09, 236MB/s]
pytorch_model-00002-of-00002.bin:  36% 1.29G/3.54G [00:05<00:09, 237MB/s]
pytorch_model-00002-of-00002.bin:  37% 1.32G/3.54G [00:05<00:09, 237MB/s]
pytorch_model-00002-of-00002.bin:  38% 1.35G/3.54G [00:05<00:09, 236MB/s]
pytorch_model-00002-of-00002.bin:  39% 1.38G/3.54G [00:05<00:09, 234MB/s]
pytorch_model-00002-of-00002.bin:  40% 1.42G/3.54G [00:06<00:09, 234MB/s]
pytorch_model-00002-of-00002.bin:  41% 1.45G/3.54G [00:06<00:08, 236MB/s]
pytorch_model-00002-of-00002.bin:  42% 1.48G/3.54G [00:06<00:08, 237MB/s]
pytorch_model-00002-of-00002.bin:  43% 1.51G/3.54G [00:06<00:08, 238MB/s]
pytorch_model-00002-of-00002.bin:  44% 1.54G/3.54G [00:06<00:08, 239MB/s]
pytorch_model-00002-of-00002.bin:  44% 1.57G/3.54G [00:06<00:08, 233MB/s]
pytorch_model-00002-of-00002.bin:  45% 1.60G/3.54G [00:06<00:08, 231MB/s]
pytorch_model-00002-of-00002.bin:  46% 1.64G/3.54G [00:07<00:08, 229MB/s]
pytorch_model-00002-of-00002.bin:  47% 1.67G/3.54G [00:07<00:08, 228MB/s]
pytorch_model-00002-of-00002.bin:  48% 1.70G/3.54G [00:07<00:08, 227MB/s]
pytorch_model-00002-of-00002.bin:  49% 1.73G/3.54G [00:07<00:07, 227MB/s]
pytorch_model-00002-of-00002.bin:  50% 1.76G/3.54G [00:07<00:07, 228MB/s]
pytorch_model-00002-of-00002.bin:  51% 1.79G/3.54G [00:07<00:07, 227MB/s]
pytorch_model-00002-of-00002.bin:  52% 1.82G/3.54G [00:07<00:07, 226MB/s]
pytorch_model-00002-of-00002.bin:  52% 1.86G/3.54G [00:08<00:07, 225MB/s]
pytorch_model-00002-of-00002.bin:  53% 1.89G/3.54G [00:08<00:07, 225MB/s]
pytorch_model-00002-of-00002.bin:  54% 1.92G/3.54G [00:08<00:07, 225MB/s]
pytorch_model-00002-of-00002.bin:  55% 1.95G/3.54G [00:08<00:07, 225MB/s]
pytorch_model-00002-of-00002.bin:  56% 1.98G/3.54G [00:08<00:06, 225MB/s]
pytorch_model-00002-of-00002.bin:  57% 2.01G/3.54G [00:08<00:06, 226MB/s]
pytorch_model-00002-of-00002.bin:  58% 2.04G/3.54G [00:08<00:06, 226MB/s]
pytorch_model-00002-of-00002.bin:  59% 2.08G/3.54G [00:08<00:06, 229MB/s]
pytorch_model-00002-of-00002.bin:  59% 2.11G/3.54G [00:09<00:06, 228MB/s]
pytorch_model-00002-of-00002.bin:  60% 2.14G/3.54G [00:09<00:06, 229MB/s]
pytorch_model-00002-of-00002.bin:  61% 2.17G/3.54G [00:09<00:05, 231MB/s]
pytorch_model-00002-of-00002.bin:  62% 2.20G/3.54G [00:09<00:05, 232MB/s]
pytorch_model-00002-of-00002.bin:  63% 2.23G/3.54G [00:09<00:05, 234MB/s]
pytorch_model-00002-of-00002.bin:  64% 2.26G/3.54G [00:09<00:05, 233MB/s]
pytorch_model-00002-of-00002.bin:  65% 2.30G/3.54G [00:09<00:05, 235MB/s]
pytorch_model-00002-of-00002.bin:  66% 2.33G/3.54G [00:10<00:05, 236MB/s]
pytorch_model-00002-of-00002.bin:  67% 2.36G/3.54G [00:10<00:05, 222MB/s]
pytorch_model-00002-of-00002.bin:  67% 2.39G/3.54G [00:10<00:05, 227MB/s]
pytorch_model-00002-of-00002.bin:  68% 2.42G/3.54G [00:10<00:04, 230MB/s]
pytorch_model-00002-of-00002.bin:  69% 2.45G/3.54G [00:10<00:04, 233MB/s]
pytorch_model-00002-of-00002.bin:  70% 2.49G/3.54G [00:10<00:04, 235MB/s]
pytorch_model-00002-of-00002.bin:  71% 2.52G/3.54G [00:10<00:04, 236MB/s]
pytorch_model-00002-of-00002.bin:  72% 2.55G/3.54G [00:11<00:04, 238MB/s]
pytorch_model-00002-of-00002.bin:  73% 2.58G/3.54G [00:11<00:04, 238MB/s]
pytorch_model-00002-of-00002.bin:  74% 2.61G/3.54G [00:11<00:03, 239MB/s]
pytorch_model-00002-of-00002.bin:  75% 2.64G/3.54G [00:11<00:03, 239MB/s]
pytorch_model-00002-of-00002.bin:  75% 2.67G/3.54G [00:11<00:03, 240MB/s]
pytorch_model-00002-of-00002.bin:  76% 2.71G/3.54G [00:11<00:03, 239MB/s]
pytorch_model-00002-of-00002.bin:  77% 2.74G/3.54G [00:11<00:03, 236MB/s]
pytorch_model-00002-of-00002.bin:  78% 2.77G/3.54G [00:11<00:03, 234MB/s]
pytorch_model-00002-of-00002.bin:  79% 2.80G/3.54G [00:12<00:03, 234MB/s]
pytorch_model-00002-of-00002.bin:  80% 2.83G/3.54G [00:12<00:03, 234MB/s]
pytorch_model-00002-of-00002.bin:  81% 2.86G/3.54G [00:12<00:02, 232MB/s]
pytorch_model-00002-of-00002.bin:  82% 2.89G/3.54G [00:12<00:02, 230MB/s]
pytorch_model-00002-of-00002.bin:  83% 2.93G/3.54G [00:12<00:02, 230MB/s]
pytorch_model-00002-of-00002.bin:  83% 2.96G/3.54G [00:12<00:02, 229MB/s]
pytorch_model-00002-of-00002.bin:  84% 2.99G/3.54G [00:12<00:02, 225MB/s]
pytorch_model-00002-of-00002.bin:  85% 3.02G/3.54G [00:13<00:02, 223MB/s]
pytorch_model-00002-of-00002.bin:  86% 3.05G/3.54G [00:13<00:02, 224MB/s]
pytorch_model-00002-of-00002.bin:  87% 3.08G/3.54G [00:13<00:02, 224MB/s]
pytorch_model-00002-of-00002.bin:  88% 3.11G/3.54G [00:13<00:01, 224MB/s]
pytorch_model-00002-of-00002.bin:  89% 3.15G/3.54G [00:13<00:01, 225MB/s]
pytorch_model-00002-of-00002.bin:  90% 3.18G/3.54G [00:13<00:01, 224MB/s]
pytorch_model-00002-of-00002.bin:  91% 3.21G/3.54G [00:13<00:01, 224MB/s]
pytorch_model-00002-of-00002.bin:  91% 3.24G/3.54G [00:14<00:01, 228MB/s]
pytorch_model-00002-of-00002.bin:  92% 3.27G/3.54G [00:14<00:01, 231MB/s]
pytorch_model-00002-of-00002.bin:  93% 3.30G/3.54G [00:14<00:01, 233MB/s]
pytorch_model-00002-of-00002.bin:  94% 3.33G/3.54G [00:14<00:00, 232MB/s]
pytorch_model-00002-of-00002.bin:  95% 3.37G/3.54G [00:14<00:00, 234MB/s]
pytorch_model-00002-of-00002.bin:  96% 3.40G/3.54G [00:14<00:00, 235MB/s]
pytorch_model-00002-of-00002.bin:  97% 3.43G/3.54G [00:14<00:00, 236MB/s]
pytorch_model-00002-of-00002.bin:  98% 3.46G/3.54G [00:14<00:00, 237MB/s]
pytorch_model-00002-of-00002.bin:  99% 3.49G/3.54G [00:15<00:00, 238MB/s]
pytorch_model-00002-of-00002.bin: 100% 3.54G/3.54G [00:15<00:00, 231MB/s]
Downloading shards: 100% 2/2 [00:59<00:00, 29.55s/it]
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
config.json: 100% 4.76k/4.76k [00:00<00:00, 26.3MB/s]
Loading checkpoint shards: 100% 2/2 [00:04<00:00,  2.15s/it]
generation_config.json: 100% 124/124 [00:00<00:00, 1.12MB/s]
preprocessor_config.json: 100% 316/316 [00:00<00:00, 2.97MB/s]
pytorch_model.bin: 100% 1.71G/1.71G [00:08<00:00, 192MB/s]
You are using a model of type llava to instantiate a model of type llava_llama. This is not supported for all configurations of models and can yield errors.
Loading checkpoint shards: 100% 2/2 [00:02<00:00,  1.20s/it]
WARNING:root:Some parameters are on the meta device device because they were offloaded to the cpu.
WARNING:root:Some parameters are on the meta device device because they were offloaded to the cpu.
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:515: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(
Traceback (most recent call last):
  File "/content/drive/My Drive/LLaVA/run_demo.py", line 31, in <module>
    eval_model(args)
  File "/content/drive/My Drive/LLaVA/llava/eval/run_llava.py", line 115, in eval_model
    output_ids = model.generate(
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/content/drive/My Drive/LLaVA/llava/model/language_model/llava_llama.py", line 138, in generate
    return super().generate(
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 1758, in generate
    result = self._sample(
  File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 2397, in _sample
    outputs = self(
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 166, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/content/drive/My Drive/LLaVA/llava/model/language_model/llava_llama.py", line 92, in forward
    return super().forward(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 1164, in forward
    outputs = self.model(
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 968, in forward
    layer_outputs = decoder_layer(
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 166, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 727, in forward
    hidden_states = self.mlp(hidden_states)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 166, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 216, in forward
    down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/activation.py", line 396, in forward
    return F.silu(input, inplace=self.inplace)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py", line 2102, in silu
    return torch._C._nn.silu(input)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 14.00 MiB. GPU ```

Screenshots:
![image](https://github.com/haotian-liu/LLaVA/assets/95133674/ec4090fd-c34e-44c4-8efc-bfa85a4d8a32)
![image](https://github.com/haotian-liu/LLaVA/assets/95133674/f8ff98d1-9f54-45d6-b622-8c8ade4b766d)
![image](https://github.com/haotian-liu/LLaVA/assets/95133674/70ef88e0-238d-418d-a773-e7ef6c360a3d)
anas-zafar commented 3 weeks ago

@lmcjrrg what GPU did you use to solve this?

lmcjrrg commented 3 weeks ago

@lmcjrrg你使用什么 GPU 来解决这个问题？

L4 can do it, the two demo fragments are executed separately