ZeroDivisionError while loading pretrained models

[Gianluca124]

I was trying the Colab Notebook: in particular, I modified the script in order to generate an image for a list of prompts. At first everything went well, but then the runtime disconnected, probably because GPU usage limits have been exceeded (I have the free version of Colab). Since then, with the same exact code, I got a ZeroDivisionError when I try to generate the images, in particular when the Text Embedding model is loaded. The error is the following:

│ /usr/local/lib/python3.10/dist-packages/accelerate/big_modeling.py:466 in                        │
│ load_checkpoint_and_dispatch                                                                     │
│                                                                                                  │
│   463 │   │   │   "'sequential'."                                                                │
│   464 │   │   )                                                                                  │
│   465 │   if device_map != "sequential":                                                         │
│ ❱ 466 │   │   max_memory = get_balanced_memory(                                                  │
│   467 │   │   │   model,                                                                         │
│   468 │   │   │   max_memory=max_memory,                                                         │
│   469 │   │   │   no_split_module_classes=no_split_module_classes,                               │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/utils/modeling.py:526 in get_balanced_memory  │
│                                                                                                  │
│    523 │   module_sizes = {n: v for n, v in module_sizes.items() if n not in leaves}             │
│    524 │   # Once removed, leaves are the final modules.                                         │
│    525 │   leaves = [n for n in module_sizes if len([p for p in module_sizes if n == "" or p.st  │
│ ❱  526 │   mean_leaves = int(sum([module_sizes[n] for n in leaves]) / len(leaves))               │
│    527 │   buffer = int(1.25 * max(buffer, mean_leaves))                                         │
│    528 │   per_gpu += buffer 

ZeroDivisionError: division by zero

Has anyone idea how to fix this?

[shlyahin]

I have the same problem with original notebook.

[KapDEK]

Same

[adamdavidcole]

Same here

Fairly reckless temporary workaround: I patched modeling.py:526 to be: mean_leaves = int(sum([module_sizes[n] for n in leaves]) / ( len(leaves) or 1) )

Now I am able to generate images with the sample notebook

[Gianluca124]

Same here

Fairly reckless temporary workaround: I patched modeling.py:526 to be: mean_leaves = int(sum([module_sizes[n] for n in leaves]) / ( len(leaves) or 1) )

Now I am able to generate images with the sample notebook

It works, thank you

[sayedathar11]

Same here

Fairly reckless temporary workaround: I patched modeling.py:526 to be: mean_leaves = int(sum([module_sizes[n] for n in leaves]) / ( len(leaves) or 1) )

Now I am able to generate images with the sample notebook

@adamdavidcole @Gianluca124 can you please help me with how to implement this patch in google colab ?

[Gianluca124]

Same here Fairly reckless temporary workaround: I patched modeling.py:526 to be: mean_leaves = int(sum([module_sizes[n] for n in leaves]) / ( len(leaves) or 1) ) Now I am able to generate images with the sample notebook

@adamdavidcole @Gianluca124 can you please help me with how to implement this patch in google colab ?

In my case I forked the accelerate repository here on GitHub, I made the modification and then loaded on Colab my repository with the correction suggested by @adamdavidcole

[sayedathar11]

here

Can you please help me with how to fork the accelerate repository in colab environment with steps. Thank You.

[Gianluca124]

here

Can you please help me with how to fork the accelerate repository in colab environment with steps. Thank You.

1) Go to https://github.com/huggingface/accelerate and fork the repository 2) Now go to the forked repository (it should be something like https://github.com/{your_username}/accelerate) and made the modification inside the https://github.com/{your_username}/accelerate/blob/main/src/accelerate/utils/modeling.py script 3) In Colab, instead of installing the original accelerate package you use !pip install git+https://github.com/{your_username}/accelerate

The rest of the notebook stays the same

[sayedathar11]

here

Can you please help me with how to fork the accelerate repository in colab environment with steps. Thank You.

1. Go to https://github.com/huggingface/accelerate and fork the repository
2. Now go to the forked repository (it should be something like https://github.com/{your_username}/accelerate) and made the modification inside the https://github.com/{your_username}/accelerate/blob/main/src/accelerate/utils/modeling.py script
3. In Colab, instead of installing the original accelerate package you use !pip install git+https://github.com/{your_username}/accelerate

The rest of the notebook stays the same

Thanks alot will check this

[shlyahin]

Same here

Fairly reckless temporary workaround: I patched modeling.py:526 to be: mean_leaves = int(sum([module_sizes[n] for n in leaves]) / ( len(leaves) or 1) )

Now I am able to generate images with the sample notebook

Thank you, it works

[sayedathar11]

here

Can you please help me with how to fork the accelerate repository in colab environment with steps. Thank You.

1. Go to https://github.com/huggingface/accelerate and fork the repository
2. Now go to the forked repository (it should be something like https://github.com/{your_username}/accelerate) and made the modification inside the https://github.com/{your_username}/accelerate/blob/main/src/accelerate/utils/modeling.py script
3. In Colab, instead of installing the original accelerate package you use !pip install git+https://github.com/{your_username}/accelerate

The rest of the notebook stays the same

Thanks alot @Gianluca124 @adamdavidcole I was able to run the code and generate images :)

[ximo1984]

It's works for me with this adjust. Thank you. But now is in line 536.