Downloading get stuck in infinite loop

[jarimustonen]

When trying to download the models, I get stuck in some infinite loop.

This repeats once every second until I terminate the program.

Environment:

• pyllama version: commit 321d475f01c88e179c8a30d68b5281e2caca5b07 (HEAD -> main, tag: v0.0.9, origin/main, origin/HEAD)
• OS: macOS 13.2.1
• Hardware: Apple M1 Max
• After installing the following packages were installed manually:
  • Transformers
  • This was missing from requirements.txt
  • Version 4.27.3
  • Command used pip install transformers
  • Itree
  • Was incompatible with M1 architecture
  • Work-around instructions: https://pypi.org/project/py-itree/
  • Command used: pip uninstall py-itree ; pip install https://github.com/juncongmoo/itree/archive/refs/tags/tag-bf9f3aada064acf3ce4db6fc58ed2e744caee0a3.tar.gz

[elven2016]

I have this issue too，but the os is WSL2-ubuntu

[Macavity77]

I have the same issue here. @juncongmoo can you help take a look?

[juncongmoo]

I wish I could reproduce the error above but my run always succeeds. Maybe it is because I don't have an M1 Mac book? Or can anybody create a colab so that I can reproduce the error?

[jarimustonen]

Is there a way to add more logging?

@elven2016, what hardware are you using?

@juncongmoo, what is colab and how can I create one?

[elven2016]

@jarimustonen HP Victus laptop， win11

[XiangkunFang]

Same problems here. I'm using the M1 chip Macbook Pro, and the downloading was in an infinite loop. I check the size of the file downloaded, and it changes between 0b to 200b, so basically the downloading procedure was just looping.

[synchroazel]

Same problem here on a Macbook Pro 14" with M1 Pro 🙋

[sskorol]

The problem on Mac is with the underlying download script (associative arrays and nested loop). It's Linux-oriented and should be adopted to MacOS. TBH, I quit trying to fix different issues on Mac last weekend. It was too time-consuming and disappointing. So I stopped and switched back to Linux.

[mindon]

under MacOS you need to brew install md5sha1sum to get md5sum to pass the verify

[jarimustonen]

under MacOS you need to brew install md5sha1sum to get md5sum to pass the verify

I tested this and this won't work.

[sskorol]

Because it's not the root cause. It never enters this loop.

[hinofafa]

under MacOS you need to brew install md5sha1sum to get md5sum to pass the verify

This is worked for M2 Pro too, I also stuck in infinite loop before installation.

Before:

After:

[yMayanand]

I am running on colab with gpu and downloading is stuck in infinite loop. Any solutions to solve this problem?