3dem / model-angelo

Automatic atomic model building program for cryo-EM maps
MIT License
116 stars 18 forks source link

Checksum of downloaded file .../.../nucleotides_no_seq.zip does not match expected checksum. #81

Closed t00shadow closed 11 months ago

t00shadow commented 11 months ago

issue goes out when: Setting up bundle with name: nucleotides_no_seq for the first time.

The download is disrupted beacause of wrong checksum.

jamaliki commented 11 months ago

Hi,

Could you please detail what steps you went through to get this error? Did you just use the installation script?

t00shadow commented 11 months ago

Hi Jamali,

I just followed the installation steps in my Win11 Ubuntu sub-system.

After the installation, model_angelo build -h works well.

Then I try to use model_angelo build_no_seq -v XXXXXX -o XXXXXX, it start download a package nucleotides_no_seq.zip...

Setting up bundle with name: nucleotides_no_seq for the first time.
 38%|██████████████████████████████████████████▊                                                                     | 635M/1.62G [29:55<48:23, 371kB/s]

When the file is downloaded about 635M, the error comes out showing:

Checking checksum of downloaded file.
2023-10-13 at 16:59:46 | ERROR | Error in ModelAngelo
Traceback (most recent call last):
  File "/home/qiboxu/Software/model-angelo/model_angelo/apps/build_no_seq.py", line 129, in main
    model_bundle_path = download_and_install_model(
  File "/home/qiboxu/Software/model-angelo/model_angelo/utils/torch_utils.py", line 468, in download_and_install_model
    raise RuntimeError(
RuntimeError: Checksum of downloaded file /home/qiboxu/.cache/torch/hub/checkpoints/model_angelo_v1.0/nucleotides_no_seq.zip does not match expected checksum. Expected: 4fcbeba1fe469943d51e5cbb492c072e, got: 95bdc436618b454c3773ede99c31cbfa

I guess may the downloaded file is changed but the checksum is not.

jamaliki commented 11 months ago

Hmm, but the weights have stayed the same and I just checked that the checksum is also what it was before. Does your download get interrupted at 635M? The full file is around 1.7GB. Could you try downloading the weights by running the following command?

model_angelo setup_weights --bundle-name nucleotides_no_seq
t00shadow commented 11 months ago

I tried and it works!!!

btw, the speed is much more fast (10M/s) than the first time (300k/s) - don't know whether it is due to my network

t00shadow commented 11 months ago

Hey Jamali,

After that I started a job as: model_angelo build_no_seq -v ./TEST/emd_13112/emd_13112.map -o ./TEST/emd_13112/

but it truns out ERROR very soon as:

--------------------- Initial C-alpha prediction ---------------------
2023-10-16 at 10:49:54 | ERROR | Error in ModelAngelo
Traceback (most recent call last):
  File "/home/qiboxu/Software/model-angelo/model_angelo/apps/build_no_seq.py", line 174, in main
    ca_cif_path = c_alpha_infer(ca_infer_args)
  File "/home/qiboxu/Software/model-angelo/model_angelo/c_alpha/inference.py", line 182, in infer
    mask_grid = np.ones_like(grid_np)
UnboundLocalError: local variable 'grid_np' referenced before assignment

Could you know what is wrong? Thank you !

(The log file is attached.) model_angelo.log

jamaliki commented 11 months ago

Glad the download worked! This problem is strange, we have fixed this a long time ago. Could you run

model_angelo --version

Please?

Also, you could try the following in your model_angelo conda environment to upgrade the code, you don't have to reinstall the weights:

pip install --upgrade -e git+https://github.com/3dem/model-angelo.git#egg=model-angelo
t00shadow commented 11 months ago

Hey Jamali,

I've run model_angelo --version and the result is model-angelo-1.0.6.

And I also upgraded the code, but when I try to do the prediction, same issue appears.

One tip: My friend met the same issue as I in the first day. But in the 2nd day he successfully downloaded the weight by the original way model_angelo build_no_seq -v XXXXXX -o XXXXXX and the download speed is very fast. My guess is that the first failure is due to the long download time? (just guessing and giving a data point)

t00shadow commented 11 months ago

another data point:

My friend's is model_angelo 1.0.1

jamaliki commented 11 months ago

Ok thank you for the extra information. I'm very sorry that I was not able to figure out the issue before. The problem is that currently ModelAngelo does not recognize maps that do not end with .mrc. You can change the filename just to end with .mrc and it will work fine.

I realize this is unfortunate and I will push a fix for this today.

Sorry again.

t00shadow commented 11 months ago

Oh thank you so much!!

jamaliki commented 11 months ago

Does it work now?

t00shadow commented 11 months ago

Another error comes out...

---------------------------- ModelAngelo -----------------------------
By Kiarash Jamali, Scheres Group, MRC Laboratory of Molecular Biology
--------------------- Initial C-alpha prediction ---------------------
  0%|                                                                                          | 0/5832 [00:00<?, ?it/s]
Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory
Aborted
t00shadow commented 11 months ago

I searched online for the lib file but it looks complicated///

t00shadow commented 11 months ago
(model_angelo) qiboxu@OMEN-WINDOWS-10:~/Software/model-angelo$ sudo find /usr/ -name 'libcuda.so.*'
/usr/lib/wsl/lib/libcuda.so.1
/usr/lib/wsl/lib/libcuda.so.1.1
/usr/lib/wsl/drivers/nvhdc.inf_amd64_21fb5c984e3a7ddf/libcuda.so.1.1
t00shadow commented 11 months ago

Hi Jamali,

It works after I add this in the .bashrc:

export LD_LIBRARY_PATH=/usr/lib/wsl/lib:$LD_LIBRARY_PATH

Thank you for all your help!

jamaliki commented 11 months ago

That is excellent! Thank you for your patience. The libcuda error is related to weird driver issues that might be happening in WSL, great that you were able to solve it.

t00shadow commented 11 months ago

Thank you for your kindly helping!

jiahao-xie2021 commented 6 months ago

Hi, @jamaliki I'm currently working with model-angelo v1.0.12 and have encountered a checksum issue similar to the one @t00shadow reported, accompanied by a notably slow download speed of approximately 10kb/s. The error message I received was:

RuntimeError: Checksum of downloaded file /opt/ohpc/pub/apps/model_angelo_weights/hub/checkpoints/model_angelo_v1.0/nucleotides.zip does not match expected checksum. Expected: ce568d75b24f8a4654f1fa2f7a89cb27, got: ed3419aa9152f7ab251cd2805b705d7b

Following your advice, I attempted to use the command:

model_angelo setup_weights --bundle-name nucleotides

to separately download the bundle. Unfortunately, I encountered the same problem again:

[root@desktop01 relion]# model_angelo setup_weights --bundle-name nucleotides Please make sure you have set the environment variable TORCH_HOME to a suitable directory, visible to all relevant users! Setting up bundle with name: nucleotides for the first time. 0%|▌ | 8.56M/1.98G [09:20<36:46:17, 16.0kB/s] Checking checksum of downloaded file. Traceback (most recent call last): File "/opt/ohpc/pub/apps/anaconda3/envs/model_angelo/bin/model_angelo", line 8, in sys.exit(main()) File "/opt/ohpc/pub/apps/anaconda3/envs/model_angelo/lib/python3.10/site-packages/model_angelo/main.py", line 58, in main args.func(args) File "/opt/ohpc/pub/apps/anaconda3/envs/model_angelo/lib/python3.10/site-packages/model_angelo/utils/setup_weights.py", line 28, in main model_dst = download_and_install_model(parsed_args.bundle_name) File "/opt/ohpc/pub/apps/anaconda3/envs/model_angelo/lib/python3.10/site-packages/model_angelo/utils/torch_utils.py", line 468, in download_and_install_model raise RuntimeError( RuntimeError: Checksum of downloaded file /opt/ohpc/pub/apps/model_angelo_weights/hub/checkpoints/model_angelo_v1.0/nucleotides.zip does not match expected checksum. Expected: ce568d75b24f8a4654f1fa2f7a89cb27, got: 723d47c02f228ca64baff9c6f1378693

I'm at a loss for how to resolve this issue and would greatly appreciate any further advice or steps I could take to successfully download and setup the required bundle.

Thank you for your time and assistance.

Jiahao

t00shadow commented 6 months ago

Hey Jiahao,

I remember my former issue was probably due to my local status of network(the download speed is super slow). Please check your local network and see whether it is also the reason of your issue.