Closed gideonite closed 2 months ago
Please use this link: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/modulus/resources/modulus_datasets_cwa
Please note that you must download the image only via NGC CLI. Direct/wget download won't work.
Just ran the following command:
ngc registry resource download-version "nvidia/modulus/modulus_datasets_cwa:v1"
Resulting in a 467.8 GB
download which I am now waiting to complete. Note that this does not match what is documented as a 97.65 GB
dataset when compressed. I guess what is available online is no longer compressed? Specifically, I see a status bar as follows:
⠼ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • 0.7/467.8 GiB • Remaining: 1:22:16 • 101.6 MB/s • Elapsed: 0:00:09 • Total: 1 - Completed: 0 - Failed: 0
Hi Gideon. All seems well. Let us know if the download fails. 97GB is probably inaccurate.
From: Gideon Dresdner @.> Date: Tuesday, May 7, 2024 at 8:05 AM To: NVIDIA/modulus @.> Cc: Noah Brenowitz @.>, Mention @.> Subject: Re: [NVIDIA/modulus] 🐛[BUG]: CorrDiff dataset is unreadable when downloaded via wget (Issue #431)
Just ran the following command:
ngc registry resource download-version "nvidia/modulus/modulus_datasets_cwa:v1"
Resulting in a 467.8 GB download which I am now waiting to complete. Note that this does not match what is documented as a 97.65 GB dataset when compressed. I guess what is available online is no longer compressed? Specifically, I see a status bar as follows:
⠼ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • 0.7/467.8 GiB • Remaining: 1:22:16 • 101.6 MB/s • Elapsed: 0:00:09 • Total: 1 - Completed: 0 - Failed: 0
— Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA/modulus/issues/431#issuecomment-2098637215, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAKSRERSEQYVPT57WDYWL63ZBDURHAVCNFSM6AAAAABF64EDKCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJYGYZTOMRRGU. You are receiving this because you were mentioned.Message ID: @.***>
@gideonite you are on the right track. The compressed size is 467.8 GB
, as you see in the status bar. UI shows the compressed size inaccurately, and that's a known bug with NGC catalog.
Version
latest
On which installation method(s) does this occur?
No response
Describe the issue
Following the link on the CorrDiff README, e.g. this link, I selected the
wget
option and ran the command belowThe result was a 98G zip archive
When I tried to unzip, I got the following error
I did a quick search and the top results indicate one of two possibilities:
I tried downloading the dataset again (~ 1 hr) but same error.
Pinging you as we discussed. Please let me know if I can provide any more info @nbren12
Minimum reproducible example
No response
Relevant log output
No response
Environment details
No response