search

princeton-computational-imaging / SeeingThroughFog

MIT License
298 stars 59 forks source link

Can't fully download the dataset #28

Closed giho-sung closed 3 years ago

giho-sung commented 3 years ago

Hello! Thanks for your works.

I tried to download SeeingThroughFogCompressed from chrome or wget. However, the download was said to be done before downloading about 5% of the dataset. I have tried to download it a lot of times, but it didn't work. Is there any way to reliably download the dataset?

MarioBijelic commented 3 years ago

Hi giho-sung,

can you explain your problem in more detail? Do you have an error message? Typically the link gets replaced every 2-4 weeks due to technical reasons. Then it is necessary to register again to obtain a new link. Other than that the server capabilities should be sufficient enough to download core zip files within a day. Can you start downloading single zip files? For example the Lidar zip is one of the smallest.

Best, Mario

giho-sung commented 3 years ago

Hi! MarioBijelic,

These are the details of the error and what to do in the next step.

Detail: The downloads usually ended up without any error messages such as it worked well. But when I checked, the amount of data were 1 to 15GB of 500GB or 700GB. Sometimes, I can see an error message on the network error. Next: (1) downloading single zip files in the same network as you suggested (2) downloading directories in another network

I will post how it goes on later

Best regards, giho

connorlee77 commented 3 years ago

@giho-sung How long does download take for you?

My speeds are ~300-500 kbps for SeeingThroughFogCompressed.zip. I compared the download speed to NuScenes (at least 12 mbps) and it is much slower. I'm downloading from California, if that makes a difference. @MarioBijelic Any idea how long the data download usually takes?

giho-sung commented 3 years ago

@connorlee77 Now, speeds show ~50-1000 kbps. It has a lot of variation. Actually, speed is a second problem to me. Instability to stop downloading is a main problem as I mentioned.

connorlee77 commented 3 years ago

@giho-sung thanks for responding! Glad to know I'm not the only one with this issue.

MarioBijelic commented 3 years ago

We will look into the download speed. For now you can concentrate in downloading following folders cam_stereo_left, lidar_hdl_last_stereo (Projected Depth into image), lidar_hdl_64_last, lidar_hdl_64_strongest, gated0_rect, gated1_rect, gated2_rect and radar_targets.