search

partofthestars / LGI-PPGI-DB

LGI PPGI Face Video Database (Paper) Local Group Invariance for Heart Rate Estimation from Face Videos in the Wild
Creative Commons Attribution 4.0 International
41 stars 0 forks source link

Downloads either crash or give corrupted zip files #10

Open LeonidBeynenson opened 2 years ago

LeonidBeynenson commented 2 years ago

Hi all,

Unfortunately, all our attempts to download dataset archives failed. Each download was either crashed, or the result is not a valid zip archive. Sometimes the archive is almost valid -- 7zip allows to extract almost all files from it (but not all of them).

It seems that there is an error on the server side:
when we compared first 10,000,000 bytes of two versions of one archive, received from two different attempts to download, we found out that from some place two pieces of archive were swapped (size of each piece was about 3000 bytes, but it was sufficient to break the zip format).

We used to download on Linux curl, wget, aria2, Firefox, and Google Chrome. We used to extract archives 7zip (several versions) and Linux unzip utility.

Could you say how could we download the archives?

partofthestars commented 2 years ago

Hi, I just downloaded the id3.zip with firerfox. I can open the zip with no problem.

LeonidBeynenson commented 2 years ago

Hi @partofthestars,

I'm glad that you managed to download the LGI dataset files without errors. But we cannot do it.

I investigated the issue a bit more. See all the files in the attached archive lgi_logs_for_send.zip It has the following structure

lgi_logs_for_send
├── download.sh
├── extraction.log
├── file_sizes.txt
└── md5sum.txt

I downloaded several files from LGI dataset several times -- see the script download.sh.

As the result, I received several files, and the files downloaded from the same URI (e.g. id3.1.zip, id3.2.zip, id3.3.zip) have the same length -- see the file file_sizes.txt.

But all these files have different md5sum-s. I ran md5sum tool and received different results even for id3.1.zip, id3.2.zip, id3.3.zip. See the file md5sum.txt.

When I tried to make extraction, I received errors -- and there were different errors in different archives (see the file extraction.log -- it is the log of extraction id3.1.zip, id3.2.zip, id3.3.zip) Please, note that I used for extraction the latest version of 7zip, since it is able to extract archives partially. Other extractors (e.g. Linux unzip utility) failed to do it.

LeonidBeynenson commented 2 years ago

As far as I understand, if I downloaded one file several times and receive different md5sum-s, it means that it was downloaded incorrectly. And note that extraction of the different variants of the same archive gives different errors.

Please, also note that I managed to download other rPPG public datasets without errors. I have the issue with LGI dataset only.

Could you upload the dataset archives to another place too (Google Drive? OneDrive?)

partofthestars commented 2 years ago

I can only tell you again, if I download something from there, I can open the zip files normally and use the content. and no i can't upload the data somewhere else. I'm sorry for that.