search

sedenion / OpenModMan

Open Mod Manager - Open source and generic Mod ("Modifications") manager.
GNU General Public License v3.0
84 stars 5 forks source link

Checksum mismatch with network repo on 1.2 #55

Closed deephack1982 closed 6 months ago

deephack1982 commented 6 months ago

Trying out the new 1.2 version and it looks really nice. I have an issue though with my network repo. I created all my own livery mods using the built in "Mod Pack Editor" feature. I then create a repo OMX file using the "Repository Editor" tool. I do this with local copies of all the mods but I then add custom download links for each pointing to a GCP bucket with public access where I store them. I can use these links manually without issue but if I try to download these mods from the "Network Library" tab I always get the error message that the checksum does not match. In the debug log I see

[21:04:19] X ModHub[DCS].ModChan[DCS].NetPack[25_Squadron_v1.0].finalizeDownload: downloaded data checksum mismatch the reference

sedenion commented 6 months ago

I don't know what exactly is a GCP bucket, but since it seem to be another "cloud" interface from google, I suspect it prevent direct download of raw data. Can you give an example of those "link" you attempt to use for Mod download ?

deephack1982 commented 6 months ago

Ah sorry should have explained properly. Google Cloud Platform and a bucket is simply a block storage account. They are normally used for things like web assets as part of websites and such. They are designed to be publically accessible.

My repo file is here https://github.com/virtualairforce/mod-repo/raw/main/VAF.omx

One of the mod links is here but you'll see them all in the omx file of course https://storage.googleapis.com/new_vaf_mods/25_Squadron_v1.0.ozp

sedenion commented 6 months ago

The checksum of the file I download from your provided URL is 14125a6edbc1f212 while the reference in your repository indicate 6829b2b1627c0762 indeed they are not the same. The most probable reasons are either Google altered the file on its side, or you altered the file after having referenced it in the repository definition, so the repository definition kept an old checksum.

Check by yourself if the file you download from google have the same checksum the one you used locally to create the repository... within the repository, the file checksum is stored in the 'xxhsum' attribute of the corresponding reference entry... simply create a dummy repository file to verify computed checksums are consistents.

Anywat this tells me the Repository Editor lack an option to re-check and update files checksum...

deephack1982 commented 6 months ago

Ah thanks so much for looking. So I guess I should re-download the mods and make the repo omx file again then I'll get updated checksums.

sedenion commented 6 months ago

I would be rather worry about how and why Google may alter the files... checksum is here to very file integrity and that there is no "fraud" so to speak... In your case, something explicitely tells you that the file you downloaded is not the same the file used to create the repository reference, so I suggest you investigate a bit to understand what exactly is happening, because it is not "normal".

deephack1982 commented 6 months ago

For sure I understand the implication. I just want to get something that works now and then I'll work backwards from there to fix the core issue. I'll report back when I'm done with my testing.

deephack1982 commented 6 months ago

Okay something weird is going on still. I've deleted all my local copies of the mods. Directly downloaded them all from the GCP bucket and generated a new omx file based on these downloaded files. The checksums generated match exactly the ones in my previous copy of the omx file and I still get a checksum error.

I also tried your approach of creating a dummy repo file just to get the checksums and they also match so it doesn't look like Google is making any changes to the files. I now have 3 repo omx files all with the same checksum.

deephack1982 commented 6 months ago

Okay got it. I don't understand why but Google offers two end points for the files an API end point and a normal download link. I switched from using the normal download link to the API link and now it all works. This suggests that my browser somehow handles the other end point differently than OMM does. At the end of the day it doesn't matter. As long as people use the storage.googleapis.com links then everything will work with OMM. This is definitely not an OMM bug.

sedenion commented 6 months ago

Ok, I found the problem and this is because of google cloud API refusing raw download in some way. While download works fine from web browser, google is rejecting the Open Mod Man request (due to wrong header referrer or something like that), so what Open Mod Man actually receive is not the file, but 170ko of html garbage from Google API.

Here is an excerp (the first bytes): <!doctype html><html lang="en-US" dir="ltr"><head><base href="https://accounts.google.com/v3/signin/"><meta name="referrer" content="origin"><style data-href="https://www.gstatic.com/

I will try to find out why the HTTP request is rejected, but I cannot guarantee any success.