search

godotengine / godot-asset-library

PHP frontend for Godot Engine's asset library
https://godotengine.org/asset-library
MIT License
291 stars 86 forks source link

Sha256 hash error on some godot assets #207

Closed lungdart closed 4 years ago

lungdart commented 4 years ago

Issue

All godot demo assets I've attempted to download have bad sha256 sums. This causes download failures in the asset library in the godot engine.

Expected behaviour

Sha256 hash sums recorded in the library and the actual hash sum of the file should match, enabling use of the demo asset

Environment

Reproduction

  1. Download 2D Physics Platformer Demo (RigidBody) in the asset manager.
  2. Observe the error 
    Asset Download Error:
Bad download hash, assuming file has been tampered with.
Expected: c1b5af9d8001a68a79ff51a0719ef64fadcfacd7e7243f81ee9fb9714b547358
Got: d99cb5081ee6807f3c1b878368508a7a5e6f361f35e2fb9e205aa12590f6ce36
  3. Goto the official asset library page for this demo (https://godotengine.org/asset-library/asset/119) and observe the hash sum is the same as the above expected result: 
    Sha256 Hash: c1b5af9d8001a68a79ff51a0719ef64fadcfacd7e7243f81ee9fb9714b547358
  4. Download the file and use sha256sum to evaluate the hash manually, and observe it's the same as the above got result: 
    $ sha256sum ./2d_physics_platformer.zip 
d99cb5081ee6807f3c1b878368508a7a5e6f361f35e2fb9e205aa12590f6ce36  ./2d_physics_platformer.zip
Calinou commented 4 years ago

Sha256 hash sums recorded in the library and the actual hash sum of the file should match, enabling use of the demo asset

Unfortunately, GitHub doesn't guarantee that automatically ZIP archives will be stable over time, even if you specify a tag or commit hash. This means their checksums can change unexpectedly. There's nothing you can do about it except require asset developers to manually upload ZIP archives, which is tedious.

We'd like to keep the effort required for asset authors as low as possible, so I think the only solution is to remove the hash verification system entirely. We can do this by sending an empty hash in the API, as the editor will skip hash verification in this case. Corrupted downloads aren't very common in the first place anyway, and even if they do occur, we can add something in the editor to automatically retry failed downloads.

The other upside is that asset library moderators will no longer have to compute the asset's checksum manually every time a new version is uploaded, which decreases the maintenance burden.

akien-mga commented 4 years ago

@Calinou In my experience though ZIP archives are stable if you use a stable reference (commit hash, tag that doesn't get force deleted by the repo owner).

Here it's likely just that the wrong sha256sum were pasted when validating those assets.

lungdart commented 4 years ago

@Calinou Is there an override option to install even if the hash check fails?

@akien-mga This affects every asset I've attempted download after a fresh install. I have not tried every asset though. I just installed Godot for the first time this morning

bojidar-bg commented 4 years ago

Something happened when updating the demo projects assets, probably the hashsums were not valid when automatically accepting the edits.

Looking into it now..

Calinou commented 4 years ago

@akien-mga This is probably valid in the short term, but I wonder how well that holds up for assets that were uploaded more than a year ago. If GitHub updates the ZIP library they use internally, it's technically enough to create ZIPs that have a different checksum (even if the individual files are identical). I should try to write a script to compare all assets' hashes and see how many of them are still valid to verify this.

Is there an override option to install even if the hash check fails?

No, there isn't :slightly_frowning_face:

lungdart commented 4 years ago

@Calinou I would propose a launch flag to override hash checks.

./Godot-v3.2.1-stable_x11.64 --ignore-hash-checks

Calinou commented 4 years ago

@lungdart A command-line argument just for this would be overkill. You can still download assets from the Web interface and install them manually in the meantime.

bojidar-bg commented 4 years ago

Alright, I manually edited all the assets to have correct hashsums. Now validating that all download properly.

bojidar-bg commented 4 years ago

All the demo projects seem to download correctly. Closing as fixed, please comment if there are any undownloadable demo projects left.

Federico-Ciuffardi commented 4 years ago

Hi this is also happening with the following assets:

bojidar-bg commented 4 years ago

Fixed those as well -- not sure why it happened.

I'm going to add a way for people with permissions to accept edits to manually change the hashes, so that it would not require directly changing the DB :smiley: .