Android downloader - Hash mismatch error

[lent1-andrealphusgames]

Hi,

We've implemented the downloader (android) for some months now.

We're experiencing mismatch error during the download process. It seems to be random. We're building weekly versions of our game. Some builds are working perfectly, others leads to the hash mismatch issue.

Here's the relevant data I found (I've altered the logs output and I might missed some data).

/base/update/log.txt:

Blockfile delete /data/user/0/com.andrealphusgames.loveandsexsecondbase_downloader_patreon.weekly/files/base/update/cbe085aaa3971910103e94c46d2de16022d259c9b42d4a2145172bca8d092bbe.rpu.
download_block_file: /data/user/0/com.andrealphusgames.loveandsexsecondbase_downloader_patreon.weekly/files/base/update/cfced82334cdc023e6472557b84915a1d0603017fe9412e932fbec4bb8e2ac55.rpu
cfced82334cdc023e6472557b84915a1d0603017fe9412e932fbec4bb8e2ac55.rpu (15, 2097152)
  -> /data/user/0/com.andrealphusgames.loveandsexsecondbase_downloader_patreon.weekly/files/base/game/music/roa_music/juice.ogg (0, 2097152) 
Hash mismatch on cfced82334cdc023e6472557b84915a1d0603017fe9412e932fbec4bb8e2ac55.rpu offset 15 size 2097152. expected: e9eff49f0e981e157b7a2ce299457737d59ebf68370c4997217a145e0eb55e29 actual: c83ad8a5a49e892ac50d08f747e318a2cb74bfa5a858f54fb97f98de2291283f.
Traceback (most recent call last):
  File "game/updater_module/updater_ren.py", line 446, in run
  File "game/updater_module/updater_ren.py", line 585, in update
  File "game/updater_module/updater_ren.py", line 734, in rpu_update
  File "renpy/update/update.py", line 187, in update
  File "renpy/update/update.py", line 578, in execute_plan
  File "renpy/update/update.py", line 553, in execute_file_plan
renpy.update.update.UpdateError: Hash mismatch on cfced82334cdc023e6472557b84915a1d0603017fe9412e932fbec4bb8e2ac55.rpu offset 15 size 2097152. expected: e9eff49f0e981e157b7a2ce299457737d59ebf68370c4997217a145e0eb55e29 actual: c83ad8a5a49e892ac50d08f747e318a2cb74bfa5a858f54fb97f98de2291283f.

From gameonly.files.rpu (json converted to yaml because I prefer reading yaml):

files:
  - name: game/music/roa_music/juice.ogg
    segments:
      - offset: 0
        size: 2097152
        hash: e9eff49f0e981e157b7a2ce299457737d59ebf68370c4997217a145e0eb55e29
        compressed: 0
      - offset: 2097152
        size: 1358508
        hash: 4752c835b2851e9147a411d36161fa51ed1cfb5ad5d058913c820741fb30dd3c
        compressed: 0
blocks:
  - name: cfced82334cdc023e6472557b84915a1d0603017fe9412e932fbec4bb8e2ac55.rpu
    segments:
      - offset: 15
        size: 2097152
        hash: e9eff49f0e981e157b7a2ce299457737d59ebf68370c4997217a145e0eb55e29
        compressed: 0
      - offset: 2097167
        size: 1358508
        hash: 4752c835b2851e9147a411d36161fa51ed1cfb5ad5d058913c820741fb30dd3c
        compressed: 0

Original file size match with segments size:

ls -l juice.ogg 
    -rw-r--r-- 1 lent1 lent1 3455660 févr.  8  2023 juice.ogg

I retrieved the cfced8... rpu file from our server and manually split it regarding the offset/size provided in gameonly.files.rpu. Hash didn't match, and resulting ogg file is unplayable.

So, I manually built an rpu file for this only file, segments and their hash were the same as the ones in gameonly.files.rpu.

files:
  - name: game/music/roa_music/juice.ogg
    segments:
      - offset: 0
        size: 2097152
        hash: e9eff49f0e981e157b7a2ce299457737d59ebf68370c4997217a145e0eb55e29
        compressed: 0
      - offset: 2097152
        size: 1358508
        hash: 4752c835b2851e9147a411d36161fa51ed1cfb5ad5d058913c820741fb30dd3c
        compressed: 0
blocks:
  - name: cfced82334cdc023e6472557b84915a1d0603017fe9412e932fbec4bb8e2ac55.rpu
    segments:
      - offset: 15
        size: 2097152
        hash: e9eff49f0e981e157b7a2ce299457737d59ebf68370c4997217a145e0eb55e29
        compressed: 0
      - offset: 2097167
        size: 1358508
        hash: 4752c835b2851e9147a411d36161fa51ed1cfb5ad5d058913c820741fb30dd3c
        compressed: 0

Using the previous way to split cfced82334cd....rpu file, I managed to retrieve an audible ogg.

I checked system/hardware/anything not related to game engine/code to see if I can find something, but I didn't find anything relevant.

To build our games, we're using a docker image to ensure there's no leftover somewhere.

We're pushing the built archives/directories with rsync, so, it's (should be) unlikely there's a corruption during copy between build servers and release servers.

I don't know where to search now. If you have any insights/tips I'll take it.

If you need more data, I'll be happy to provide them.

Thank for your help

[renpytom]

Does the problem occur outside of docker?

[lent1-andrealphusgames]

I don't know, we're only building our releases in docker. Some releases don't have issues, others have.

If you want/need I can give access to our docker image

I'm registered on your discord server if you want.

[renpytom]

I don't support docker builds - if they break, you're responsible for them. I don't know Docker well enough to debug issues in the scripts you created.

If you can repeat the issue with desktop Ren'Py, then I'll address them.

[renpytom]

And for a bit more context - building an update and getting it on the server involves copying files around, such that the docker scripts could be the cause of the problem. I want to eliminate them.

[lent1-andrealphusgames]

I don't support docker builds - if they break, you're responsible for them. I don't know Docker well enough to debug issues in the scripts you created.

If you can repeat the issue with desktop Ren'Py, then I'll address them.

Understood. Thanks for your time. I'll close this issue

[renpytom]

Sure - please reopen it if you can repeat the problem on desktop Ren'Py.

[renpytom]

(or even scripted Ren'Py, outside of the docker environment.)