update_all.sh from releases page - on the very first run it uses old code which reduces verbosity and timeouts

[movisman]

Hey @theypsilon - I figured a new issue for this would be the best way forward. It was mentioned as point 3 here: https://github.com/atrac17/MiSTer_MAME_SCRIPTS/issues/33

I know how to resolve the issue, I think there are one or two solutions to the problem (maybe you can think of something better though!).

To summarise the issue, I noticed when update_all.sh is run for the first time only, the verbosity when downloading MAME or HBMAME files is suppressed somewhat. I also felt that timeouts were lower than they should be.

However, on subsequent runs of the update_all.sh script, timeouts appear extended, and also the verbosity when downloading MAME or HBMAME files is much nicer and shows progress.

As an example, on the VERY first run of update_all.sh (after downloading the release here on GitHub), the output is like this:

MRA: /media/fat/_Arcade/mra_backup/_alternatives/_Crazy Kong/Crazy Kong Part II -1 - HBMame.mra
ZIP: ckong.zip (Ver 0217)

MRA: /media/fat/_Arcade/mra_backup/_alternatives/_Donkey Kong/Donkey Kong (2600 graphics) - HBMame.mra
ZIP: dkong.zip (Ver 0217)

However, on subsequent runs of the script, the output is better, more like this:

MRA: /media/fat/_Arcade/_alternatives/_Donkey Kong/Donkey Kong (2600 graphics) - HBMame.mra
ZIP: dkong.zip (Ver 0217)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 69  297k   69  207k    0     0   107k      0  0:00:02  0:00:01  0:00:01  198k
100  297k  100  297k    0     0   152k      0  0:00:01  0:00:01 --:--:--  276k

MRA: /media/fat/_Arcade/_alternatives/_Final Fight/Final Fight Anniversary Edition - HBMame.mra
ZIP: ffight.zip (Ver 0217)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  5 8405k    5  431k    0     0   206k      0  0:00:40  0:00:02  0:00:38  401k
 36 8405k   36 3087k    0     0  1022k      0  0:00:08  0:00:03  0:00:05 1539k
 88 8405k   88 7407k    0     0  1846k      0  0:00:04  0:00:04 --:--:-- 2470k
100 8405k  100 8405k    0     0  2002k      0  0:00:04  0:00:04 --:--:-- 2639k

Timeouts also appear to be extended.

Anyways, at first I had no idea why this was occurring, and although i'm no coder I started looking at the script files, but I couldn't see anything initially.

In the end I wondered if the initial update_all.sh could be responsible.

It seems to be the case.

The update_all.sh in the releases area on GitHub: https://github.com/theypsilon/Update_All_MiSTer/releases/tag/1.3

When unzipped, it is dated 14th June 2020.

This release is what the majority of users will download when they want to use update_all.

The current master here: https://github.com/theypsilon/Update_All_MiSTer/blob/master/update_all.sh

Has a number of commits (since the .sh contained within the 1.3 release), and is dated 24th October 2020.

I compared the two files, and they are quite different. Mainly around timeouts (old script = 120, new one = 600), and also the old version is using '--silent --show-error', which is causing the output to be less detailed (i'm guessing).

Once I discovered this, I thought I must be onto something.

What seems to happen, is when update_all.sh is run for the first time, although it is downloading the new version of the script, it will only use that new version on future runs (the old .sh is replaced with the new one). However, on the first run, it still uses the old code - causing the issues I mention above, and possibly other things (which have been tweaked since).

The solution is fairly straightforward thankfully :)

I guess there are two options, I don't know which you prefer to implement.

1) Update the "release" so it contains the latest copy of update_all.sh. This will solve the problem for future installations, however the downside with this is if you update or tweak the master update_all.sh, the release file would need to be updated as people running it for the first time will use the old code.

2) Preferred option - change the logic/code when update_all.sh is launched for the very first time. For example, implement some steps so that after it downloads the latest update_all.sh (which it seems to do when the script is run I think?), have some steps in place to 'relaunch' the script (which will force it to use the new version) - instead of proceeding to update using the old script.

Does that make sense? Either solution would work.

If you need further explanation, happy to assist. I've tested this numerous times now and this is definitely what's occurring.

I hope this helps.

Thanks!

[theypsilon]

Thank you so much for the investigation and the comprehensive explanation.

I solved it by fixing the zip files by now. In the future I would like to have these zips being updated automatically whenever a change happens. That will make sure that this issue is not happening again.

Issue tracking the automation of the release zips: https://github.com/theypsilon/Update_All_MiSTer/issues/63

[movisman]

Absolutely no worries at all, glad I could help out.

Fixing the zip files for now is perfect - i'll test and let you know if there are any issues. Automating the release zips is a solution I didn't think of, that sounds like a good idea to me.

Thanks again!