All files in same folder deleted

[abenzick]

When installing a new update to an executable, the tufup script that runs in the cmd line window deletes ALL files in the same folder as your original executable.

My users will commonly keep my .exe on their desktop or in their downloads folder. In either of those locations, users open the app, a new update is detected, the update is downloaded and applied but then EVERYTHING in the same folder as the original executable is deleted and the only thing remains is the new properly updated executable.

The patch process works perfectly but all the files being deleted in the same folder is a CRITICAL error.

[abenzick]

I was so excited to launch my first auto-update but had to yank it once I tested this out and confirmed files were being deleted.

[dennisvang]

@abenzick I am very sorry to hear you are having this trouble. It is indeed a serious inconvenience.

Please find an explanation and proposed solution below.

Background

The default script used to copy the update files follows the exact same behavior as pyupdater, as can be seen here. This means it uses robocopy with the /purge option to clear the app installation directory.

The behavior is mentioned in the readme section "How updates are applied":

By default, updates are applied by replacing all files in the current app installation path with files from the latest archive. ...

and:

... a script is started that clears the current app installation dir, and moves the new files into place.

The underlying assumption is that your application lives in its own directory, for example in %PROGRAMFILES%\MyApp\, as a normal windows application would.

Unfortunately, your issue proves that this assumption is not always correct.

Temporary workaround

In your specific case a quick workaround would be to make sure your executable is placed in a separate directory.

The Client.download_and_apply_update() method, on Windows, does accept an extra_robocopy_options argument, which allows one to exclude individual files or folders from the purge. However, I don't think this would be any help in your specific case.

Actions to be taken on our side

Obviously we recognize the seriousness of this issue, and we aim to mitigate the dangers involved in the purge behavior.

For this reason I would propose the following improvements:

• Make the purge behavior optional, instead of default.
• Add a more explicit warning to the readme, highlighting the dangers of the purge behavior in case applications are not placed in a separate directory.
• For increased flexibility, allow override of all robocopy options, instead of extra_robocopy_options

Please let me know if this would work for you.

[dennisvang]

An alternative would be to maintain a file manifest, and delete only files and directories mentioned in that manifest, instead of using the purge option. However, this also has its drawbacks and dangers.

[abenzick]

This would be some great steps moving forward. Having the flexibility to keep applications untethered to an install folder will be a common scenario and a good feature to include.

At the moment, I’m unable to push out updates.

Aaron

On Sep 9, 2022, at 3:06 AM, Dennis @.***> wrote:

 @abenzick I am very sorry to hear you are having this trouble. It is indeed a serious inconvenience.

Please find an explanation and proposed solution below.

Background

The default script used to copy the update files follows the exact same behavior as pyupdater, as can be seen here. This means it uses robocopy with the /purge option to clear the app installation directory.

The behavior is mentioned in the readme section "How updates are applied":

By default, updates are applied by replacing all files in the current app installation path with files from the latest archive. ...

and:

... a script is started that clears the current app installation dir, and moves the new files into place.

The underlying assumption is that your application runs in its own directory, for example in %PROGRAMFILES%\MyApp\, as a normal windows application would.

Unfortunately, your issue proves that this assumption is not always correct.

Temporary workaround

In your specific case a quick workaround at this point would be to place your executable in a separate directory.

The Client.download_and_apply_update() method, on Windows, does accept an extra_robocopy_options argument, which allows one to exclude individual files or folders from the purge. However, I don't think this would be any help in your specific case.

Actions to be taken on our side

Obviously we recognize the seriousness of this issue, and we aim to mitigate the dangers involved in the purge behavior.

For this reason I would propose the following improvements:

Make the purge behavior optional, instead of default. Add a more explicit warning to the readme, highlighting the dangers of the purge behavior in case applications are not placed in a separate folder. For more flexibility, allow override of all robocopy options, instead of only allowing extra_robocopy_options. Please let me know if this would work for you.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[dennisvang]

@abenzick A new release is now available (0.4.0) which should mitigate the issue:

The app_install_dir is no longer purged by default.

Purging is still possible by calling client.download_and_apply_update() with purge_dst_dir=True (default is False), as illustrated in the tufup-example repo here.

I also sprinkled additional warnings throughout the code, and added a very explicit warning to the readme.

Hopefully this will prevent similar issues in the future.

[abenzick]

Very nice response! It is appreciated. I'm giving a test right now!

[dennisvang]

@abenzick Just wondering: Did you encounter any more problems here?

[abenzick]

I’ve implemented a build with 0.4 but I haven’t tested it yet as I still have the 0.3 version in the wild. I’m waiting a period of time to force users to update to the new build through my old API version checking way before I do an automatic update. I’ll keep you updated.

On Sep 13, 2022, at 2:04 AM, Dennis @.***> wrote:

@abenzick https://github.com/abenzick Just wondering: Did you encounter any more problems here?

— Reply to this email directly, view it on GitHub https://github.com/dennisvang/tufup/issues/38#issuecomment-1244987516, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABW6DM7I4TGMEDIRZ6W66Y3V6ARQJANCNFSM6AAAAAAQIFDYVE. You are receiving this because you were mentioned.

[dennisvang]

@abenzick Allright, thanks for the info.

[abenzick]

I'm to the point of testing this out. On my initial test environment, things are working well and the files in the same folder remain after an updated.

When testing it in a controlled corporate environment, the patch process is failing and not updating. The command prompt opens quickly and then disappears. The old version remains.

The previous version applied the patch no problem. I don't have any log outputs so I'm not sure exactly where it is failing but I will work on getting some more intel on that process. Is there an option to output the the results from the command prompt patch process to a file for checking for errors?

[dennisvang]

... Is there an option to output the the results from the command prompt patch process to a file for checking for errors?

@abenzick Unfortunately there's no option yet to create log files from the install script.

However, you could try calling Client.download_and_apply_update() with the debug=True argument. That should prevent the console from closing.

[abenzick]

Well, I've passed on debug=true with:

client.download_and_apply_update(skip_confirmation=True, debug=True, progress_hook=progress_hook)

The window disappears on both environments. Also of note, there is no display in the cmd.exe window. It's just totally black, no output text, even when the patch applies successfully in the one environment.

[dennisvang]

... The window disappears on both environments. Also of note, there is no display in the cmd.exe window. It's just totally black, no output text, even when the patch applies successfully in the one environment.

@abenzick That is strange... Did you try what happens without the progress_hook?

[dennisvang]

@abenzick To help figure this out, I've released 0.4.1, which replaces the debug argument by a log_file_name argument.

If you specify a log file name, the output (including errors) from the default install script (on windows) should be logged to a file by that name, which is placed in your installation directory.

Also see the updated tufup-example.

Hope this helps.

[abenzick]

You continue to impress with your professional responses. Thank you!I’ll be testing this shortly.

[abenzick]

@dennisvang 0.4.1 logging options is excellent. Things are working properly and I have things updating ok. Please close this issue out and THANK YOU for the logging

[dennisvang]

@abenzick Good to hear everything is working properly now. :-)