setup script changes

[kwindrem]

A couple of things in your setup script:

EDITED:

endScript will not return so your exit call won't be executed so it's not needed. However you need to set the exit error code or PackageMaanger will assume the package installed properly. This could result in repeated install attempts.

I am adding a package conflict exit error code EXIT_PACKAGE_CONFLICT in the next release of SetupHelper but for now use EXIT_ERROR from EssentialResources which shows up on the GUI as unknown error but it's better than indicating install succeeded.

The best way to set the exit error code is to call setInstallFailed with the appropriate exit error code and a message explaining the cause of failure than call endScript. This will insure you don't leave the system with a partial install.

Until my SetupHelper package conflict resolution gets implemented (which won't be in v6.12), you should insure that RpiGpioSetup is uninstalled when installing RemoteGPIO. Without this, PackageMaanger might update RpiGpioSetup and wipe out your gpio_list (and possibly other) changes. You can insert this code in the INSTALL section:

# insure RpiGpioSetup is uninstalled since its functionalty conflicts with this package
if [ -f "$installedVersionPrefix""RpiGpioSetup" ]; then
    logMessage "uninstalling the RpiGpioSetup - it conflicts with RemoteGPIO"
    if [ -e "/data/RpiGpioSetup/setup" ]; then
        "/data/RpiGpioSetup/setup" "uninstall" "auto" "deferReboot" "deferGuiRestart"
    else
        logMessage "WARNING can't uninstall RpiGpioSetup - no package or no setup script"
    fi
    if [ -e "/data/settupOptions/RpiGpioSetup" ]; then
        touch "/data/settupOptions/RpiGpioSetup/DO_NOT_AUTO_INSTALL"
        touch "/data/settupOptions/RpiGpioSetup/DO_NOT_AUTO_ADD"
        touch "/data/settupOptions/RpiGpioSetup/FORCE_REMOVE"
    fi
fi

Minor: Starting with SetupHelper v6.0, you no longer need to set restartGui. It's handled if you update any GUI files with updateActiveFile. There are other service restarts automated by SetupHelper that might conflict with yours. updateRestartFlags in CommonResources sets various restart flags which are then handled in endScript

I would not cause your install to fail if GuiMods is not yet installed. This can occur with automatic installs since I can't control the order of package installations. It would be appropriate to log a message that GuiMods is needed but that won't be seen in the GuiMods menus. I think it's best to let the install succeed without GuiMods which will result in some menus not being available or up to date but you can explain that in your documentation

I will add RemoteGPIO to the default packages in SetupHelper v6.12.

[kwindrem]

Hold off. I'm adding forcePackageUninstall () to SetupHelper v6.12 so you can call the function rather than inserting the above code.

[kwindrem]

I'm making decent progress on a mechanism to identify package conflicts at the time one is installed.

I had hoped to detect conflicts automatically based on what is installed by other packages and what the package about to be installed would install. However, I'm tabling that for now.

The current plan is for a package to identify known package dependencies.

The PackageDependencies file in the package directory would list any known priorities. For RemoteGPIO:

GuiMods required RpiGpioSetup uninstall

SetupHelper would then prevent the install if all required packages are not currently installed or if any of the uninstall packages are installed. The GUI (and command line) would indicate "package conflict: install GuiMods first" and package conflict: uninstall RpiGpioSetup first". The install would then fail. The user would then need to follow the instructions then reattempt the install again manually.

Comments?

[kwindrem]

I have a beta version for you to play with. Set your SetupHelper branch/tag to beta

Here's a dependency file for you to use that requires GuiMods to be installed and RpiGpioSetup to be uninstalled

packageDependencies.zip

Here's what it looks like in the PackageManager menu with both conflicts:

[kwindrem]

FYI, here is an excerpt from the SetupHelper development guidelines document that discusses this issue: Package Dependencies Packages may interact with each other in undesirable ways. For example, one package that modifies the same file as another will install over the other package, hiding it’s modifications. Uninstalling either package will result in the stock file being used. The packageDependencies file located in the package directory defines basic requirements that prevent the package from being installed. Each line of the file includes a package name and whether that package must be installed or uninstalled in order for the package to be installed. For example the file for RemoteGPIO might be: RpiGpioSetup uninstalled GuiMods installed These lines tell SetupHelper to block the install of RemoteGPIO unless RpiGpioSetup is uninstalled and unless GuiMods is installed. Note that no changes to other package installations occurs so it would be acceptable for the dependency file for RpiGpioSetup to also specify that RemoteGPIO should be uninstalled. This way only one can be installed but the user has that choice.

[Lucifer06]

Sounds good. Giving a try now...

[Lucifer06]

Where am I supposed to put the packageDependencies file, which directory?

[kwindrem]

It goes in the package directory along with version etc.

I would also suggest you add a firstCompatbleVersion file as well if you aren't going to support versions prior to v3.20. This will prevent no file set messages and more correctly report version compatibility.

If you like what I've done so far, I'm going to migrate it to latest.

I have done a little more thinking about the package dependency mechanism and think it will be easy to add buttons to the GUI to automatically install/uninstall conflicting packages. Working on that this morning.

[kwindrem]

I've got conflict resolution implemented.

The Package editor menu will show a Show Details button if there is a package conflict detected. Pressing that will list the conflicts and enable a Proceed button.

Pressing Proceed will trigger the necessary installs and uninstalls.

When it's all done, the package conflict message should disappear and Install should enable.

Right now, the GUI may restart or the system may reboot as part of this process if the packages installed/uninstalled require it. I'll see if I can prevent this and provide Now and Later buttons like a normal install/uninstall.

EDIT: the above DOES wait for user input. (I was running an old version of PackageManager.py when I noticed the problem.)

In the Show Details view, Cancel simply returns to the normal menu without making changes.

These are all available for testing on the SetupHelper beta branch.

[Lucifer06]

I have SetupHelper v6.12–1 installed.

I have indeed a firstCompatibleVersion file. I'm adding the packageDependencies at the root with a request to have a dumb package installed, and it's effectively rejecting the install. Nice:

After resolving the conflict (removed the request for the dumb package), and reinstalling, I see the GUI keep stuck with the message "installing RemoteGPIO".

If I go back to the menu showing Active packages, It reports v3.0.17 is installed, also reports the package conflict message.

If I go again to the RemoteGPIO installation package menu, it is still stuck to "installing RemoteGPIO"....

I think there is something wrong in my setup script but can't find the problem. Any hints?

[kwindrem]

I don't see anything obvious. If you run your setup script from the command line (turn off Auto install first !!!), do you see any errors?

See what the return code is:

/data/RemoteGPIO/setup install auto echo $?

success is indicate by 0. Any other indicates the script is failing some way. You can compare this to the EXIT_... values in /data/SetupHelper/HelperResources/EssentilaResources

[Lucifer06]

Auto install off and echo $? returns 0

[Lucifer06]

and running /data/RemoteGPIO/setup uninstall auto returns 123 - Reboot needed

Doing install from command line goes fine. and then checking the gui shows package correctly installed with no errors. So the error is only when I trig the install from gui

[kwindrem]

If your script is exiting with 0 and no errors are reported then I'm not at all sure what's going on.

When PackageManager installs a package it includes deferReboot and deferGuiRestart on the command line. This will cause endScript to bypass those actions. If your problem is only occuring when PackageManager does the install then you can set these options on your setup call and see what happens.

The version of the setup script in the beta-3.0 branch shows a call to endScript if GuiMods installed file is missing. endScript is seeing a script action of INSTALL so this could result in processing based on a succeding install. For one thing, this removes the flag I use to prevent an install following a failure or manual removal. Any time you call endScript for reasons that the install shoudl not proceed, you need to change the scriptAction and in most cases that should be to UNINSTALL.

I am chasing a bug in the current beta code that causes repeated install following an uninstall triggered by an attempt to update an already updated file. I don't think your issue is related but it might be.

Other things I noticed but probably not the cause of the stuck install.

You should never set rebootNeeded to false. This is a global script variable and other things in HelperResources may set it. You could prevent a needed reboot by doing so.

Note that any things you add/change in /data/RemoteGPIO will be removed/changed if the package directory is updated (download manual or automaitc). I note /data/RemoteGPIO/conf several places in your script including a remove during uninstall.

Package options that you want to persist through a package update can be placed in /data/setupOptions/RemoteGPIO.

You should not be copying files to the qml directory, or anywhere else for that matter. These shoul be installed with updateActiveFile so that all checks are made and can be cleanly uninstalled. And with SetupHelepr v6.0+, anything you install with updateActiveFile does not need to be uninstalled. endScript does it for you. This includes services. However dBus settings are not removed on uninstall since these generally should persist for the next install.

SetupHelper now identifies most service resets and does them for you without setting flags: dbus-systemcalc-py, dbus-digitalinputs, the GUI are all handeled. Many resets to: gpip_list, RPI overlays, udev/rules.d changes all trigger a reboot.

Another thing you can do is use dbus-spy to look things in com.victronenergy.packageManager. This may provide a clue as to what's happening.

[drtinaz]

Greetings all. in regards to the issue of RemoteGPIO being installed from the gui, showing as being installed but the status still saying that it was still installing...I have seen this same behavior on my test system, but when I installed the package on my production system it completed properly and the status did not stick at "installing". In other words, it acted right. The difference may be that when I installed it on my production system it was a virgin install, but on my test system RemoteGPIO had been installed and uninstalled multiple times. Hope that helps.

[kwindrem]

I fixed the install hang. The call to the setup script was never returning and had to do with something being sent to stdout or stderr. I don't use those values so I just shut them off.

SetupHelper v6.13~1 is available now.

Checks are now made to prevent one package from overwriting the modification of another, so RemoteGPIO will fail because it is updating MbItemDigitalInput.qml. I needed to comment that line out of my copy of the RemoteGPIO setup script.

There is one addition to this file in RemoteGPIO: Generic I/O not in GuiMods. No response to my query about this on the Victron forum so I'll plan to add it in GuiMods soon.

[Lucifer06]

v6.13~1 installed. Tested with Venus OS Large. I can confirm this has fixed the issue I was seeing. I have updated the setup script following your recommendations. Regarding the uninstall, are you suggesting I should remove all the restoreActiveFile calls?

[Lucifer06]

hold-on before trying again the setup on rpi, I have a regression with beta 3.0.18

[Lucifer06]

v3.0.21 is available for testing rpi and normal image and fixed the regression above

[Lucifer06]

Install from shell or gui works like a charm with ask for rebooting gui or Venus as required. Nice!

[Lucifer06]

I noticed that after un-installing and executed the required reboot, trying to remove the RemoteGPIO package from the GUI doesn't seem to work. I hit the Remove button, but nothing happens. Am I supposed to handle something in the setup script?

[kwindrem]

Regarding restoreActiveFile, yes they can be removed.

Running SetupHelper prior to v6.0 would then not uninstall the files. What I did in my packages was include HelperFiles in the package. HelperFiles has its own version which IncludeHelpers compares tbetween SetupHelper and the package and uses the latest.

The restoreActiveFile calls won't do anything bad if left in place since..

[kwindrem]

Regarding uninstall, try restarting PackageManager:

svc -t /service/PackageManager

then try the uninstall again. If it succeeds, the issue is in PackageManager and I'll track that down. Otherwise, the setup script will need to be debugged. I can help with that since RemoteGPIO installs on my RPI test system at least enough to test the script.

/data/log/SetupHelper might identify a problem with the uninstall itself. Or you can run the uninstall from the command line and see if there are any issues:

/data/RemogeGPIO/setup uninstall deferReboot deferGuiRestart; echo $?

The echo at the end will report the exit code for the script which might be useful.

You can also check /data/log/PackageManager/current Most blocked operations would not be reported but if there's an error there might be something in there.

[Lucifer06]

Uninstall works fine. I'm talking about the Remove button, that I believe is supposed to remove the package from the Active Package page.

[Lucifer06]

there was a typo in previous-previous message. I do un-install, and then try to remove

[kwindrem]

Got it. Reproduced here. Debugging now.

I had to restore the stdout and stderr gathering for installs as it IS needed for some operations. But it all seems to work with v3.0.22

I noticed the v3.0.22 still has MbItemDigitalInput.qml in the setup script and fileList. This is trowing a conflict error with GuiMods. This worked with earlier versions of SetupHelper because I did not check for conflicts but I do now so you need to remove MbItemDigitalInput.qml from your package. GuiMods v10.9 which I just released has the Generic I/O selection.

[kwindrem]

Fixed Remove not working.

I also added a PackageManager restart command in the last menu in the PackageManager main menu. If you get into a state where the PackageManager menus don't work, try that then report here. Might help speed debugging.

[Lucifer06]

OK great you reproduce it. I just released v3.0.23 with no more MbItemDigitalInput.qml

[kwindrem]

I'm not seeing a v3.0.23. The beta branch shows a version of 3.0.22 and the main branch v2. something

[Lucifer06]

My mistake v3.0.23 is now available

[kwindrem]

PackageManager now checks for package file conflicts in the background like I do for dependencies, so these conflicts are visible even before an install. The install will be blocked in this case since doing so would fail anyway.

The problem with the old way is that a conflict would not be shown until an install was attempted and failed but the conflict information would be lost if PackageManager restarts. This condition also resulted in repeated install and failure.

Considering an automatic resolution for this type of conflict like I did with dependencies.

[kwindrem]

Yes, I see v3.0.23 now.

[Lucifer06]

Hi Kevin, may I introduce the request for an additional way to modify files during setup? There are some cases that may require multiple packages to update the same file. For instance /etc/udev/rules.d/serial-starter.rules were it is required to add more lines in this configuration file. It would make sense to let various package to update that file than trying to manage conflicts. Also it would be nice then if PackageManager keep track of what was added by which Package. What do you think? Should I open this topic as an "issue" in kwindrem/SetupHelper to discuss further?

[kwindrem]

Added the issue here: https://github.com/kwindrem/SetupHelper/issues/73

[Lucifer06]

Setup fixed. Thx to Kevin.

[kwindrem]

check out the issue discussing SetupHelper v.8.0 beta as it provides the functionality I think you are looking for:

https://github.com/kwindrem/SetupHelper/issues/73