[OH3] Prepare for openHAB 3.x packaging changes

[BClark09]

Issue information:

The deb packaging for openHAB 3 will be different from openhab2 packages. In brief, any path, file or executable that points to **/openhab2/** will now be **/openhab/**. This means things we should be prepared for include:

• Transitions to these directories in ZRAM.
• Installation of openhab through apt rather than openhab2 (currently, I have made it so that openhab2 will not automatically upgrade to openhab. You have to uninstall the former first.)
• Anything else that reference openhab2 paths explicitly.

TODO

• [x] re-create openHAB3 branch (Ethan)
• [x] write transition routine: on switch branch, replace files on disk and call openhab2_setup with "oh2"/"oh3" next -> Markus, PR #1187
• [x] replace openhab2 with openhab in any filename in the code (Ethan)
• [x] replace openhab2.service with openhab.service in any file/dirname -> Ethan
• [x] change openhab2_setup to openhab_setup and to take "openHAB3" as argument (and install OH3 of course)-> Markus ensure old OH is uninstalled first (when package names change) double-check to uninstall/install addons packages, too ?
• [x] replace ASCII art logo -> Ethan, PR #1184 merged
• [x] think twice -> everybody
• [x] test, test, test -> everybody

[ecdye]

My question, how do we make this transition while we still support openHAB 2.5.X?

[BClark09]

It might be best to add another branch for openHAB 3.x, and add a feature in the current branch to transition. The feature would make the changes outlined in the issue, and the git checkout to the new branch.

[BClark09]

Any thoughts on this? openHAB 3.x packages will soon (within the week) be available on the apt repo. Whilst it will only install them if a user specifically asks for it (i.e. with sudo apt install openhab whilst on the snapshot repo), I don't understand ZRAM or the newer changes in openHABian well enough to understand if this will cause a problem if they do.

[ecdye]

I did not know how soon this would become an issue. I will make it a high priority. @mstormi we should probably make this our primary concern for the moment as it is a more central openhabian issue.

[mstormi]

First OH3 release is targeted for December. I agree that we should make it a high priority, but I'd really like to finish all the stuff we've been working on (fix this ZRAM stuff and get image and refactoring out that is) Note I'm always pinning the 3 top-most urgent issues.

[BClark09]

I'll be putting some instructions up for standard Linux users when the first package appears in the snapshot repo (probably tomorrow?). My instructions currently contain a warning to openHABian users but ideally I'd like it to be a little softer than it's currently written, do you have any insight to what might happen on a normal (Pi image) install?

https://gist.github.com/BClark09/ac2e259d8e0c03c845d25d2eb90de4b2

Perhaps, if we say that ZRAM should be disabled first through the menu, then OH3 can be tested using the APT specific instructions with some probably openHABian related bugs (e.g. Fronttail not working) (and obviously they will no longer have the benefits of ZRAM)

[ecdye]

Looks pretty good, but yes ZRAM should be disabled until we update the paths for the new version and other than that the end user should be fine. Perhaps leave a link to this issue it the file to report any issues so we can investigate and resolve before pushing the fix as part of openHABian. Personally I feel that we should let this sit for a bit to allow us time to implement it the right way and avoid any potential issues caused by a rushed patch to allow upgrades.

@mstormi thoughts?

[mstormi]

Fine for me. It shouldn't harm if ZRAM kept running it just doesn't apply to OH3 dirs. But yes you should recommend users to switch it off via menu, and be it just for the sake of making them aware it doesn't work for OH3 yet.

Oh, and you should recommend in your doc to make a full OS backup, too, using amanda or whatever.

[ecdye]

I have created a new branch for us to track openHAB 3 changes on: openHAB3.

[mstormi]

Hmm. Would we, from now on, need to apply any change to both branches, master and openhab3 ? I wouldn't want that. Let's apply OH3 specific changes only to that new branch, and rebase it at regular intervals.

[ecdye]

That is what I had intended to do it should be fairly simple as most of the changes should not conflict.

[mstormi]

So what's the full plan: do we want to create an openHABian branch that works for OH3 ONLY or do we aim to allow for running both, OH2 and 3, on the same machine ? The former is simpler and maybe ok for beta testers but we'll have to think about the latter sooner or later, i.e. how would an ordinary user migrate ? Keep in mind they probably will want to migrate forth and back a couple of times so would we need to enable for that inside the current openHABian master branch ?

[BClark09]

The way I saw it happening was that you have a openhabian-config script in the master branch that which runs a migration script and changes itself to the openhab3 branch and should only be advertised for testing purposes. The openhab3 branch will eventually become the master branch.

Keep in mind they probably will want to migrate forth and back a couple of times

I can't see how this would be easy to do, especially with the openHAB apt package.

[mstormi]

I can't see how this would be easy to do, especially with the openHAB apt package.

my possibly naive first approximation would be: apt purge openhab apt install openhab2 (plus config import). Wouldn't that work ?

[ecdye]

I think we start with a fully functional openHAB3 only branch to work out all the development bugs, and then create a script to allow for switching back and forth.

[mstormi]

Ok, now that we finally made it to our release let's take on this. Let's make a plan (yeah, I'm German 😄 ).

1) I suggest we first dump and re-create the openHAB3 branch as a copy of current stable because that is (currently) equivalent to 1.6 release hence a well-defined starting point to branch from. Ethan would you please do and add that single commit for "includes/" 7c2f7829306e893439255a163bd3880d05515472 ?

2) We should handle all changes to the openHAB3 branch like we do for production, i.e. put up PRs and have someone other than yourself review and merge it so we all are aware of everything 'hot' around this OH3 task. Any PR should also rebase on master, too, to not have openHAB3 and master branches get out of sync.

3) live testing is as "simple" as to switch branches. You can set clonebranch=openHAB3 in /etc/openhabian.conf and have openhabian-config update itself on start to get there. Use the branch selector menu item to switch back and forth to/from master if needed. Helpful to find out what would happen to an existing installation if some user did the same.

4) TODO list I'll take notes here for the time being. Frankly I feel a little unsure as I have no own idea yet what we need to put on this list. Starting with Benjy's post #1, item #3 Ethan you have done that already, right ?

1 I'll do myself. Currently I'd think we can simply add them to ztab. After boot, ZRAM will not consume any (substantial) amount of RAM for entries in ztab until you make changes to that directory. As we'll only ever run either openHAB2 or 3 but never both in parallel, there should be nothing wrong with adding them both. Let me know if you think I'm mistaken somewhere.

[I have to give that another thought and testing but I'd think if I'm right we can even apply that ztab extension to master already]

Ethan would you please take care of #2 ?

Holger will you be able to help, too ? Maybe at least with testing ?

TODO moved to post #1

[ecdye]

1 I already did @mstormi so we are set in that regard.

I'll work on #2

[mstormi]

ztab you mean ? Or the branch re-creation ?

[ecdye]

ztab you mean ? Or the branch re-creation ?

Branch recreation, I did that right after we released 1.6

[ecdye]

@BClark09 Will the apt repo links for openhab3 be the same or will they be changing?

[mstormi]

openhab2.service also changed to openhab.service so we need to adapt that, too

adapt logo, too. See https://github.com/openhab/openhab-distro/issues/1139

[01:00:38] openhabian@devpi:~$ !?810
ssh openhab@localhost -p 8101

                           _   _     _     ____
   ___   ___   ___   ___  | | | |   / \   | __ )
  / _ \ / _ \ / _ \ / _ \ | |_| |  / _ \  |  _ \
 | (_) | (_) |  __/| | | ||  _  | / ___ \ | |_) )
  \___/|  __/ \___/|_| |_||_| |_|/_/   \_\|____/
       |_|       3.0.0-SNAPSHOT - Build #1967

Use '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
To exit, use '<ctrl-d>' or 'logout'.

openhab>

[ecdye]

@mstormi here is a RD that I put together that I can create a separate PR for if you like it WDYT? https://github.com/ecdye/openhabian/blob/openHAB3/includes/bash_profile

[mstormi]

yes please. Mind that in the original, the "open" is colored orange

[BClark09]

@BClark09 Will the apt repo links for openhab3 be the same or will they be changing?

Yes, they (snapshot, testing, stable) will still be hosted in the same repos as the openhab2.

[mstormi]

Still wondering what would be the most intelligent way to handle this. It's ok to have specific static paths in the branches so the right dirs are used when user changes branch to openHAB3. That however will not change any of the non-duplicated data on disk. I'm considering to replace that on the fly... what comes to my mind is /etc/ztab /etc/amanda/*/disklist /etc/systemd/system/frontail.service but I'm unsure if that's all. wdyt ? too risky we miss anything or mess anything up ?

Eventually it's safer to reinstall Frontail/Amanda/ZRAM ?

PS if someone is willing to migrate to OH3 he will be prepared for that (and has a backup). But I think that makes no difference for the decision to take above

[ecdye]

I'm not sure I understand what you are talking about, is this in relation to zram or backups or what?

[mstormi]

This is for everything that exists ON DISK that uses dir- or filenames to contain openhab2 (because when it was installed it was the OH2 code so openhab2 dirnames were used). It's the logfile names for Frontail, the list of to be backed-up directories in Amanda and the list of directories to apply ZRAM to. When we change branch to openHAB3, "openhab2" needs to be replaced by "openhab". And eventually the inverse if we change back to master/stable. The list of files to change is what I remember ATM.

[ecdye]

Oh, ok. I think we replace them all as we have already warned that branch switching has potential issues.

Also when we finish OH3 and it has its first release, we should probably rename our master branch to main in line with the new git recommendations.

[mstormi]

I think we replace them all

Ok so I'll change my "ZRAM-only" PR to run sed on all of these these files when switching branches

[holgerfriedrich]

@mstormi Not sure if I got the plan correctly.

I have the feeling that there are different possible goals:

• Adapt openHABian to OH3 -> changes to be done on openhab2 branch, right now renaming of directories and config files from oh2 to oh, etc.
• Add support for OH2->OH3 transition (will be needed anyway at some time to aid transition of existing installs)
• Add support for OH3->OH2 transition (rollback)
• Providing an OH3 image of the latest OH3 milestone (for test purposes, lowering the efforts for users to start testing OH3)?

What are the priorities here?

Hard to get an overview what is needed here. For practical reasons, I think we should split this task to several issues and tag them with OH3.

[mstormi]

@mstormi Not sure if I got the plan correctly.

I think you didn't... so to explain along your questions:

(1) Adapt openHABian to OH3

No, we don't change any files in the master/stable branches. I assume that was a typo and you mean openHAB3 branch. Yes. This is the essential part we need to address first.

(2) Add support for OH2->OH3 transition (will be needed anyway at some time to aid transition of existing installs)

Transitioning is done by selecting the openHAB3 branch from the menu. This is what I'm implementing right now for everything that is on disk, still pointing to OH2 What's missing is to implement apt purge openhab2; apt install openhab (plus ...-addons, eventually). Added that TODO to the table above. Eventually export config before/import after, too.

(3) Add support for OH3->OH2 transition (rollback)

Well. We're all not convinced that this will work fine so strictly speaking this is out of spec. We'll nonetheless implement the branch switch back to master and inverse replacement in files.

(4) Providing an OH3 image of the latest OH3 milestone (for test purposes, lowering the efforts for users to start testing OH3)?

No. It's as simple as to set clonebranch=openHAB3 before installation.

What are the priorities here?

1,2,3

I think we should split this task to several issues and tag them with OH3.

Will use the tracking table here and link to PRs. Will open separate issues only if more complex.

[mstormi]

are we set? Haven't gotten any user feedback so far so will leave this open a couple of days more.

I know it's a little late but we should re-think keeping the openHAB3 branch Now that the migration code is the same in both branches isn't the whole difference that's left just about static directory names ? If so we should be able to replace them by variables, shouldn't we ?

[ecdye]

Possibly, however, if we do get rid of it we should migrate our master branch to main. Additionally there are some benefits to keeping it separate as it helps to indicate that OH3 is not necessarily ready for prime time.

[mstormi]

yes I'm undecided too. It was a really annoying hassle though to get changes into both branches, lots of conflicts on rebasing and wasted work in the first place until I realized the script replaces itself so I'm clearly biased. But that PR wasn't fun at all.

[mstormi]

Short of undiscovered bugs, the OH3 implementation seems to work now.