Migrate OctoFarm-Installer into OctoFarm

[NotExpectedYet]

This is under the assumption that we should create MSI / DEB packages for our systems for choice which is Ubuntu Linux / Windows. Any other should be disregarded to reduce support efforts or allow for a competent person to PR if they really wish.

The way I see it currently is:

• Users should only install with the packaged versions on our two supported systems.
• OctoFarm then takes over for any of the required tasks within the UI itself. So Restarting of it's service and also updating itself internally.

There's some work to do to make this a reality, the .deb and msi installers need finishing off and making sure they can be built with github action workflow. This would come after the recent workflow for building the client on development / master before creating a release.

• [ ] Create .deb installer
• [ ] Create .msi installer
• [ ] Update OctoFarms internal utility to use the release downloader from OctoFarm installer.
• [ ] Add plenty of checks in OctoFarm to make sure node is available and the correct version and stall if not.
• [ ] Add a check in to see if this is a .git repo && != development build and stall the startup process. Instruct user to install with .deb / msi going forward.
• [ ] Remove the pm2 commands / setup instructions
• [ ] Remove the UI references for pm2 / server checks as we will not longer need it. We can now use native installer methods on each system. This requires a little more user setup linux side but if they want the internal restart / update commands to work this maybe required if they're on a sudo locked system.
• [ ] Re-organise the OctoFarm folder to suit this + the installer script to download releases.
• [ ] Replicate the OctoFarm-Installer scripts into the OctoFarm UI so users may action functions.
• [ ] Create a .env file with the .deb / msi to identify the system and make OctoFarms internal commands react to this.

[davidzwa]

As discuseed locally we've decided to go for the dark road of innosetup. Still a todo on that brainstorm: how to run octofarm as a windows service (pm2 is not restart-safe, weirdly enough)

• I dont feel comfortable using https://github.com/jon-hall/pm2-windows-service as it is outdated
• Using the startup folder ... is also pretty weird and does not auto-restart https://stackoverflow.com/a/66675399/2733437
• Any other solution?

In the end I'd like users to really consider using docker... as much as possible really.

[NotExpectedYet]

As discuseed locally we've decided to go for the dark road of innosetup. Still a todo on that brainstorm: how to run octofarm as a windows service (pm2 is not restart-safe, weirdly enough)

• I dont feel comfortable using https://github.com/jon-hall/pm2-windows-service as it is outdated
• Using the startup folder ... is also pretty weird and does not auto-restart https://stackoverflow.com/a/66675399/2733437
• Any other solution?

In the end I'd like users to really consider using docker... as much as possible really.

Them using docker seems better and better to me at the moment. Did docker finally support Windose Home? That was my initial reservation for that. Docker covers OSX / Windows, deb/Docker for Linux. We have full control and support isn't a nightmare.

[NotExpectedYet]

As discuseed locally we've decided to go for the dark road of innosetup. Still a todo on that brainstorm: how to run octofarm as a windows service (pm2 is not restart-safe, weirdly enough)

• I dont feel comfortable using https://github.com/jon-hall/pm2-windows-service as it is outdated
• Using the startup folder ... is also pretty weird and does not auto-restart https://stackoverflow.com/a/66675399/2733437
• Any other solution?

In the end I'd like users to really consider using docker... as much as possible really.

Them using docker seems better and better to me at the moment. Did docker finally support Windose Home? That was my initial reservation for that. Docker covers OSX / Windows, deb/Docker for Linux. We have full control and support isn't a nightmare.

Additionally, I don't have to do your bugfix if all linux XD

[davidzwa]

Im doubting. The true question is: "How tech savvy or familiar with windows is the user?"

• Not at all: setup.exe + innosetup + windows service registration is the way
• A bit: well.... still high probability of questions/support required on discord
• Good enough: they prolly already have WSL...

WSL with Docker for Desktop on windows is truly another challenge for some inexperienced users. Here's why:

• users have to setup WSL (Prof/Home/ ... any variation should use WSL. and not old HyperV which was Prof only & quite slow)
• WSL installation consists of 5 steps. And users should skip the 6th (as they dont need an extra linux OS) https://docs.microsoft.com/en-us/windows/wsl/install-win10
• Users need to ensure that virtualization is turned on in BIOS... jep. That bad.
• Once WSL is setup, they need to understand that WSL can eat up RAM badly... there's a fix with a config in the user's folder, but how to get that information to users :/ C:\Users\david/.wslconfig

  [wsl2]
  # https://docs.microsoft.com/en-us/windows/wsl/wsl-config#configure-global-options-with-wslconfig
  memory=4GB
  processors=2

Yep. InnoSetup at least makes sure we can install under Program Files, but we'd need to look into service registration. It's the hard way which is gonna fix our conundrum.

[NotExpectedYet]

There's: nw.js using node-webkit (UI based not service based) DOH! forgot about pkg, no idea if that can create a windows service though. Maybe coupled with nssm.exe?

also: https://www.npmjs.com/package/os-service

Also https://github.com/coreybutler/node-windows

[NotExpectedYet]

@davidzwa UNDONE!? XD

[davidzwa]

We've analysed and decided that the node-windows and node-linux options go best with our requirements (no node-gyp packages). InnoSetup to build the setup. No what to put in the service! That's the question.

[davidzwa]

We should add an innosetup to provide OctoFarm-Installer as a CLI and add our binary to System Path - just like docker above. The name should be octofarm.exe and it will allows users to run octofarm status, octofarm start , octofarm stop and octofarm update. This only works for the installation version ofc.

This CLI should be a compiled x64/x86 binary for AMD64 and not based on PKG as this is a huge download. Considering writing it in Rust. Because why not.

[davidzwa]

And we should find out how make this binary interact with systemd or windows services. If this is not easy with Rust, we should drop it and go for something else like powershell. I also find comfort in writing a dotnet console application as long as it doesnt require the dotnet runtime.

[NotExpectedYet]

And we should find out how make this binary interact with systemd or windows services. If this is not easy with Rust, we should drop it and go for something else like powershell. I also find comfort in writing a dotnet console application as long as it doesnt require the dotnet runtime.

Systemd is totally fine as far as I'm aware as dpkg -i octofarm.deb command replaces the binary then restarts the service anyway. I will double confirm that once I've done the history stuff and got the PR ready.

I believe windows is going to be the issue here. I deffer to your judgement regarding windows as I'm not intimate with it at all.

[davidzwa]

Wouldnt the CLI I suggest work better for Unix as well? We're basically doing what PM2 does, but then in our way and in our control. The CLI can call .deb installs behind the scene

[davidzwa]

I want to make sure were in complete control, if the user has to run dpkg -i octofarm.deb that's an issue.

[NotExpectedYet]

I'm not saying a user has to run it btw, I'm just using that as an example to why it doesn't lock the binary we will use on linux.

I'm fine with a cli been written for linux I just feel it's not required/overkill in that situation. It would give us a lot more control though and be outside of the node process that's running so it wouldn't make a difference if it's killed.

[davidzwa]

Oh I completely believe you on the safety aspect, Im just worried about the installation from A-Z and then for updates.

Btw I just looked up how small a self-contained single file dotnet core console app can be. It aint pretty, but it's much better than pkg starting at 77MB.

Big advantage of .NET5 is that we get interop with windows and System.* libraries

[NotExpectedYet]

Well as we've consistently learned whatever is out of the user control the better. Not relying on the node process is again better too. CLI it is.

Honestly whatever's easier, I like PKG as it's all node based but I ain't really quibbling over the MB size of that currently. I'm not sure on the node binary size currently but they we're 25MB+ for windows and Linux was upwards of 40MB+ I believe so those would need to be taken into consideration. Tried having a quick search but couldn't find the exact sizes but will double check.

[davidzwa]

Here is a published version of a WorkerService in Dotnet Core published for Linux x64 standalone (many others also possible like arm, x86, etc) It has a systemd package installed internally, so it can interact with systemd and journald. There are also variants possible to interact with Windows Services the same way.

This service can kick-start and monitor OF

[NotExpectedYet]

Here is a published version of a WorkerService in Dotnet Core published for Linux x64 standalone (many others also possible like arm, x86, etc) It has a systemd package installed internally, so it can interact with systemd and journald. There are also variants possible to interact with Windows Services the same way.

This service can kick-start and monitor OF

Cool as!

[davidzwa]

We'd end up with:

• a .NET5 console CLI on PATH, which can make sure this daemon is started and it's service is correctly specified in systemd or Windows Service
• a .NET5 worker daemon starts and makes sure OF PKG is started and checks if it is still running
• .deb/pkg OF somewhere next to daemon

Yes adding the worker service seems complicated. True. The thing is I want it to be able to update OctoFarm whilst having nothing to do with it (no file locks, no nothing).

[NotExpectedYet]

We'd end up with:

• a .NET5 console CLI on PATH, which can make sure this daemon is started and it's service is correctly specified in systemd or Windows Service
• a .NET5 worker daemon starts and makes sure OF PKG is started and checks if it is still running
• .deb/pkg OF somewhere next to daemon

Yes adding the worker service seems complicated. True. The thing is I want it to be able to update OctoFarm whilst having nothing to do with it (no file locks, no nothing).

For Windows thats required I gather and if it's all replicable through one system then it's much preferred compared to the two options we currently have with inno/deb creation.

How easy would it be for you to get a mock up as I'm not familiar with .net in the slightest. Interested to learn though. Would we use PKG as a base for the binarys and eventually create our own? I gather a staged approach to this would be preferable.

[davidzwa]

I have a .NET5 worker ready and it is already calling a secondary process to start the PKG OF version binary of OctoFarm in win10 (just having issues with the bg_default.jpg and MONGO on this laptop so bear with me a sec).

[davidzwa]

But hey at least I have a daemon which can boot OF!

[davidzwa]

And here the proof that NodeJS is being run

[NotExpectedYet]

Absolutely legendary man! This is coming along epically!

[davidzwa]

Now its also detecting any exits from OF... and reboots it. I do wonder if we should introduce a timed backoff mechanism to avoid spamming the CPU with OF restarts. Thoughts?