Deployment (integrate deployment tools as MXE components)

[treeswift]

• Prerequisites

• [x] openssl (host, target)

• [x] curl (target)

• [x] wget (target)

• [x] sshd/telnetd (target, host assumed — needs to work w/o MXE installation)

• [x] tar (target) — required by dpkg

• [x] perl (target) — dependency of some dpkg functions

• [ ] bash or a functionally comparable shell (target)

• [ ] coreutils or a BusyBox/ToyBox-like implementation

• Signing

• [x] snakeoil key/certificate generation (host, env-controlled entity name)

• [x] osslsigncode using command for timestamping (patch, host)

• [x] command-line equivalent of a TSA server (port, host)

• [x] sign while building (env-controlled cert) — explore injection in $(LD), $(INSTALL)

• Packaging and installation

• [x] post-build: package footprint separation

• [x] post-build: package archive generation (*.tar.gz, *.zip)

• [ ] post-build: DEB generation (chiefly, control.tar)

• [ ] post-build: repo maintenance (file tree, metadata, signing)

• [ ] dpkg/apt/cdebootstrap (ports, target)

• Remote execution

• [x] wmc (to provide mc for nssm compilation)

• [x] nssm for service management

• [x] sshd/sftpd/rsyncd (ports, target)

[treeswift]

Analysis and proposal:

• While -s --strip-program <script> is a viable option combination to inject a custom runnable into install, the $(INSTALL) variable is utterly disrespected by actual packages.
• The ultimately robust way to alter the payload of a build is to add a post-build step.
• The post-build step is added prior to the [done] log message in ./Makefile. It may use a timestamp placed immediately after the [build] message to enumerate files newer than the timestamp.
• touch k; touch l; find -name l -newer k does not find l on my system (the file modification time isn't granular enough to capture nano- and even microseconds), but touch k; sleep 0.2 || sleep 1; touch l; find -name l -newer k does. (The disjunction catches up in case of non-GNU sleep that wouldn't allow for sub-second granularity.)
• The catch-up script should be called mxe.postbuild.sh. its positional arguments would be ./Makefile's $(1) (package) and $(3) (target) and named (environment) arguments should include PREFIX, TARGET and CUTOFF (the pre-build timestamp file). PREFIX and CUTOFF should be specified as absolute paths and TARGET as a triplet.
• file <file> | grep 'PE32 executable' can be used to test for deliverable files that need signing (both *.exe and *.dll pass the test).
• A reasonable location for the CUTOFF timestamp is probably usr/.last-build-start.

[treeswift]

Intermediate progress: make snakeoil in subsystem/deployment (commit b7134c1188b621754a4416e27136a95f42cb7be4 ) produces usr/bin/selfsign.sh <file> that signs any PE32 binary in place; if the file is not a PE32 binary, it's left alone. All the needed certificates are automatically generated; all signatures are backdated to Y2K and stay valid at least until Y3K; every tool is run in place, no services and no network activity are involved.

Known issues (probably harmless but annoying):

make: Circular /home/lxe/Code/mxe-shared/usr/x86_64-pc-linux-gnu/installed/cc <- /home/lxe/Code/mxe-shared/usr/x86_64-pc-linux-gnu/installed/snakeoil dependency dropped.
make: Circular download-x86_64-pc-linux-gnu~cc <- download-x86_64-pc-linux-gnu~snakeoil dependency dropped.

I wanted to make snakeoil a prerequisite of every $(MXE_TARGETS) target (and thus made cc depend on it) but must have messed up the dependencies. @pahaze, would appreciate if you take a look.

In the meantime, I am porting bash/apt/dpkg (and tar if tar is involved) and adding the catch-up hook as described above.

[treeswift]

Commit 1d7ffe3ec3c6cb0a0632a934ebdbe0f8f06e4953 in subsystem/deployment provides implicit signing.

[treeswift]

https://github.com/armdevvel/mxe-SHARED/pull/23 for wget

[treeswift]

https://github.com/armdevvel/mxe-SHARED/pull/24 for perl

[treeswift]

https://github.com/armdevvel/mxe-SHARED/pull/27 for tar

[treeswift]

NSSM (Never Surrender Service Manager) progress tracked here: https://github.com/treeswift/nssm/issues/1

[treeswift]

https://github.com/armdevvel/mxe-SHARED/pull/30 for nssm

[treeswift]

Narrowing down our sshd/telnetd options.

1. dropbear is simple and its fork() use cases are trivial (it's possible to make sure that only one successor survives). Ditto tinyssh.
2. KPym uses a Jurassic version of cryptlib (which itself has a messy license + one has to file a form to get source code), but is otherwise well adapted to Windows. Should be relatively easy to fork and adapt it to libssh (LGPL, but I'll bite the bullet).
3. An open source Windows friendly telnetd should be actually be good enough for development/debugging. Unfortunately, hk-telnet-server (the best option I could find) doesn't have published source code (same "e-mail us and let's talk" approach).

Advancing in all three directions, mainly in (1) and (2).

[treeswift]

Related: https://github.com/armdevvel/mxe-SHARED/issues/42

[treeswift]

Note: I checked sshd/sftp as completed, but rsyncd is still a nice to have.

[treeswift]

Consider https://github.com/void-linux/xbps as the package manager of choice (evaluate).

Reason: xbps is 2-clause BSD while dpkg is GPL.

[treeswift]

Toybox aliases are now created as batch files, e.g. renice.cmd. Since there are no regular-file symlinks on NTFS (only directory symlinks implemented as reparse points), the nearest replacement seems to be hardlinks. Hardlinks packed into archives become regular files. A temporary solution may involve toybox's self-registration in the directory of its deployment as its whitelisted aliases. A permanent solution would involve a post-install script (the earlier we have xbps, the better).

This is, so far, one of the two primary use cases of a post-install script, the other one being sshd registration as a service.