test packaging - Githubissues

borgified commented 4 years ago

@Alan-R have a look at this when you get a chance. after creating .deb package, its uploaded to the GH release page as an asset: https://github.com/assimilation/assimilation-official/releases/tag/v1.1.8-pre

there's a bunch of hardcoded stuff for now but those are easily made into parameters, just wanted to show the overall workflow and make sure it was ok.

Alan-R commented 4 years ago

This looks like great progress! There are a few things that need to be added to the package. The ones that come to mind are the inclusion of all the various discovery scripts, the secure crypto directory. Probably some other things that were present in the previous packages for the R1 stuff. Maybe we should discuss this. Is this using fpm?

borgified commented 4 years ago

yes using fpm, you can see it happening here. if you can make a list of all the files you want to be included in the package and their installed location then i can add it to the package.

borgified commented 4 years ago

this is what's included in package now

root@ff4ad3e61641:/workdir# dpkg -c nanoprobe_2.0.0_amd64.deb
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/bin/
-rwxr-xr-x 0/0         3551254 2020-08-17 06:39 ./usr/bin/nanoprobe
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/
drwxrwxr-x 0/0               0 2020-08-30 22:35 ./usr/share/crypto.d/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/
-rwxr-xr-x 0/0             241 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/install-path.sh
-rwxr-xr-x 0/0            2832 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/install.sh
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/1.5/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/1.5/files/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/1.5/files/etc/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/1.5/files/etc/default/
-rw-r--r-- 0/0               0 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/upstart/1.5/files/etc/default/nanoprobe
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/1.5/files/etc/init/
-rw-r--r-- 0/0             987 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/upstart/1.5/files/etc/init/nanoprobe.conf
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/0.6.5/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/0.6.5/files/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/0.6.5/files/etc/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/0.6.5/files/etc/default/
-rw-r--r-- 0/0               0 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/upstart/0.6.5/files/etc/default/nanoprobe
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/upstart/0.6.5/files/etc/init/
-rw-r--r-- 0/0            1039 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/upstart/0.6.5/files/etc/init/nanoprobe.conf
-rwxr-xr-x 0/0             261 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/generate-cleanup.sh
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/sysv/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/sysv/lsb-3.1/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/sysv/lsb-3.1/files/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/sysv/lsb-3.1/files/etc/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/sysv/lsb-3.1/files/etc/default/
-rw-r--r-- 0/0               0 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/sysv/lsb-3.1/files/etc/default/nanoprobe
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/sysv/lsb-3.1/files/etc/init.d/
-rwxr-xr-x 0/0            3993 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/sysv/lsb-3.1/files/etc/init.d/nanoprobe
-rw-r--r-- 0/0             101 2020-08-30 23:46 ./usr/share/pleaserun/nanoprobe/delete.sh
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/systemd/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/systemd/default/
-rw-r--r-- 0/0              33 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/systemd/default/install_actions.sh
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/systemd/default/files/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/systemd/default/files/etc/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/systemd/default/files/etc/systemd/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/systemd/default/files/etc/systemd/system/
-rw-r--r-- 0/0             592 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/systemd/default/files/etc/systemd/system/nanoprobe.service
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/systemd/default/files/etc/default/
-rw-r--r-- 0/0               0 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/systemd/default/files/etc/default/nanoprobe
-rw-rw-r-- 0/0             253 2020-08-31 00:14 ./usr/share/pleaserun/nanoprobe/fix.sh
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/launchd/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/launchd/10.9/
-rw-r--r-- 0/0              54 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/launchd/10.9/install_actions.sh
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/launchd/10.9/files/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/launchd/10.9/files/Library/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/pleaserun/nanoprobe/launchd/10.9/files/Library/LaunchDaemons/
-rw-r--r-- 0/0            1931 2020-08-17 07:56 ./usr/share/pleaserun/nanoprobe/launchd/10.9/files/Library/LaunchDaemons/nanoprobe.plist
drwxrwxr-x 0/0               0 2020-08-31 00:16 ./usr/share/assimilation/
-rw-rw-r-- 0/0             189 2020-08-30 22:36 ./usr/share/assimilation/copyright
drwxrwxr-x 0/0               0 2020-08-31 00:16 ./usr/share/assimilation/discovery_agents/
-rwxrwxr-x 0/0            4851 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/sshd
-rwxrwxr-x 0/0            1806 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/os
-rwxrwxr-x 0/0           11228 2020-08-17 06:02 ./usr/share/assimilation/discovery_agents/netconfig
-rwxrwxr-x 0/0            9634 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/tcpdiscovery
-rwxrwxr-x 0/0            2892 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/cpu
-rwxrwxr-x 0/0            5069 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/docker
-rwxrwxr-x 0/0            7306 2020-08-17 06:02 ./usr/share/assimilation/discovery_agents/packages
-rwxrwxr-x 0/0            2136 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/upstart
-rwxrwxr-x 0/0            3100 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/login_defs
-rwxrwxr-x 0/0            2979 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/nsswitch
-rwxrwxr-x 0/0            4915 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/assim_common.sh
-rwxrwxr-x 0/0            4086 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/findmnt
-rwxrwxr-x 0/0            3181 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/monitoringagents
-rwxrwxr-x 0/0            5065 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/pam
-rwxrwxr-x 0/0             877 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/arpcache
-rwxrwxr-x 0/0            2873 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/auditd_conf
-rwxrwxr-x 0/0           17716 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/sudoers
-rwxrwxr-x 0/0            1971 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/partitions
-rwxrwxr-x 0/0            1943 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/ulimit
-rwxrwxr-x 0/0            8981 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/fileattrs
-rwxrwxr-x 0/0            7886 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/mdadm
-rwxrwxr-x 0/0            2980 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/vagrant
-rwxrwxr-x 0/0            1806 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/commands
-rwxrwxr-x 0/0            6316 2020-08-17 06:02 ./usr/share/assimilation/discovery_agents/proc_sys
-rwxrwxr-x 0/0            4257 2019-08-04 00:50 ./usr/share/assimilation/discovery_agents/checksums
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/doc/
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/share/doc/nanoprobe/
-rw-r--r-- 0/0             138 2020-08-31 00:16 ./usr/share/doc/nanoprobe/changelog.gz
drwxr-xr-x 0/0               0 2020-08-31 00:16 ./usr/lib/
drwxrwxr-x 0/0               0 2020-08-31 00:16 ./usr/lib/ocf/
drwxrwxr-x 0/0               0 2020-08-31 00:16 ./usr/lib/ocf/resource.d/
drwxrwxr-x 0/0               0 2020-08-31 00:16 ./usr/lib/ocf/resource.d/assimilation/
-rwxrwxr-x 0/0            8670 2020-08-30 22:35 ./usr/lib/ocf/resource.d/assimilation/neo4j

it'll also figure out if its a systemd or non-systemd system and install the start up scripts appropriately

Alan-R commented 4 years ago

If you go back far enough I guess there were three styles of init:

systemV
upstart (short-lived alternative - ubuntu only?)
systemd

If it handles them all, then that's great! Ideally, I'd like to have only 3 versions of .deb, and a couple for RH, and a couple for SUSE - the fewer the better - or at least that's my first thought. Not sure how possible that is...

borgified commented 4 years ago

i think it handles all of them... here's the documentation: https://fpm.readthedocs.io/en/latest/source/pleaserun.html ive tested it on systemd and sysv and it seems to run fine. it says it also supports upstart but this i have not tested.

Alan-R commented 4 years ago

That's impressive. Sounds like I should approve it!

borgified commented 4 years ago

made a couple more improvements:

avoid hardcoding the version (based on docker/nanoprobe/docker-tag.txt)
automate checksum testing (based on /tmp/build/bin/sums which was generated during dockit)

one of the last things remaining is what you want the title of the release to say. atm, it's hardcoded to "fake release to test attaching binary" but we can base it off something like the title of a changelog

borgified commented 4 years ago

i need something akin to a changelog file from which i can use as the title/body of the release or we can skip that and just make manual modifications to the release description to get it to say what we want

borgified commented 4 years ago

@Alan-R ready to merge if we dont wanna include the changelog into the release title/text

Alan-R commented 4 years ago

I'm going to merge. Can you create a dummy changelog and make the changes to incorporate it?

borgified commented 4 years ago

changelog would need to have a specific format so i can write a script to awk/grep specific lines from it to use as input for the github release. thats why i wanted to ask you for it.

borgified commented 4 years ago

something like this could work: https://raw.githubusercontent.com/optimizely/go-sdk/master/CHANGELOG.MD

assimilation / assimilation-official

test packaging #87