Question: Synth in Jails

[basbebe]

Maybe this question is stupid and I misunderstood the purpose of Synth. I'm using Jails to separate my different services. Some of them have custom options, so I used to compile them using Postmaster (Dovecot, Postfix, NGINX, etc.). Synth seemed to be an amazing new tool to replace Portmaster. But as I learned, synth doesn't work inside Jails:

mount: tmpfs: Operation not permitted

raised REPLICANT.SCENARIO_UNEXPECTED : /sbin/mount -t tmpfs tmpfs /usr/obj/synth-live/SL09 => failed with code 1

Is there a way to use Synth's capabilities with Jails? Or did I misunderstand something completely?

[jrmarino]

No, it won't work in jail as you've seen by the tmpfs limitation. But why does it need to be executed in a jail? It's creating its only jail-like environment inside. maybe you just need to have the jails access the resultant repository? (In other words: build outside the jail, install the packages from inside the jail)

[basbebe]

That sounds like a nice solution (something like poudriere would do?) – yet I'm not sure how I would make the repository that Synth creates available to the Jail from the host system.

[jrmarino]

null mount host directory to jail directory?

[basbebe]

/var/synth/live_packages?

[jrmarino]

sure. create /packages in jail area and null mount /var/syn/live_packages to it before starting jail.

There are other solutions but that should work I would think.

[jrmarino]

(and then create a pkg conf file to look there)

[jrmarino]

e.g.

ironwolf# cat /usr/local/etc/pkg/repos/00_synth.conf
# Automatically generated.

Synth: {
  url      : file:///packages,
  priority : 0,
  enabled  : yes,
}

[basbebe]

what would then be the correct command to build them without installing them on the base system?

I assume I could create a text file jail_ports.txt:

www/nginx
mail/dovecot2
[…]

and do synth just-build jail_ports.txt?

[jrmarino]

synth just-build for all and on the last package, "synth build" and answer "yes" to rebuild repository, "no" to install packages

[jrmarino]

in other words, never rebuild repository until the last package is built

[jrmarino]

in the case of "jail_ports.txt" then use "synth build" and answer the questions yes, no

[basbebe]

But if I then did "upgrade-system" the next time, all the ports that I have configured for my jails would also be installed on the host, right?

[jrmarino]

no. Upgrade system checks what is installed. For example, you have 5000 packages built in local repository, and 20 packages installed on host. When you "upgrade system", it's only going to consider the 20 packages already installed, no matter how many are available in the repository. And it only installs if the repository versions are newer.

[basbebe]

I'm getting more and more excited about this ;-) Is there a way to create a cronjob to synth build jail_ports.txt – yes, no? And is this only going to rebuild those ports that have newer versions / dependencies available?

[jrmarino]

you could do some kind of kludge by making a script that rules "/usr/local/bin/synth just-build < file >" followed by "/usr/local/bin/synth install < some small port >". it would rebuild the repository and then do nothing because the small port would already be up to date.

Synth doesn't have a "build-then-repo" command at the moment and that's what you would really need. OTOH, you could just run "just-build" in cron and then manually run a "build" command later to force repo to build. it would only take a few minutes and I'm assuming you aren't installing packages via cron.

[jrmarino]

"And is this only going to rebuild those ports that have newer versions / dependencies available?"

yes, but one change (e.g. to perl) can cause hundreds of ports to rebuild.

[basbebe]

Thank you! A lot.

[jrmarino]

no problem. feel free to reopen if you want to continue conversation.

[basbebe]

could a list file also include comments?

[jrmarino]

no, it can't. it will choke on them.

[basbebe]

I stumbled upon this:

synth build /usr/local/etc/jail_ports.txt
Stand by, updating external repository catalogs ... done.
Scanning existing packages.
After inspection, it has been determined that there are no packages that
require rebuilding; the task is therefore complete.
Would you like to rebuild the local repository (Y/N)? y
Scanning entire ports tree.
 progress: 98.34%

raised ADA.IO_EXCEPTIONS.NAME_ERROR : unknown directory "All"

This is my /usr/local/etc/jail_ports.txt

devel/git
lang/php56
lang/php56-extensions
lang/python3
mail/dovecot2-pigeonhole
mail/dovecot2-antispam-plugin
mail/opendkim
mail/opendmarc
mail/postfix-current
mail/rmilter
mail/rspamd
mail/ssmtp
security/ca_root_nss
security/clamav
security/openssl
www/uwsgi
www/nginx
www/nginx-devel

After that, my repo must have gotten broken or something. Now Synth also wants to rebuild the ports that already were build before.

[jrmarino]

do you have an errant directory named /usr/ports/All ?

if not, use "find" command and figure out where "All" directory is.

[basbebe]

This is my result: several of these (symlinked to /var/synth/live_packages/All):

/usr/jails/[JAILNAME]/var/synth/live_packages/All

and these:

/usr/obj/synth-live/SL01/packages/All
/var/synth/live_packages/All

[jrmarino]

unfortunately, that's not interesting. Can you paste the contents of /usr/local/etc/synth/synth.ini ?

[basbebe]

; This Synth configuration file is automatically generated
; Take care when hand editing!

[Global Configuration]
profile_selected= LiveSystem

[LiveSystem]
Operating_system= FreeBSD
Directory_packages= /var/synth/live_packages
Directory_repository= /var/synth/live_packages/All
Directory_portsdir= /usr/ports
Directory_options= /var/db/ports
Directory_distfiles= /usr/ports/distfiles
Directory_buildbase= /usr/obj/synth-live
Directory_logs= /var/log/synth
Directory_ccache= /var/cache/ccache
Directory_system= /
Number_of_builders= 2
Max_jobs_per_builder= 2
Tmpfs_workdir= true
Tmpfs_localbase= true
Display_with_ncurses= true
leverage_prebuilt= true

[jrmarino]

"several of these (symlinked to /var/synth/live_packages/All):"

symlinked or null mounted?

what happens when you "ls" /var/synth/live_packages/All ?

[basbebe]

sorry! null mounted! mount_nullfs -o r /var/synth/live_packages /usr/jails/[JAILNAME]/var/synth/live_packages

ls /var/synth/live_packages/All gives me a long list of packages.

[jrmarino]

it seems to be that it completed the port tree scan It was probably starting the scan of /var/synth/live_packages/All for some reason it says that's a bad directory, but maybe it was a permissions problem because the directory obviously exists. But you run as root, so that's probably not it either. I'm a bit stumped.

The bit about "it wants to build all the packages again" might be related. Is "All" readable?

[basbebe]

drwxr-xr-x  2 root  wheel   192B 28 Jan 21:39 All

[jrmarino]

is it repeatable? try to build one port with "build" and answer "yes" to rebuild repository. It can be one that's already been rebuilt.

[basbebe]

I did a rebuild-repository and then a just-build, The latter on the file. Both worked

[jrmarino]

so it's not repeatable?
in other words, it's not happening anymore?

[basbebe]

No, it's not happening anymore. Also a synth build /usr/local/etc/jails_ports.txt ran fine.