PHPldapadmin install failed, jail image generation

[n-connect]

Hello CBSD team,

After a successful install of Redmine jail, tried to install following your ldapsrv link phpldapadmin, or openldap or even ldapsrv, but run into the same errors [root@hots ~]# cbsd repo action=get sources=img name=phpldapamin REPO: https://bsdstore.ru fetch: https://bsdstore.ru/img/amd64/amd64/13.2/phpldapamin/mirror.html: Not Found retry: 1/2 fetch: https://bsdstore.ru/img/amd64/amd64/13.2/phpldapamin/mirror.html: Not Found retry: 2/2 Fetching phpldapamin jail: https://bsdstore.ru/ fetch: https://bsdstore.ru/img/amd64/amd64/13.2/phpldapamin/phpldapamin.img: Not Found retry: 1/2 fetch: https://bsdstore.ru/img/amd64/amd64/13.2/phpldapamin/phpldapamin.img: Not Found retry: 2/2 No such remote file or network problem

[root@host ~]# cbsd repo action=get sources=img name=openldap REPO: https://bsdstore.ru fetch: https://bsdstore.ru/img/amd64/amd64/13.2/openldap/mirror.html: Not Found retry: 1/2 fetch: https://bsdstore.ru/img/amd64/amd64/13.2/openldap/mirror.html: Not Found retry: 2/2 Fetching openldap jail: https://bsdstore.ru/ fetch: https://bsdstore.ru/img/amd64/amd64/13.2/openldap/openldap.img: Not Found retry: 1/2 fetch: https://bsdstore.ru/img/amd64/amd64/13.2/openldap/openldap.img: Not Found retry: 2/2 No such remote file or network problem

[root@host ~]# cbsd repo action=get sources=img name=ldapsrv REPO: https://bsdstore.ru fetch: https://bsdstore.ru/img/amd64/amd64/13.2/ldapsrv/mirror.html: Not Found retry: 1/2 fetch: https://bsdstore.ru/img/amd64/amd64/13.2/ldapsrv/mirror.html: Not Found retry: 2/2 Fetching ldapsrv jail: https://bsdstore.ru/ fetch: https://bsdstore.ru/img/amd64/amd64/13.2/ldapsrv/ldapsrv.img: Not Found retry: 1/2 fetch: https://bsdstore.ru/img/amd64/amd64/13.2/ldapsrv/ldapsrv.img: Not Found retry: 2/2 No such remote file or network problem

Checked out the marketplace templates, phpldapadmin is not there (and Redmine is missing since some days too). Based on your repositories, it seems the marketplace jails has a repository named modules-forms-* like modules-forms-redmine

How should I install / build a jail image for phpldap from this repository (and for Redmine from its own repo - because it is missing now :) )?

If I "clone" locally a sampled modules-forms repository and apply the phpldapamin' existing files' contents and merge them all it work? I did not found details about how to create new jail image yet.

Also there's another solutions which I would make additional jail images for future use.

Thanks

[olevole]

Hi. The ldap image was left due to lack of support - also, this image was never transferred to the CBSDfile, which is currently the source for generating the image library at https://marketplace.bsdstore.ru ( source: https://github.com/cbsd/cbsdfile-recipes/tree/master/jail )

As for redmine, recently I drop all images manually (it was a necessary measure). I'll check the logs for the redmine on the infrastructure build soon.

The main idea is to periodically (every few weeks) use the CBSDfile to update and generate the image. At the same time, we use a 'tests' directory of tests to pass the specified criteria. This means that the image will not be generated if the tests fail, e.g.:

https://github.com/cbsd/cbsdfile-recipes/tree/master/jail/pgadmin4/tests https://github.com/cbsd/cbsdfile-recipes/tree/master/jail/gitlab/tests ..

This is exactly what happened to Redmine image - Redmine service did not pass the build and/or tests ( the fact that the last working redmine build was deleted -- is my fault ;-).

[n-connect]

@olevole

Thank you for the details. I hope the Redmine build issues are not a big hurdle. Based this logic, it is possible there will be build issues in the future, especially with softwares with breaking changes. Just had my fair share about it recently with a new solution just changed version. Should I make regular backups of images, or the baseline is to keep the previous succesfull builds? :)

Is there a way locally testing out new CBSDfile based new jails/build jail images before it could/would go into https://github.com/cbsd/cbsdfile-recipes/ as PR for example? I've just found this https://www.bsdstore.ru/en/articles/cbsd_puppet_jail_images.html at CBSD

[olevole]

in fact, I would like to configure Jenkins or any other CI so that everyone can see the build logs (and fix). As for image backups, I did not expect to delete the image and the latest working image should have remained accessible "by design" ;-)

As for Redmine, the problem at the moment was not in the cbsd scripts - the redmine package is not available from the official freebsd repositories:

cbsdbuilder133:/usr/home/freebsd@[21:49] # pkg install redmine50
Updating FreeBSD-latest repository catalogue...
Fetching packagesite.pkg: 100%    7 MiB   7.4MB/s    00:01    
Processing entries: 100%
FreeBSD-latest repository update completed. 33963 packages processed.
All repositories are up to date.
pkg: No packages available to install matching 'redmine50' have been found in the repositories

There is nothing on the bug tracker for redmine50 right now ( except https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277661 ). Perhaps this is some kind of temporary error and the package will be returned soon..

I welcome contributions of new images to the registry. I am currently using a rented server (using donor funds) for the needs of the CBSD project and the server has enough disk space to build a large number of images. However, as I said, I would like to make it more transparent so that everyone can see the logs and tasks.

You can build static templates while I prefer to use the power of the community and try to use configuration systems to reconfigure services. 'cbsd forms' do this - this is not such a trivial task as a static template, but it has incomparable advantages in terms of capabilities and further labor costs (in dynamic forms, as a rule, you will spend effort once when writing a layer). I wrote more about this approach here: https://freebsdfoundation.org/wp-content/uploads/2022/03/CBSD-Part-1-Production.pdf

theoretically, any framework can be taken instead of Puppet: (R)ex, Ansible, Chef, SaltStack ..

If you plan to contribute, try to write tests that will check that the service is working - look at existing 'tests' directory as examples.

To test locally, just use 'cbsd up', e.g. to build sambashare image:

git clone https://github.com/cbsd/cbsdfile-recipes.git
cd cbsdfile-recipes/jail/sambashare/
cbsd up

to test:

./tests/process.sh

(CI infrastructure automatically scans the contents of the 'tests/' directory and consistently run all the scripts in it. If at least one of them fails, the image will not be builded&uploaded to registry)

If you upload a new CBSDfile to the git repo, it will be automatically builded approximately once every 2 weeks for each stable freebsd ( 13.3, 14.0 ) version and published on https://marketplace.bsdstore.ru

I hope someday I can make a registry service ;)

[n-connect]

Thanks for the details.

I do remember I checked your PDF before, while we've decided which jail framework we should go with, re-reading the critical parts was best for this whole subject, thanks. I'll go with them, and go after the CBSDFile direction you shared above. Short time goal is to have a working ldap jail template, one way another. Right now I have made the need to work with a quick lldap install/config, but I think best to get the the phpldapadmin/openldap jail.

For Redmine the pkg issue may be already gone, one of the hosts shows this for redmine50 package:

[root@host ~]# date;pkg install redmine50
Tue Mar 19 20:05:50 CET 2024
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 265 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        ImageMagick7: 7.1.0.62_9
...
        redmine50: 5.0.7_2
        ruby: 3.1.4_1,1
        ruby31-gems: 3.4.20
...
Number of packages to be installed: 263
Number of packages to be upgraded: 2

The process will require 1 GiB more space.
284 MiB to be downloaded.

Proceed with this action? [y/N]: n

Checked the jail images availability directly, with Redis as a counter part. It seems 13.2, 14.0, 15 images are available to download directly, all from Jan 3, 2024. The 13.3 version missing only.

[root@host ~]# curl -R -O https://dl.bsdstore.ru/img/amd64/amd64/14.0/redis/redis.img
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 47.4M  100 47.4M    0     0  15.4M      0  0:00:03  0:00:03 --:--:-- 15.4M
[root@host ~]# curl -R -O https://dl.bsdstore.ru/img/amd64/amd64/13.2/redmine/redmine.img;mv redmine.img redmine-13.2.img
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  771M  100  771M    0     0  10.8M      0  0:01:10  0:01:10 --:--:-- 8152k
[root@host ~]# curl -R -O https://dl.bsdstore.ru/img/amd64/amd64/13.3/redmine/redmine.img;mv redmine.img redmine-13.3.img
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   146  100   146    0     0    231      0 --:--:-- --:--:-- --:--:--   231
mv: rename redmine.img to redmine-13.3.img: No such file or directory
[root@host ~]# curl -R -O https://dl.bsdstore.ru/img/amd64/amd64/14.0/redmine/redmine.img;mv redmine.img redmine-14.0.img
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  795M  100  795M    0     0  12.1M      0  0:01:05  0:01:05 --:--:-- 1146k
[root@host ~]# curl -R -O https://dl.bsdstore.ru/img/amd64/amd64/15.0/redmine/redmine.img;mv redmine.img redmine-15.0.img
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  796M  100  796M    0     0  10.4M      0  0:01:16  0:01:16 --:--:-- 12.6M
[root@host /]# ls -lah *img
-rw-r--r--  1 root  wheel    47M Mar  4 02:07 redis.img
-rw-r--r--  1 root  wheel   771M Jan  3 02:53 redmine-13.2.img
-rw-r--r--  1 root  wheel   795M Jan  3 19:53 redmine-14.0.img
-rw-r--r--  1 root  wheel   797M Jan  4 19:57 redmine-15.0.img

The cbsd repo command works on a 13.2 host and the image date in sync too:

[root@host ~]# cbsd repo action=get sources=img name=redmine usehelpers=1 runasap=1 newjname=redmine5
REPO: https://bsdstore.ru
Found new mirror for redmine : https://dl.bsdstore.ru
Fetching redmine jail: https://dl.bsdstore.ru/
retrieve redmine.img from dl.bsdstore.ru, size: 771m
/storage/cbsd/import/redmine5.img               0% of  771 MB 3917 kBps 03m18s
fetch: transfer interrupted
retry: 1/2
retrieve redmine.img from dl.bsdstore.ru, size: 771m
/storage/cbsd/import/redmine5.img               2% of  771 MB 4014 kBps 02m52s
fetch: transfer interrupted
retry: 2/2
0
pull ok:
Use img helpers: yes
Done... Importing...
Importing image, please stand by: redmine
CBSD Image, version: 4
Image was created on node: cbsdbuilder132.my.domain
Image was created at: 20240103
Image size: 16m
Environment flat size after extracting: 3g
Compress level: 6
Jail already exist: redmine. May be new_jname= args can help?
Import complete
Do you want to remove source image /storage/cbsd/import/redmine5.img ?
[yes(1) or no(0)]
1
No such jail: redmine5
No such jail: jname="redmine5"

So it seems the earlier images there even redmine not listed on marketplace currently, and the redmine50 pkg is good (at least 13.2) too. Please let me know if Redmine jail available again, found no subscription/RSS/etc. on the marketplace page :)

[olevole]

For Redmine the pkg issue may be already gone, one of the hosts shows this for redmine50 package:

Hmm, package exist in quarterly but not in latest. Temporary switch to quarterly - image was uploaded for FreeBSD 13.2 / 14.0 - https://marketplace.bsdstore.ru/

Please let me know if Redmine jail available again, found no subscription/RSS/etc. on the marketplace page :)

RSS? sounds like a nice feature request ;), noted.

[olevole]

@n-connect lldp -- https://github.com/lldap/lldap?tab=readme-ov-file#from-source ?

have you tried this on freebsd? If it works we can make a port and/or image

[n-connect]

Yes, there's a manual cross build, plus and rc.d scrip too. Will make some updates today on the binary and upload it.

Its deployable and works as a system service with non-privileged user, However its kind of a hack/not standard - its running from its own subdir, with additional other subdirs for web-ui and additonal other config-data, beside the main config file. This is a design comes with it, need finetunes one that area.

[n-connect]

For Redmine the pkg issue may be already gone, one of the hosts shows this for redmine50 package:

Hmm, package exist in quarterly but not in latest. Temporary switch to quarterly - image was uploaded for FreeBSD 13.2 / 14.0 - https://marketplace.bsdstore.ru/

Please let me know if Redmine jail available again, found no subscription/RSS/etc. on the marketplace page :)

Thank you!

RSS? sounds like a nice feature request ;), noted.

Yes an RSS or any other notification subscription would be nice, helps sharing the words when/how new or existing jails got listed, updated (or removed for any reason :) )

[n-connect]

@n-connect lldp -- https://github.com/lldap/lldap?tab=readme-ov-file#from-source ?

have you tried this on freebsd? If it works we can make a port and/or image

Yes, that's one. There's now a semi-automated, but locally working cross build. The connected FreeBSD release PR is ongoing, I need to fix/rewrite(?)/create a new static build process. Their existing static-build uses on-the-fly-downloaded static musl-cc binaries (Linux/Windows built CCs exists only) and therefore no Rust 'cross' available in their own custom CI image by design. Than comes their build flow script. Still reading after what's best, can I just insert the cross into some way.

Its deployable and works as a system service with non-privileged user, Minimal mem/cpu/disk usage - can coexist with other service in a jail.- CBSD forms already has this possibility, if I remember well.

However its kind of a hack/not standard - its running from its own subdir, with additional other subdirs for web-ui and additonal other config-data, beside the main config file. This is a design comes with it, need finetunes one that area.

Will make some updates today on the binary and upload it/share here later.

Edit: here's an updated x86_64-freebsd_lldap-0.5.1.tar.gz, basic install steps in the upload details .

Just run into APPJail on account of Wazuh in FreeBSD Jail, I'm sure you heard about it. So a Wazuh jail is now on my list too..