FreshPorts was unable to extract/find any pkg message (where files/pkg-message.in exists), pkg-message: (nothing, where pkg-message exists)

[grahamperrin]

For example:

https://github.com/freebsd/freebsd-ports/blob/main/x11/nvidia-driver/files/pkg-message.in

www https://www.freshports.org/x11/nvidia-driver/#message:

FreshPorts was unable to extract/find any pkg message

The same at dev …

[dlangille]

The extract value is PKGMESSAGE, obtained like this:

[21:56 mydev dan /usr/ports/x11/nvidia-driver] % make -V PKGMESSAGE 
/var/ports/usr/ports/x11/nvidia-driver/work/pkg-message

However, in the /jail/freshports/ jail, we get:

root@freshports:/usr/ports/x11/nvidia-driver # make -V PKGMESSAGE 
/usr/ports/x11/nvidia-driver/work/pkg-message
root@freshports:/usr/ports/x11/nvidia-driver # 

Which matches the logs for that commit:

Grabbing make -V PKGMESSAGE
$pkgmessagepath='/usr/ports/x11/linux-nvidia-libs/pkg-message'
about to invoke '/usr/local/bin/sudo /usr/sbin/jexec freshports /realpath.sh /usr/ports/x11/linux-nvidia-libs/pkg-message'
results are '0'
errorcode='256'
$RealPKGMESSAGEPath='0'
pkgmessagepath does look like a valid file to me: '/usr/ports/x11/linux-nvidia-libs/pkg-message' (result of make -V PKGMESSAGE)
but _GetRealPath() claims that file does not exist. Perhaps it is '*/work/pkg-message.in' or similar
No, that is not a /work/ pkg-message. There is no pkg-message for this port at all

So FreshPorts is not being told about files/pkg-message.in

[dlangille]

also an issue with https://dev.freshports.org/www/angie-module-testcookie

Note that https://dev.freshports.org/www/angie-module-testcookie says: FreshPorts was unable to extract/find any pkg message

[grahamperrin]

A comparable example, pkg-message (not files/pkg-message.in):

https://github.com/freebsd/freebsd-ports/blob/main/multimedia/webcamd/pkg-message

At www https://www.freshports.org/multimedia/webcamd/#message:

pkg-message:

Screenshot:

[dlangille]

https://dev.freshports.org/multimedia/webcamd/#message is good.

NOTE: From here down is a separate issue.

[dlangille]

Interesting. The expected values are in the prod database:

freshports.org=> select element_pathname(element_id), pkgmessage from ports where id = (select id from ports_active where name = 'webcamd');
        element_pathname        |                              pkgmessage                               
--------------------------------+-----------------------------------------------------------------------
 /ports/head/multimedia/webcamd | [                                                                    +
                                | { type: install                                                      +
                                |   message: <<EOM                                                     +
                                | 1) add webcamd_enable="YES"                                          +
                                |                                                                      +
                                | to your /etc/rc.conf                                                 +
                                |                                                                      +
                                | 2) Please restart devd to start webcamd                              +
                                |                                                                      +
                                |         # service devd restart                                       +
                                |                                                                      +
                                | 3) Optionally add a user to the "webcamd" group                      +
                                |                                                                      +
                                |         # pw groupmod webcamd -m <username>                          +
                                |                                                                      +
                                | 4) If webcamd still did not start, consult the installed webcamd rc.d+
                                | script for more help and instructions on how to start webcamd.       +
                                | EOM                                                                  +
                                | }                                                                    +
                                | ]
(1 row)

freshports.org=> 

[dlangille]

Clearing the cache doesn't help:

[11:37 aws-1-nginx01 dvl ~] % cd ~freshports/cache/ports/          
[11:37 aws-1-nginx01 dvl ~freshports/cache/ports] % sudo rm -rf multimedia/webcamd
[11:37 aws-1-nginx01 dvl ~freshports/cache/ports] % sudo rm -rf multimedia/webcamd
[11:37 aws-1-nginx01 dvl ~freshports/cache/ports] % sudo grep -r webcamd_enable  multimedia/webcamd
grep: multimedia/webcamd: No such file or directory
[11:38 aws-1-nginx01 dvl ~freshports/cache/ports] % sudo ls -l multimedia/webcamd     
ls: multimedia/webcamd: No such file or directory
[11:38 aws-1-nginx01 dvl ~freshports/cache/ports] % sudo ls -l multimedia/webcamd
total 3
-rwxrwxr--  1 www freshports    334 Mar 16 11:38 Detail.Part1.head.PageSize500.PageNum1.html
-rwxrwxr--  1 www freshports  10648 Mar 16 11:38 Detail.Part2.head.PageSize500.PageNum1.html
-rwxrwxr--  1 www freshports 164794 Mar 16 11:38 Detail.Part3.head.PageSize500.PageNum1.html
[11:38 aws-1-nginx01 dvl ~freshports/cache/ports] % sudo grep -r webcamd_enable  multimedia/webcamd
[11:38 aws-1-nginx01 dvl ~freshports/cache/ports] % 

[dlangille]

Data is correct in dvl, dev, test, & stage.

Data displays only on dev - which prompts me to search for uncommitted changes.

[dlangille]

No uncommitted code noticed.

[dlangille]

Next on fact collection: If pkgmessage is empty, you should see FreshPorts was unable to extract/find any pkg message:

re: https://github.com/FreshPorts/freshports/blob/2.9/classes/port-display.php#L1690

[dlangille]

I added some debugging output. Note the extra commas at the start on test.

From dev:

string(420) "[ { "type": "install", "message": "1) add webcamd_enable=\"YES\"\n\nto your /etc/rc.conf\n\n2) Please restart devd to start webcamd\n\n\t# service devd restart\n\n3) Optionally add a user to the \"webcamd\" group\n\n\t# pw groupmod webcamd -m \n\n4) If webcamd still did not start, consult the installed webcamd rc.d\nscript for more help and instructions on how to start webcamd." } ] "

From test:

string(424) ", [ , { "type": "install", "message": "1) add webcamd_enable=\"YES\"\n\nto your /etc/rc.conf\n\n2) Please restart devd to start webcamd\n\n\t# service devd restart\n\n3) Optionally add a user to the \"webcamd\" group\n\n\t# pw groupmod webcamd -m \n\n4) If webcamd still did not start, consult the installed webcamd rc.d\nscript for more help and instructions on how to start webcamd." } ] "

[dlangille]

The problem is related to libucl-0.9.0 - which is installed on the webservers - this is not an ingress issue. It is a display issue only. However, as shown below, the data-extraction-from-Makefile occurs on the ingress nodes - and needs pkg and a working DNS.

dev was running libucl-0.8.2 and did not have the issue.

Updated the packages to libucl-0.9.0 and now dev has the issue.

[grahamperrin]

https://github.com/FreshPorts/freshports/issues/543#issuecomment-1960020647

Should https://dev.freshports.org/www/angie-module-testcookie#message display a package message?

(Sorry, I'm slightly confused.)

[dlangille]

Problem found today on test-ingress01: dns is not working:

root@freshports:/usr/ports/www/angie-module-testcookie # make extract apply-slist -DNO_DIALOG
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
===>   angie-module-testcookie-1.3.2 depends on file: /usr/local/sbin/pkg - not found
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
=> freebsd-pkg-1.20.9_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/freebsd/pkg/tar.gz/1.20.9?dummy=/freebsd-pkg-1.20.9_GH0.tar.gz
fetch: https://codeload.github.com/freebsd/pkg/tar.gz/1.20.9?dummy=/freebsd-pkg-1.20.9_GH0.tar.gz: Host does not resolve
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/freebsd-pkg-1.20.9_GH0.tar.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/freebsd-pkg-1.20.9_GH0.tar.gz: Host does not resolve
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/ and try again.
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/www/angie-module-testcookie
*** Error code 1

Stop.
make: stopped in /usr/ports/www/angie-module-testcookie
root@freshports:/usr/ports/www/angie-module-testcookie # host distcache.FreeBSD.org
^C

[dlangille]

This fixed that:

[12:59 test-ingress01 dvl ~] % sudo cp -inp /etc/resolv.conf /jails/freshports/etc 
[13:00 test-ingress01 dvl ~] % sudo jexec freshports                              
root@freshports:/ # host distcache.FreeBSD.org
distcache.freebsd.org is an alias for distcache.geo.FreeBSD.org.
distcache.geo.FreeBSD.org has address 96.47.72.71
distcache.geo.FreeBSD.org has IPv6 address 2610:1c1:1:606c::50:1
distcache.geo.freebsd.org mail is handled by 0 .

[dlangille]

However, we are now getting complicated... I don't want to build ports in order to extract information.

root@freshports:/usr/ports/www/angie-module-testcookie # make extract apply-slist -DNO_DIALOG
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
===>   angie-module-testcookie-1.3.2 depends on file: /usr/local/sbin/pkg - not found
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
===> Fetching all distfiles required by pkg-1.20.9_1 for building
===>  Extracting for pkg-1.20.9_1
===>  License BSD2CLAUSE accepted by the user
===> Fetching all distfiles required by pkg-1.20.9_1 for building
=> SHA256 Checksum OK for freebsd-pkg-1.20.9_GH0.tar.gz.
===>  Patching for pkg-1.20.9_1
===>  Applying FreeBSD patches for pkg-1.20.9_1 from /usr/ports/ports-mgmt/pkg/files
===>  Configuring for pkg-1.20.9_1
No installed jimsh or tclsh, building local bootstrap jimsh0
./autosetup/autosetup-find-tclsh: ./jimsh0: not found
No working C compiler found. Tried echo and gcc.
exec: -o jimsh0 ./autosetup/jimsh0.c
false: not found
===>  Script "configure" failed unexpectedly.
Please report the problem to pkg@FreeBSD.org [maintainer] and attach the
"/usr/ports/ports-mgmt/pkg/work/pkg-1.20.9/config.log" including the output
of the failure of your make command. Also, it might be a good idea to provide
an overview of all packages installed on your system (e.g. a
/usr/ports/ports-mgmt/pkg/work/pkg-1.20.9/src/pkg-static info -g -Ea).
*** Error code 1

Stop.
make[4]: stopped in /usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/www/angie-module-testcookie
*** Error code 1

Stop.
make: stopped in /usr/ports/www/angie-module-testcookie
root@freshports:/usr/ports/www/angie-module-testcookie # 

[dlangille]

Installing pkg fixes that. Perhaps this is a new minimum level.

root@freshports:/usr/ports/www/angie-module-testcookie # pkg install pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
pkg-static: sqlite error while executing ALTER TABLE packages ADD licenselogic INTEGER NOT NULL DEFAULT(1); in file pkgdb.c:2353: no such table: packages
root@freshports:/usr/ports/www/angie-module-testcookie # rm -rf /var/db/pkg
root@freshports:/usr/ports/www/angie-module-testcookie # pkg install pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[freshports] Installing pkg-1.21.3...
[freshports] Extracting pkg-1.21.3: 100%
Updating FreeBSD repository catalogue...
[freshports] Fetching meta.conf: 100%    178 B   0.2kB/s    00:01    
[freshports] Fetching data.pkg: 100%    7 MiB   7.3MB/s    00:01    
Processing entries: 100%
FreeBSD repository update completed. 34072 packages processed.
All repositories are up to date.
Updating database digests format: 100%
Checking integrity... done (0 conflicting)
The most recent versions of packages are already installed
root@freshports:/usr/ports/www/angie-module-testcookie # make extract apply-slist -DNO_DIALOG
===> Skipping 'config' as NO_DIALOG is defined
===>  License BSD2CLAUSE accepted by the user
===>   angie-module-testcookie-1.3.2 depends on file: /usr/local/sbin/pkg - found
=> kyprizel-testcookie-nginx-module-64137c2f7e66e8866f56fdecaaf09ac10a034da9_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/kyprizel/testcookie-nginx-module/tar.gz/64137c2f7e66e8866f56fdecaaf09ac10a034da9?dummy=/kyprizel-testcookie-nginx-module-64137c2f7e66e8866f56fdecaaf09ac10a034da9_GH0.tar.gz
fetch: https://codeload.github.com/kyprizel/testcookie-nginx-module/tar.gz/64137c2f7e66e8866f56fdecaaf09ac10a034da9?dummy=/kyprizel-testcookie-nginx-module-64137c2f7e66e8866f56fdecaaf09ac10a034da9_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/kyprizel/testcookie-nginx-module/tar.gz/64137c2f7e66e8866f56fdecaaf09ac10a034da9?dummy=/kyprizel-testcookie-nginx-module-64137c2f7e66e8866f56fdecaaf09ac10a034da9_GH0.tar.gz: size of remote file is not known
kyprizel-testcookie-nginx-module-64137c2f7e66e          20 kB 2708 kBps    00s
===> Fetching all distfiles required by angie-module-testcookie-1.3.2 for building
===>  Extracting for angie-module-testcookie-1.3.2
=> SHA256 Checksum OK for angie-1.3.2.tar.gz.
=> SHA256 Checksum OK for kyprizel-testcookie-nginx-module-64137c2f7e66e8866f56fdecaaf09ac10a034da9_GH0.tar.gz.

[dlangille]

Seems OK now at https://test.freshports.org/www/angie-module-testcookie#message

[dlangille]

Now fixed on dev/test/stage/prod.

Fixed by rerunning the commit. We could have waited until the next commit....