DocDB in AlmaLinux 9

[wgseligman]

I tried to upgrade my DocDB installation to run in AlmaLinux 9. I did not succeed. The main problem is that AlmaLinux 9 runs perl 5.32, which has some major differences vs. the perl 5.16 installation of CentOS 7.

I know that the CentOS 7 end-of-life is still about a year away, and from personal experience I know the codebase upgrade won't be easy. I'm primarily noting this issue so that when you're able to get to it, I'll receive a notification.

[pms967]

I'm running DocDB (8.8.7 + a few assorted patches) on Ubuntu 22.04, which has Perl 5.34 One of the most important modification I had to apply to make it work without too many changes to the code ( see Issue #107 ) was to add the following:

# fix for obsolete assumption of CWD in @INC
use FindBin 1.51 qw( $RealBin );
use lib $RealBin;

to all the Perl files where it was needed.

Other minor changes were needed as a consequence of the much newer CGI module, too.

P.S.: I have been running the same since Ubuntu (server) 16.04, and then all the subsequent LTS releases (18.04 and 20.04) up to now. Will see how it goes with the next one...

[wgseligman]

I figured that part out too, though my solution was dumber; I put the following at the top of all the scripts, use before require "DocDBGlobals.pm";

use lib ".";

While this worked, there were issues with perl hashes (perhaps due to the CGI module, as you said) that caused crashes that I could not debug. For example, when trying to upload a document, the 'submitter' and 'author' lists were blank; this could also have been a database issue, since I'm not facile with MySQL and moving databases between servers.

[pms967]

Make sure to apply this one, too: ( issue #13 )

https://github.com/ericvaandering/DocDB/pull/13/commits/ab18bbcde5dee36010161202b67ce1389c60b7a1

I'll check for other relevant mod I had to do and post here later.

[pms967]

This was required for the newer libCGI:

--- DocDBGlobals.pm.orig        2016-07-12 16:15:26.391099458 +0200
+++ DocDBGlobals.pm     2021-05-28 14:39:37.818916058 +0200
@@ -302,4 +302,18 @@
   $Tar = $GTar;
 }

+# Added by Paolo Saggese, Mar 21, 2017
+# fix for obsolete startform/endform
+package CGI;
+our %EXPORT_TAGS;
+$EXPORT_TAGS{':standard'} = [ @{$EXPORT_TAGS{':standard'}},
+                                  'startform', 'endform' ];
+sub startform {
+        &start_form;
+    }
+sub endform {
+        &end_form;
+    }
+
+
 1;

Edit: but no more - at least since DocDB 8.8.9 the relevant changes have been included in the official code.

[wgseligman]

I started from a fresh DocDB script installation and applied the changes you suggested. However, it did not solve my problems. Apart from difficulties in fully listing all the documents associated with authors and topics, uploading a new document was not possible; the Submitter and Author fields were blank, so I could not select a valid choice.

Unfortunately, there are no error messages associated with these problems in neither the main server logs, the web server logs, nor the MySQL logs. I can't trace anything.

It appears that there's something else that's funky about AlmaLinux 9, aside from the different perl version, that affects DocDB scripts.

[pms967]

You may try to add:

use CGI::Carp 'fatalsToBrowser';
use warnings;

to the scripts to help debugging.

[wgseligman]

Sigh This is turning into a massive project on the part of someone (me) who doesn't know these perl libraries. My main question stands: Does someone who knows the DocDB package plan to work on AlmaLinux 9 compatibility, given that both FNAL and CERN plan to switch to AlmaLinux 9 by the end of this year?

[pms967]

Issue #113 is one of the many that will likely affect AlmaLinux 9 too.

[wgseligman]

Yes, I had that problem too. However, the problems I had elsewhere (not all documents showing in lists, entry problems on the New Document page) seemed of higher priority.

[pms967]

Yes, I had that problem too. However, the problems I had elsewhere (not all documents showing in lists, entry problems on the New Document page) seemed of higher priority.

well, of course. You should provide at least the relevant log file errors to try to understand what's goin' on...

[wgseligman]

There are no errors in any of the logfiles. That's part of the problem.

You offered a way of generating diagnostics in a previous reply, but my position is the same: This leads me down a rabbit hole of tracing through uncommented Perl code. I'd rather that someone familiar with the package do this work.

I've already written to the Fermilab Computing Division. They are the ones who made the decision to switch to AlmaLinux 9 (a decision I agree with), and they also have many DocDB installations among all their experiments. I'm sure they'll figure something out, and I'll copy-cat their solution (even if it's to move to CERN's Indico).

I just want to know when that happens!

[pms967]

There are no errors in any of the logfiles. That's part of the problem.

could it be that you have the apache (or whichever web server you're using) logging options set too quiet?

You offered a way of generating diagnostics in a previous reply, but my position is the same: This leads me down a rabbit hole of tracing through uncommented Perl code. I'd rather that someone familiar with the package do this work.

indeed.

(even if it's to move to CERN's Indico).

LOL. I doubt it, Indico does a completely different job... :D

BTW, we do use Indico as well. For managing conferences, meetings, etc. But we use DocDB for storing our papers, reports, presentations and so on.

[pms967]

For the record: yet another problem which is relevant for AL9, too: Issue #114

[wgseligman]

With your changes in issues 113 and 114, my copy of DocDB on AlmaLinux 9 seems "closer" than ever. Lists of documents now appear to be correct. Unfortunately, there's still one big issue: adding new documents.

When I try to add a new document in the AL9 version with your changes, this is what I see:

https://www.dropbox.com/scl/fi/p02niaj7jy32wg87kdemr/DocDB-AddDocument-AL9.png?rlkey=365zq1q9lnzbcfabiv9v9aias&dl=0

In contrast, this is the page I see for the same database in the CentOS 7 version:

https://www.dropbox.com/scl/fi/27xvreeojuj11si28m5n2/DocDB-AddDocument-C7.png?rlkey=9z4t1yogzcp17ypjppt27d6qf&dl=0

On the AlmaLinux 9 page, I can try to type in my account name manually the "Submitter" and "Author" fields, but the when I click the "Submit Document" button I get:

There were fatal errors processing your request:
    You must supply a submitter for this document.
    You must supply at least one author for this document. 

Needless to say, this is not a problem on the CentOS 7 version of the page.

Do you see this same problem?

[pms967]

When I try to add a new document in the AL9 version with your changes, this is what I see:

In contrast, this is the page I see for the same database in the CentOS 7 version:

that's just a configurable option. Edit the file "DocDBGlobals.pm", and look for the following (which is the default on new installations):

$Preferences{Topics}{Selector}           = "tree";   # tree, multi, or single
$Preferences{Topics}{NColumns}           = 3;        # number of columns in the topic table
$Preferences{Authors}{Selector}          = "active"; # active, list, or field

To get the same as you had, you may want to change as follows:

$Preferences{Topics}{Selector}           = "multi"; # tree, multi, or single
$Preferences{Topics}{NColumns}           = 4;       # number of columns in the topic table
$Preferences{Authors}{Selector}          = "list";  # active, list, or field

On the AlmaLinux 9 page, I can try to type in my account name manually the "Submitter" and "Author" fields,

on my system, it automatically suggests a list of authors as soon as you start typing. Does it work that way on your AL9 system, too?

but the when I click the "Submit Document" button I get:

There were fatal errors processing your request: You must supply a submitter for this document. You must supply at least one author for this document.

Do you see this same problem?

If I choose the "active" selector, as you had, yes, I have some troubles too. But, fortunately, changing the selector to "list":

it does work just fine:

...yet another issue to submit. :-)

[pms967]

Meanwhile, there's also this one which will likely affect AL9 too: Issue #115

[wgseligman]

Hmm... things seem to be working now thanks to your work.

I don't want to rush anything, especially since I'd have to convert the CentOS database to an AlmaLinux 9 database (for the fifth time). I'll wait a week or so to see if you discover anything else.

In the meantime: Thanks for all your hard work! I hope FNAL and the other institutions who use DocDB recognize what you're doing.

[wgseligman]

I never answered one of your questions: No, when I had the "active" option, trying to fill in the Submitter/Author field did not show a list of options. But we're used to the "list" format, and I don't think anyone wants me to change that.

[pms967]

I never answered one of your questions: No, when I had the "active" option, trying to fill in the Submitter/Author field did not show a list of options.

that's weird. Could it be that you have Javascript disabled on your browser? Or maybe that you have not properly setup your web server and/or the DocDB folders dir tree? It looks like DocDB's javascripts are not working.

But we're used to the "list" format, and I don't think anyone wants me to change that.

well, sure, that's fine. But, beside that one (which may be not a problem for you), if the required JS libraries are not loaded there may be several other problems and missing features (e.g., does the context help system work? Can you sort the document lists as you like by clicking on the table headers?).

From your browser, try to right click on DocDB main page, and select "view page source". At the top you should see a few lines which begins with: <link rel="stylesheet" and <script type="text/javascript", followed by an URL (pointing to some paths on your own DocDB server).

Try opening those URLs (all of them, one by one). You should see the contents of such files. If you get a "404 Not found" or any other error on any of them, you have a problem with your setup which you should fix.

BTW: if you need to show some screenshot (or any other image) here, there's no need to upload it on an external site and paste here the link. Just copy+paste the image here on the editor, and watch the "magic" happen... that's easier and much better. ;-)

[pms967]

More troubles: can't create new (externally managed) events. It fails with error:

AH01215: DBD::mysql::st execute failed: Incorrect integer value: '' for column 'ShowAllTalks' at row 1 at /var/www/docdb/cgi.pro/MeetingSQL.pm line 517. referer: docdb/cgi/MeetingModify

Solved reverting to an older version of that file:

--- .bak/MeetingSQL.pm.orig.bad 2023-08-19 18:38:05.265992984 +0200
+++ MeetingSQL.pm       2023-07-12 16:15:26.399099483 +0200
@@ -354,10 +354,15 @@

   # FIXME: These next three lines are good for MySQL >= 4, but not MySQL 3

-  my $SessionList   = $dbh -> prepare(
-    "select SessionID from Session where DATE(StartTime)=?");
-  $SessionList -> execute($Date);
+#  my $SessionList   = $dbh -> prepare(
+#    "select SessionID from Session where DATE(StartTime)=?");
+#  $SessionList -> execute($Date);
+
+  # FIXME: These next three lines are good for MySQL 3

+  my $SessionList   = $dbh -> prepare(
+    "select SessionID from Session where StartTime like ?");
+  $SessionList -> execute($Date."%");
   $SessionList -> bind_columns(undef, \($SessionID));
   while ($SessionList -> fetch) {
     $SessionID = FetchSessionByID($SessionID);
@@ -499,7 +504,7 @@
   my $Location         = exists $ArgRef->{-location}         ?   $ArgRef->{-location}         : "";
   my $AltLocation      = exists $ArgRef->{-altlocation}      ?   $ArgRef->{-altlocation}      : "";
   my $URL              = exists $ArgRef->{-url}              ?   $ArgRef->{-url}              : "";
-  my $ShowAllTalks     = exists $ArgRef->{-showalltalks}     ?   $ArgRef->{-showalltalks}     : 0;
+  my $ShowAllTalks     = $ArgRef->{-showalltalks} > ''      ?   $ArgRef->{-showalltalks}     : 0;
   my $Preamble         = exists $ArgRef->{-preamble}         ?   $ArgRef->{-preamble}         : "";
   my $Epilogue         = exists $ArgRef->{-epilogue}         ?   $ArgRef->{-epilogue}         : "";
   my @TopicIDs         = exists $ArgRef->{-topicids}         ? @{$ArgRef->{-topicids}}        : ();
@@ -540,8 +545,6 @@
   my $AltLocation      = exists $ArgRef->{-altlocation}      ?   $ArgRef->{-altlocation}      : "";
   my $URL              = exists $ArgRef->{-url}              ?   $ArgRef->{-url}              : "";
   my $ShowAllTalks     = exists $ArgRef->{-showalltalks}     ?   $ArgRef->{-showalltalks}     : 0;
-    # print "<p>DEBUG: ShowAllTalks = $ShowAllTalks</p>\n";
-    if ($ShowAllTalks eq '') { $ShowAllTalks = 0 };
   my $Preamble         = exists $ArgRef->{-preamble}         ?   $ArgRef->{-preamble}         : "";
   my $Epilogue         = exists $ArgRef->{-epilogue}         ?   $ArgRef->{-epilogue}         : "";
   my @TopicIDs         = exists $ArgRef->{-topicids}         ? @{$ArgRef->{-topicids}}        : ();

[wgseligman]

We don't use externally managed events at our site, which is why I never picked up on this problem. Thanks for the fix!