search

sympa-community / sympa

Sympa, Mailing List Management Software
https://www.sympa.community/sympa
GNU General Public License v2.0
243 stars 96 forks source link

Warnings during family instantiation #1583

Closed qosobrin closed 1 year ago

qosobrin commented 1 year ago

Version

6.2.20

Installation method

Debian Package

Expected behavior

Family instantiation should not cause any error or warning. A clean output is expected

Actual behavior

Family instantiation generates a number of warnings even though the family is correctly instantiated.

Steps to reproduce

I instantiate my family with this command: /usr/lib/sympa/bin/sympa.pl --instantiate_family titulaciones --robot example.domain.com --input_file /etc/sympa/example.domain.com/families/titulaciones/titulaciones.xml

Additional information

This is the output of the instantiation of the family:

Use of uninitialized value in subroutine entry at /usr/lib/sympa/bin/sympa.pl line 1746.
Use of uninitialized value in subroutine entry at /usr/lib/sympa/bin/sympa.pl line 1746.
Creating lists:   0% [                                                                                                                                                   ]
Use of uninitialized value in regexp compilation at /usr/share/sympa/lib/Sympa/List.pm line 5194, <GEN2> line 102.
Use of uninitialized value in regexp compilation at /usr/share/sympa/lib/Sympa/List.pm line 5194, <GEN2> line 102.
Use of uninitialized value in regexp compilation at /usr/share/sympa/lib/Sympa/List.pm line 5194, <GEN2> line 102.
Use of uninitialized value in join or string at /usr/share/sympa/lib/Sympa/List.pm line 4858.
List "list1" (1/2) created/updated
Creating lists:  50% [=========================================================================                                                                          ]0m03s Left
Use of uninitialized value in regexp compilation at /usr/share/sympa/lib/Sympa/List.pm line 5194, <GEN15> line 98.
Use of uninitialized value in regexp compilation at /usr/share/sympa/lib/Sympa/List.pm line 5194, <GEN15> line 98.
Use of uninitialized value in join or string at /usr/share/sympa/lib/Sympa/List.pm line 4858.
List "list2" (2/2) created/updated
Creating lists: 100% [===================================================================================================================================================]D 0h00m05s

******************************************************************************

******************** INSTANTIATION of titulaciones FAMILY ********************

******************************************************************************

These lists have been created and aliases are ok :
  list1, list2

This is the sympa.log for this execution:

Jan 20 09:52:47 hostname sympa[865836]: info main::_load() Configuration file read, default log level 0
Jan 20 09:52:47 hostname sympa[865836]: notice main:: Sympa 6.2.60 Started
Jan 20 09:52:47 hostname sympa[865836]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=create_automatic_list;context=titulaciones@example.domain.com;listname=list1>
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Expecting a single line for "profile" parameter in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Bad entry "profile normal" in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Expecting a single line for "profile" parameter in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Bad entry "profile normal" in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Expecting a single line for "reception" parameter in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Bad entry "reception mail" in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: notice Sympa::Request::Handler::create_automatic_list::_twist() Synchronizing list members...
Jan 20 09:52:48 hostname sympa[865836]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=list1@example.domain.com;role=member>
Jan 20 09:52:49 hostname sympa[865836]: info Sympa::Request::Handler::include::_twist() Sympa::DataSource::SQL <context=list1@example.domain.com;id=f83f0a74;role=member;name=Enderezo_Institucional>: 2 included, 0 deleted, 0 updated, 0 kept
Jan 20 09:52:49 hostname sympa[865836]: info Sympa::Request::Handler::include::_twist() Sympa::Request <action=include;context=list1@example.domain.com;role=member>: Success, 2 added, 0 deleted, 0 updated
Jan 20 09:52:49 hostname sympa[865836]: notice Sympa::Request::Handler::create_automatic_list::_twist() ...done
Jan 20 09:52:50 hostname sympa[865836]: notice Sympa::Request::Handler::create_automatic_list::_twist() Synchronizing list members...
Jan 20 09:52:50 hostname sympa[865836]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=list1@example.domain.com;role=member>
Jan 20 09:52:50 hostname sympa[865836]: info Sympa::Request::Handler::include::_twist() Sympa::DataSource::SQL <context=list1@example.domain.com;id=f83f0a74;role=member;name=Enderezo_Institucional>: 0 included, 0 deleted, 2 updated, 0 kept
Jan 20 09:52:50 hostname sympa[865836]: info Sympa::Request::Handler::include::_twist() Sympa::Request <action=include;context=list1@example.domain.com;role=member>: Success, 0 added, 0 deleted, 2 updated
Jan 20 09:52:50 hostname sympa[865836]: notice Sympa::Request::Handler::create_automatic_list::_twist() ...done
Jan 20 09:52:50 hostname sympa[865836]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=create_automatic_list;context=titulaciones@example.domain.com;listname=list2>
Jan 20 09:52:50 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Expecting a single line for "profile" parameter in /var/lib/sympa/list_data/example.domain.com/list2/config
Jan 20 09:52:50 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Bad entry "profile normal" in /var/lib/sympa/list_data/example.domain.com/list2/config
Jan 20 09:52:50 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Expecting a single line for "reception" parameter in /var/lib/sympa/list_data/example.domain.com/list2/config
Jan 20 09:52:50 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Bad entry "reception mail" in /var/lib/sympa/list_data/example.domain.com/list2/config
Jan 20 09:52:51 hostname sympa[865836]: notice Sympa::Request::Handler::create_automatic_list::_twist() Synchronizing list members...
Jan 20 09:52:51 hostname sympa[865836]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=list2@example.domain.com;role=member>
Jan 20 09:52:51 hostname sympa[865836]: info Sympa::Request::Handler::include::_twist() Sympa::DataSource::SQL <context=list2@example.domain.com;id=d5f1937d;role=member;name=Enderezo_Institucional>: 1 included, 0 deleted, 0 updated, 0 kept
Jan 20 09:52:51 hostname sympa[865836]: info Sympa::Request::Handler::include::_twist() Sympa::Request <action=include;context=list2@example.domain.com;role=member>: Success, 1 added, 0 deleted, 0 updated
Jan 20 09:52:51 hostname sympa[865836]: notice Sympa::Request::Handler::create_automatic_list::_twist() ...done
Jan 20 09:52:52 hostname sympa[865836]: notice Sympa::Request::Handler::create_automatic_list::_twist() Synchronizing list members...
Jan 20 09:52:52 hostname sympa[865836]: notice Sympa::Spindle::ProcessRequest::_twist() Processing Sympa::Request <action=include;context=list2@example.domain.com;role=member>
Jan 20 09:52:52 hostname sympa[865836]: info Sympa::Request::Handler::include::_twist() Sympa::DataSource::SQL <context=list2@example.domain.com;id=d5f1937d;role=member;name=Enderezo_Institucional>: 0 included, 0 deleted, 1 updated, 0 kept
Jan 20 09:52:52 hostname sympa[865836]: info Sympa::Request::Handler::include::_twist() Sympa::Request <action=include;context=list2@example.domain.com;role=member>: Success, 0 added, 0 deleted, 1 updated
Jan 20 09:52:52 hostname sympa[865836]: notice Sympa::Request::Handler::create_automatic_list::_twist() ...done

These are the files used for the configuration of the list family.

config.tt2

subject [% subject %]

visibility 4-secreta

[% FOREACH o = owner %]
owner
email [% o.email %]
[% IF o.gecos -%]
gecos [% o.gecos %]
[% END %]
profile normal
visibility conceal
reception mail
[% END %]

[% IF editor %]
[% FOREACH e = editor %]
editor
email [% e.email %]
[% IF e.gecos -%]
gecos [% e.gecos %]
[% END %]
reception mail
visibility conceal
[% END %]
[% END %]

topics difusion,titulacions

lang gl

send 4-dominiomoderada

digest 5 20:56

available_user_options
reception digest,digestplain,html,mail,nomail,not_me,notice,summary,txt,urlize

default_user_options
reception mail
visibility conceal

reply_to_header
value other_email
other_email non-respostar@domain.com
apply forced

anonymous_sender non-respostar@domain.com

custom_header X-INSTITUTION-Lista: [% listname FILTER upper %]

custom_header X-INSTITUTION-Mailer: SYMPA

custom_subject [% listname %]

reject_mail_from_automates_feature on

rfc2369_header_fields archive,help,owner,post,subscribe,unsubscribe

info 2-privada

subscribe 5-pechada

add 2-pechada

unsubscribe 3-pechada

del 2-pechada

invite 4-pechada

remind 2-listmaster

review 1-owner

shared_doc
d_read 4-restinxido
d_edit 4-restinxido

process_archive on

archive
web_access 3-restrinxido
mail_access 3-restrinxido
max_month 2

bouncers_level1
notification none
action none
rate 45

bouncers_level2
notification none
action none
rate 75

dkim_signature_apply_on

clean_delay_queuemod 3

pictures_feature off

status open

inclusion_notification_feature off

include_sql_query
name Enderezo_Institucional
db_type ODBC
db_name courses-dsn
db_user user_admin
db_passwd **************
sql_query SELECT mail FROM database WHERE course = '[% codigo_query_sql %]'
nosync_time_ranges 06:30-07:45

edit_list.conf

include_sql_query   privileged_owner,owner  read

default             listmaster              write
default             privileged_owner        read
default             owner                   read
default             editor                  hidden

info.tt2

Distribution list for [% subject %]

param_constraint.conf

visibility 4-secreta

topics difusion,titulacions

lang gl

send 4-dominiomoderada

available_user_options.reception digest,digestplain,html,mail,nomail,not_me,notice,summary,txt,urlize

default_user_options.reception mail
default_user_options.visibility conceal

reply_to_header.value other_email
reply_to_header.other_email non-respostar@domain.com
reply_to_header.apply forced

anonymous_sender non-respostar@domain.com

reject_mail_from_automates_feature on

rfc2369_header_fields archive,help,owner,post,subscribe,unsubscribe

info 2-privada

subscribe 5-pechada

add 2-pechada

unsubscribe 3-pechada

del 2-pechada

invite 4-pechada

remind 2-listmaster

review 1-owner

shared_doc.d_read 4-restinxido
shared_doc.d_edit 4-restinxido

process_archive on

archive.web_access 3-restrinxido
archive.mail_access 3-restrinxido
archive.max_month 2

bouncers_level1.notification none
bouncers_level1.action none
bouncers_level1.rate 45

bouncers_level2.notification none
bouncers_level2.action none
bouncers_level2.rate 75

clean_delay_queuemod 3

pictures_feature off

status open

inclusion_notification_feature off

titulaciones.xml

<?xml version="1.0" ?>
<family>
  <list>
    <listname>list1</listname>
    <subject>Medicine Studies</subject>
    <owner multiple="1">
      <email>luis@domain.com</email>
      <gecos>Luis</gecos>
    </owner>
    <owner multiple="1">
      <email>maria@domain.com</email>
      <gecos>Maria</gecos>
    </owner>
    <editor multiple="2">
      <email>xiana@domain.com</email>
      <gecos>Xiana</gecos>
    </editor>
    <codigo_query_sql>list1</codigo_query_sql>
  </list>
  <list>
    <listname>list2</listname>
    <subject>Pharmacy Studies</subject>
    <owner multiple="1">
      <email>xiana@domain.com</email>
      <gecos>Xiana</gecos>
    </owner>
    <editor multiple="1">
      <email>luis@domain.com</email>
      <gecos>Luis</gecos>
    </editor>
    <codigo_query_sql>list2</codigo_query_sql>
  </list>
</family>
qosobrin commented 1 year ago

I know that messages Use of uninitialized value in regexp compilation at /usr/share/sympa/lib/Sympa/List.pm line 5194, <GEN2> line 103. are related to the sympa.log entries:

Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Expecting a single line for "profile" parameter in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Bad entry "profile normal" in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Expecting a single line for "reception" parameter in /var/lib/sympa/list_data/example.domain.com/list1/config
Jan 20 09:52:48 hostname sympa[865836]: info Sympa::List::_load_list_config_file() Bad entry "reception mail" in /var/lib/sympa/list_data/example.domain.com/list1/config

and are related with the presence of these two lines profile normal and reception mail inthe config.tt2 file. As far as I know the syntax of these lines is ok and their presence should not generate any error or warning. It seems like the problem with them is that they are not expected there (I'm guessing here, I have no proof for that).

But, aside form that warning, there are two more that I do not have any clue where they are coming from: Use of uninitialized value in subroutine entry at /usr/lib/sympa/bin/sympa.pl line 1746. and Use of uninitialized value in join or string at /usr/share/sympa/lib/Sympa/List.pm line 4858.

qosobrin commented 1 year ago

I have come up with a solution to the problem of message Use of uninitialized value in regexp compilation at /usr/share/sympa/lib/Sympa/List.pm line 5194, <GEN2> line 103. It seems related not with the presence but with the position of some configuration elements in the config.tt2 file. For exemple, changing this paragraph:

[% FOREACH o = owner %]
owner
email [% o.email %]
[% IF o.gecos -%]
gecos [% o.gecos %]
[% END %]
profile normal
visibility conceal
reception mail
[% END %]

to 

[% FOREACH o = owner %]
owner
profile normal
visibility conceal
reception mail
email [% o.email %]
[% IF o.gecos -%]
gecos [% o.gecos %]
[% END %]
[% END %]

results in a instantiation process without this message. I feel like there seems to be some kind of problem in the sympa.pl file that expects them in a given position. The same happens to the editor paragraph, that had to be changed to:

[% IF editor %]
[% FOREACH e = editor %]
editor
reception mail
visibility conceal
email [% e.email %]
[% IF e.gecos -%]
gecos [% e.gecos %]
[% END %]
[% END %]
[% END %]

in order for it to work properly. I understand that this requirement to set some elements in a given position in the config.tt2 file is a bug that should be corrected. If it is not a bug and it is there by design, then it should be documented somewhere.

ikedas commented 1 year ago

The template

[% FOREACH o = owner %]
owner
email [% o.email %]
[% IF o.gecos -%]
gecos [% o.gecos %]
[% END %]
profile normal
visibility conceal
reception mail
[% END %]

generates


owner
email luis@domain.com
gecos Luis

profile normal
visibility conceal
reception mail

...

That is, the newlines before [% and after %] aren't removed.

Instead, you'd be better to do:

[% FOREACH o = owner -%]
owner
email [% o.email %]
[% IF o.gecos -%]
gecos [% o.gecos %]
[% END -%]
profile normal
visibility conceal
reception mail
[% END -%]

The newlines before [%- and after -%] are removed.

qosobrin commented 1 year ago

Thank you very much, @ikedas, for your corrections to my tt2 file; they did the trick for the warnings related to line 5194 in file /usr/share/sympa/lib/Sympa/List.pm

Unfortunatelly there are still two other warnings that show up during family instantiation: Use of uninitialized value in subroutine entry at /usr/lib/sympa/bin/sympa.pl line 1746. and Use of uninitialized value in join or string at /usr/share/sympa/lib/Sympa/List.pm line 4858. Any idea where they can come from?

Best regards.

ikedas commented 1 year ago

Hi @qosobrin , Did you correct template for editor paragraph?

qosobrin commented 1 year ago

Hi @qosobrin , Did you correct template for editor paragraph?

Sure, I corrected both owner and editor paragraphs. I tried the new config.tt2 file by removing the previous family and reinstantiating a new one and warnings related to line 5194 in file /usr/share/sympa/lib/Sympa/List.pm all disappeared but the two other that I mentioned in my previous entry are still there.

Thank you very much. Best regards.

ikedas commented 1 year ago

What are the contents of /usr/lib/sympa/bin/sympa.pl line 1746 and /usr/share/sympa/lib/Sympa/List.pm line 4858 ?

qosobrin commented 1 year ago

/usr/lib/sympa/bin/sympa.pl

1715         ## listname
1716         my @children = $list_elt->getChildrenByTagName('listname');
1717
1718         if ($#children < 0) {
1719             $log->syslog(
1720                 'err',
1721                 '"listname" element is required in "list" element, line: %s',
1722                 $list_elt->line_number()
1723             );
1724             return undef;
1725         }
1726         if ($#children > 0) {
1727             my @error;
1728             foreach my $i (@children) {
1729                 push(@error, $i->line_number());
1730             }
1731             $log->syslog(
1732                 'err',
1733                 'Only one "listname" element is allowed for "list" element, lines: %s',
1734                 join(", ", @error)
1735             );
1736             return undef;
1737         }
1738         my $listname_elt = shift @children;
1739         my $listname     = $listname_elt->textContent();
1740         $listname =~ s/^\s*//;
1741         $listname =~ s/\s*$//;
1742         $listname = lc $listname;
1743         my $filename = $listname . ".xml";
1744
1745         ## creating list XML document
1746         my $list_doc =
1747             XML::LibXML::Document->createDocument($doc->version(),
1748             $doc->encoding());
1749         $list_doc->setDocumentElement($list_elt);
1750
1751         ## creating the list xml file
1752         unless ($list_doc->toFile("$family->{'dir'}/$filename", 0)) {
1753             $log->syslog(
1754                 'err',
1755                 'Cannot create list file %s',
1756                 $family->{'dir'} . '/' . $filename,
1757                 $list_elt->line_number()
1758             );
1759             return undef;
1760         }
1761
1762         push @list_to_generate, $listname;
1763     }
1764     return [@list_to_generate];
1765 }
1766
1767 __END__

/usr/share/sympa/lib/Sympa/List.pm

4820     } elsif (ref($pinfo->{$key}{'file_format'}) eq 'HASH') {
4821         $fd->print(sprintf "%s\n", $key);
4822         foreach my $k (keys %{$p}) {
4823
4824             if (defined($pinfo->{$key}{'file_format'}{$k}{'scenario'})) {
4825                 ## Skip if empty value
4826                 next
4827                     unless defined $p->{$k}{'name'}
4828                     and $p->{$k}{'name'} =~ /\S/;
4829
4830                 $fd->print(sprintf "%s %s\n", $k, $p->{$k}{'name'});
4831
4832             } elsif (($pinfo->{$key}{'file_format'}{$k}{'occurrence'} =~ /n$/)
4833                 && $pinfo->{$key}{'file_format'}{$k}{'split_char'}) {
4834                 next unless $p->{$k} and @{$p->{$k}};
4835
4836                 $fd->print(
4837                     sprintf "%s %s\n",
4838                     $k,
4839                     join(
4840                         $pinfo->{$key}{'file_format'}{$k}{'split_char'},
4841                         @{$p->{$k}}
4842                     )
4843                 );
4844             } else {
4845                 ## Skip if empty value
4846                 next unless defined $p->{$k} and $p->{$k} =~ /\S/;
4847
4848                 $fd->print(sprintf "%s %s\n", $k, $p->{$k});
4849             }
4850         }
4851         $fd->print("\n");
4852
4853     } else {
4854         if (($pinfo->{$key}{'occurrence'} =~ /n$/)
4855             && $pinfo->{$key}{'split_char'}) {
4856             ### " avant de debugger do_edit_list qui crée des nouvelles
4857             ### entrées vides
4858             my $string = join($pinfo->{$key}{'split_char'}, @{$p});
4859             $string =~ s/\,\s*$//;
4860
4861             $fd->print(sprintf "%s %s\n\n", $key, $string);
4862         } elsif ($key eq 'digest') {
4863             my $value = sprintf '%s %d:%d', join(',', @{$p->{'days'}}),
4864                 $p->{'hour'}, $p->{'minute'};
4865             $fd->print(sprintf "%s %s\n\n", $key, $value);
4866         } else {
4867             $fd->print(sprintf "%s %s\n\n", $key, $p);
4868         }
4869     }
4870
4871     return 1;
4872 }

I added the line numbers for your convenience and some more lines for context. Should you need me to post the whole files, please, let me know.

Thank you very much. Best regards.

ikedas commented 1 year ago

These codes have been almost entirely rewritten in recent versions. If you have upgraded to a more recent version of Sympa and still get the same warnings, please submit a new bug report.

qosobrin commented 1 year ago

No, I am running the Debian packaged Sympa, so I will have to wait until a new Debian comes out. If there is not solution available then you can close the issue.

Thank you very much for your prompt answers and all your help.

Best regards.

ikedas commented 1 year ago

With Debian package, recent versions are available. https://packages.debian.org/en/bookworm/sympa

racke commented 1 year ago

Also in bullseye-backports.

qosobrin commented 1 year ago

Thank to both of you, @ikedas and @racke This is a production server with 175+ lists and some other software running on it that needs time to be moved to a new version.

Best regards.