search

sympa-community / sympa

Sympa, Mailing List Management Software
https://www.sympa.community/sympa
GNU General Public License v2.0
244 stars 97 forks source link

Incorrect character encoding when set custom attribute with special characters using the SOAP API #1407

Closed Bakaras closed 2 years ago

Bakaras commented 2 years ago

Version

6.2.60

[Note by admin: Patch by PR #1402 has to be applied]

Installation method

deb

Expected behavior

Custom attributes are set with correct characters.

Actual behavior

Custom attributes are set with incorrect characters. Here with german umlauts: first name Jürgen.

Steps to reproduce

  1. Set a custom attribute: 
    # sympa_soap_client.pl --trusted_application=trustedapp --trusted_application_password=tapasswd
--soap_url=https://listen.dom.ain/sympasoap --service=setCustom --proxy_vars=USER_EMAIL=juergen@example.abc
--service_parameters=testlist1,vorname,Jürgen
calling authenticateRemoteAppAndRun( trustedapp, tapasswd, USER_EMAIL=juergen@example.abc,setCustom,testlist1,vorname,Jürgen)
0
'1'
  2. Get subscriber details: 
    # sympa_soap_client.pl --trusted_application=trustedapp --trusted_application_password=tapasswd
--soap_url=https://listen.dom.ain/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=juergen@example.abc
--service_parameters=testlist1
calling authenticateRemoteAppAndRun( trustedapp, tapasswd, USER_EMAIL=juergen@example.abc,getDetails,testlist1)
0
_custom_
    0
        _key_
            'vorname'
        _value_
            'Jürgen'
    1
        _key_
            'anrede'
        _value_
            'Herr'
    2
        _key_
            'nachname'
        _value_
            'Schmidt'
_gecos_
    'Brunnenbau Münster'
_reception_
    'mail'
_subscribeDate_
    '1653904994'
_updateDate_
    '1653904994'

    On web interface: ca1

Additional information

# locale
LANG=de_DE.UTF-8
LANGUAGE=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=de_DE.UTF-8
Bakaras commented 2 years ago

Changing the SOAP.pm file fixes the error for me:

--- SOAP.pm.orig    2022-05-30 11:58:07.784000000 +0200
+++ SOAP.pm.ok  2022-05-21 22:08:44.008000000 +0200
@@ -1498,6 +1498,10 @@
             ->faultdetail('Use : <list> <key> <value> ');
     }
     %newcustom = %{$subscriber->{custom_attribute} // {}};
+        foreach my $customkey (keys %newcustom) {
+            my $customvalue = $newcustom{$customkey}{value};
+            $newcustom{$customkey}{value} = Encode::decode_utf8($customvalue);
+        }
     #if(! defined $list->{'admin'}{'custom_attribute'}{$key} ) {
     #  return SOAP::Data->name('result')->type('boolean')->value(0);
     #}
@@ -1507,7 +1511,7 @@
         # $newcustom{$key} = $list->{'admin'}{'custom_attribute'}{$key}
         #     if !defined $newcustom{$key}
         #         and defined $list->{'admin'}{'custom_attribute'};
-        $newcustom{$key}{value} = $value;
+        $newcustom{$key}{value} = Encode::decode_utf8($value);
     }
     die SOAP::Fault->faultcode('Server')
         ->faultstring('Unable to set user attributes')

Custom attribute are set with correct characters:

# sympa_soap_client.pl --trusted_application=trustedapp --trusted_application_password=tapasswd
--soap_url=https://listen.dom.ain/sympasoap --service=setCustom --proxy_vars=USER_EMAIL=juergen@example.abc
--service_parameters=testlist1,vorname,Jürgen
calling authenticateRemoteAppAndRun( trustedapp, tapasswd, USER_EMAIL=juergen@example.abc,setCustom,testlist1,vorname,Jürgen)
0
    '1'

getDetails: correct characters

# sympa_soap_client.pl --trusted_application=trustedapp --trusted_application_password=tapasswd
--soap_url=https://listen.dom.ain/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=juergen@example.abc
--service_parameters=testlist1
calling authenticateRemoteAppAndRun( trustedapp, tapasswd, USER_EMAIL=juergen@example.abc,getDetails,testlist1)
0
    _custom_
        0
            _key_
                'anrede'
            _value_
                'Herr'
        1
            _key_
                'vorname'
            _value_
                'Jürgen'
        2
            _key_
                'nachname'
            _value_
                'Schmidt'
    _gecos_
        'Brunnenbau Münster'
    _reception_
        'mail'
    _subscribeDate_
        '1653904994'
    _updateDate_
        '1653904994'

And correct characters on web interface: ca2

ikedas commented 2 years ago

Hi @Bakaras ,

Just for information, is this problem specific to sympa_soap_client.pl? Or does it also occur in real SOAP applications?

racke commented 2 years ago

SOAP::Lite's UTF8 support seems to be buggy: https://rt.cpan.org/Public/Bug/Display.html?id=86649

ikedas commented 2 years ago

@racke, I suspect that is a different issue. sympa_soap_client.pl looks not passing arguments to SOAP::Lite as utf8 string. I guess the real applications do so correctly.

racke commented 2 years ago

@ikedas, that is also possible.

ikedas commented 2 years ago

@Bakaras , additional question: Can you try with following patch (without your patch above)?

diff --git a/src/bin/sympa_soap_client.pl.in b/src/bin/sympa_soap_client.pl.in
index 8e97c4d..4a9134a 100644
--- a/src/bin/sympa_soap_client.pl.in
+++ b/src/bin/sympa_soap_client.pl.in
@@ -26,6 +26,7 @@
 use lib '--modulesdir--';
 use strict;
 use warnings;
+use Encode qw();
 use Getopt::Long;
 use HTTP::Cookies;
 #use SOAP::Lite +trace;
@@ -170,6 +171,10 @@ sub play_soap_as_trusted {
     printf
         "calling authenticateRemoteAppAndRun( $trusted_application, $trusted_application_password, $proxy_vars,$service,$p)\n";

+    foreach my $param (@parameters) {
+        $param = Encode::decode_utf8($param);
+    }
+
     my $reponse =
         $soap->authenticateRemoteAppAndRun($trusted_application,
         $trusted_application_password, $proxy_vars, $service, \@parameters);
@@ -264,6 +269,10 @@ sub play_soap {
         @parameters = split /,/, $service_parameters;
     }

+    foreach my $param (@parameters) {
+        $param = Encode::decode_utf8($param);
+    }
+
     if ($service eq 'lists') {
         printf "\n\nlists....\n";
         $reponse = $soap->lists();
Bakaras commented 2 years ago

Hi @Bakaras ,

Just for information, is this problem specific to sympa_soap_client.pl? Or does it also occur in real SOAP applications?

I have no real SOAP application. I use sympa_soap_client.pl in shell script to import subscribers. In addition: To make the call getDetails working i have changed sympa_soap_client.pl:

         if (ref($r->result) =~ /^ARRAY/) {
             #printf "R: $r->result\n";
             @ret = @{$r->result};
+        }elsif (ref( $r->result) =~ /^HASH/) {
+            @ret = ( $r->result );
         } elsif (ref $r->result) {
             print "Pb " . ($r->result) . "\n";
             return undef;

Without this change I have this output:

calling authenticateRemoteAppAndRun( trustedapp, tapasswd, USER_EMAIL=juergen@example.abc,getDetails,testlist1)
Pb HASH(0x55eff8ca7380)
Bakaras commented 2 years ago

@Bakaras , additional question: Can you try with following patch (without your patch above)?

diff --git a/src/bin/sympa_soap_client.pl.in b/src/bin/sympa_soap_client.pl.in
index 8e97c4d..4a9134a 100644
--- a/src/bin/sympa_soap_client.pl.in
+++ b/src/bin/sympa_soap_client.pl.in
@@ -26,6 +26,7 @@
 use lib '--modulesdir--';
 use strict;
 use warnings;
+use Encode qw();
 use Getopt::Long;
 use HTTP::Cookies;
 #use SOAP::Lite +trace;
@@ -170,6 +171,10 @@ sub play_soap_as_trusted {
     printf
         "calling authenticateRemoteAppAndRun( $trusted_application, $trusted_application_password, $proxy_vars,$service,$p)\n";

+    foreach my $param (@parameters) {
+        $param = Encode::decode_utf8($param);
+    }
+
     my $reponse =
         $soap->authenticateRemoteAppAndRun($trusted_application,
         $trusted_application_password, $proxy_vars, $service, \@parameters);
@@ -264,6 +269,10 @@ sub play_soap {
         @parameters = split /,/, $service_parameters;
     }

+    foreach my $param (@parameters) {
+        $param = Encode::decode_utf8($param);
+    }
+
     if ($service eq 'lists') {
         printf "\n\nlists....\n";
         $reponse = $soap->lists();

It did not help

racke commented 2 years ago

Hi @Bakaras , Just for information, is this problem specific to sympa_soap_client.pl? Or does it also occur in real SOAP applications?

I have no real SOAP application. I use sympa_soap_client.pl in shell script to import subscribers. In addition: To make the call getDetails working i have changed sympa_soap_client.pl:

That doesn't make sense to me. Why don't you use the Sympa CLI?

         if (ref($r->result) =~ /^ARRAY/) {
             #printf "R: $r->result\n";
             @ret = @{$r->result};
+        }elsif (ref( $r->result) =~ /^HASH/) {
+            @ret = ( $r->result );
         } elsif (ref $r->result) {
             print "Pb " . ($r->result) . "\n";
             return undef;

Without this change I have this output:

calling authenticateRemoteAppAndRun( trustedapp, tapasswd, USER_EMAIL=juergen@example.abc,getDetails,testlist1)
Pb HASH(0x55eff8ca7380)

#

Bakaras commented 2 years ago

I think with Sympa CLI custom attributes are not imported (?)

ikedas commented 2 years ago

@Bakaras , additional question: Can you try with following patch (without your patch above)?

diff --git a/src/bin/sympa_soap_client.pl.in b/src/bin/sympa_soap_client.pl.in
index 8e97c4d..4a9134a 100644
--- a/src/bin/sympa_soap_client.pl.in
+++ b/src/bin/sympa_soap_client.pl.in
@@ -26,6 +26,7 @@
 use lib '--modulesdir--';
 use strict;
 use warnings;
+use Encode qw();
 use Getopt::Long;
 use HTTP::Cookies;
 #use SOAP::Lite +trace;
@@ -170,6 +171,10 @@ sub play_soap_as_trusted {
     printf
         "calling authenticateRemoteAppAndRun( $trusted_application, $trusted_application_password, $proxy_vars,$service,$p)\n";

+    foreach my $param (@parameters) {
+        $param = Encode::decode_utf8($param);
+    }
+
     my $reponse =
         $soap->authenticateRemoteAppAndRun($trusted_application,
         $trusted_application_password, $proxy_vars, $service, \@parameters);
@@ -264,6 +269,10 @@ sub play_soap {
         @parameters = split /,/, $service_parameters;
     }

+    foreach my $param (@parameters) {
+        $param = Encode::decode_utf8($param);
+    }
+
     if ($service eq 'lists') {
         printf "\n\nlists....\n";
         $reponse = $soap->lists();

It did not help

Is the result exactly the same as before applying the patch? Could you please show the actual result?

Bakaras commented 2 years ago

Yes, the result is exactly the same.

Bakaras commented 2 years ago

In addition: If a custom attribute with Umlaut defined first, after that defined or modified others, the value becomes longer.

            _key_
                'vorname'
            _value_
                'Jürgen'
            _key_
                'vorname'
            _value_
                'Jürgen'
            _key_
                'vorname'
            _value_
                'JÃÂürgen'
            _key_
                'vorname'
            _value_
                'JÃÂÃÂÃÂürgen'
            _key_
                'vorname'
            _value_
                'JÃÂÃÂÃÂÃÂÃÂÃÂÃÂürgen'
racke commented 2 years ago

I think with Sympa CLI custom attributes are not imported (?)

Oops, yes that is quite possible :innocent:

ikedas commented 2 years ago

In addition: If a custom attribute with Umlaut defined first, after that defined or modified others, the value becomes longer.

Interesting. How to reproduce it? Can you show the steps?

BTW in your first example in above, umlaut is transformed to two characters but not three characters in your previous result. Is this difference caused by my patch? If it is, I might catch the tail of bug...

Bakaras commented 2 years ago 
# cat history-cmd.sh 
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=setCustom --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1,vorname,Jürgen
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=setCustom --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1,nachname,Schmidt
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=setCustom --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1,anrede,Herr
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=setCustom --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1,titel,Dr
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=setCustom --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1,titel,Dr
sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=info2@mc-master.de --service_parameters=testliste1

Output:

root@gkd-i222(sympa):~# sh history-cmd.sh 
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=info2@mc-master.de,setCustom,testliste1,vorname,Jürgen)
0
    '1'
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=info2@mc-master.de,getDetails,testliste1)
0
    _custom_
        0
            _key_
                'vorname'
            _value_
                'Jürgen'
        1
            _key_
                'anrede'
            _value_
                'Herr'
        2
            _key_
                'nachname'
            _value_
                'Schmidt'
        3
            _key_
                'titel'
            _value_
                'Dr'
    _gecos_
        'Brunnenbau Matejka'
    _reception_
        'mail'
    _subscribeDate_
        '1653923379'
    _updateDate_
        '1653923379'

calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=info2@mc-master.de,setCustom,testliste1,nachname,Schmidt)
0
    '1'
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=info2@mc-master.de,getDetails,testliste1)
0
    _custom_
        0
            _key_
                'vorname'
            _value_
                'Jürgen'
        1
            _key_
                'titel'
            _value_
                'Dr'
        2
            _key_
                'anrede'
            _value_
                'Herr'
        3
            _key_
                'nachname'
            _value_
                'Schmidt'
    _gecos_
        'Brunnenbau Matejka'
    _reception_
        'mail'
    _subscribeDate_
        '1653923379'
    _updateDate_
        '1653923379'
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no,
USER_EMAIL=info2@mc-master.de,setCustom,testliste1,anrede,Herr)
0
    '1'
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no,
USER_EMAIL=info2@mc-master.de,getDetails,testliste1)
0
    _custom_
        0
            _key_
                'vorname'
            _value_
                'JÃÂürgen'
        1
            _key_
                'nachname'
            _value_
                'Schmidt'
        2
            _key_
                'anrede'
            _value_
                'Herr'
        3
            _key_
                'titel'
            _value_
                'Dr'
    _gecos_
        'Brunnenbau Matejka'
    _reception_
        'mail'
    _subscribeDate_
        '1653923379'
    _updateDate_
        '1653923379'

calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no,
USER_EMAIL=info2@mc-master.de,setCustom,testliste1,titel,Dr)
0
    '1'
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no,
USER_EMAIL=info2@mc-master.de,getDetails,testliste1)
0
    _custom_
        0
            _key_
                'vorname'
            _value_
                'JÃÂÃÂÃÂürgen'
        1
            _key_
                'nachname'
            _value_
                'Schmidt'
        2
            _key_
                'anrede'
            _value_
                'Herr'
        3
            _key_
                'titel'
            _value_
                'Dr'
    _gecos_
        'Brunnenbau Matejka'
    _reception_
        'mail'
    _subscribeDate_
        '1653923379'
    _updateDate_
        '1653923379'

calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no,
USER_EMAIL=info2@mc-master.de,setCustom,testliste1,titel,Dr)
0
    '1'
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no,
USER_EMAIL=info2@mc-master.de,getDetails,testliste1)
0
    _custom_
        0
            _key_
                'titel'
            _value_
                'Dr'
        1
            _key_
                'anrede'
            _value_
                'Herr'
        2
            _key_
                'nachname'
            _value_
                'Schmidt'
        3
            _key_
                'vorname'
            _value_
                'JÃÂÃÂÃÂÃÂÃÂÃÂÃÂürgen'
    _gecos_
        'Brunnenbau Matejka'
    _reception_
        'mail'
    _subscribeDate_
        '1653923379'
    _updateDate_
        '1653923379'

Sorry, have now eaten and drank some beer, left the real data as it is. Is not important now.

Bakaras commented 2 years ago

BTW in your first example in above, umlaut is transformed to two characters but not three characters in your previous result. Is this difference caused by my patch? If it is, I might catch the tail of bug...

It was probably the output after the second call. It was yesterday. Very far in the past.

ikedas commented 2 years ago

@Bakaras , sorry to interrupt your relaxation time:smile:. At the moment the fix you suggested looks promising, and I will prepare a PR based on it.

ikedas commented 2 years ago

@Bakaras , I found this workaround solves the problem.

diff --git a/src/lib/Sympa/WWW/SOAP.pm b/src/lib/Sympa/WWW/SOAP.pm
index ba4caf9..6fc986b 100644
--- a/src/lib/Sympa/WWW/SOAP.pm
+++ b/src/lib/Sympa/WWW/SOAP.pm
@@ -1501,6 +1501,11 @@ sub setCustom {
     #if(! defined $list->{'admin'}{'custom_attribute'}{$key} ) {
     #  return SOAP::Data->name('result')->type('boolean')->value(0);
     #}
+
+    $log->syslog('info', 'key=%s:utf8=%s; value=%s:utf8=%s',
+        $key, Encode::is_utf8($key), $value, Encode::is_utf8($value));
+    $key = Encode::encode_utf8($key) if Encode::is_utf8($key);
+
     if ($value eq '') {
         undef $newcustom{$key};
     } else {

The log line added by this is

info Sympa::WWW::SOAP::setCustom() key=vorname:utf8=1; value=Jürgen:utf8=

This behavior seems due to bug (or buggy behavior) of SOAP::Lite.

Bakaras commented 2 years ago

OK, with workaround, custom attribute "vorname" are set with correct characters.

# sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no
--soap_url=https://listen.gkd.re/sympasoap --service=setCustom
 --proxy_vars=USER_EMAIL=juergen@example.abc
 --service_parameters=testliste1,vorname,Jürgen
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=juergen@example.abc,setCustom,testliste1,vorname,Jürgen)
0
    '1'
# sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no
--soap_url=https://listen.gkd.re/sympasoap --service=getDetails
--proxy_vars=USER_EMAIL=juergen@example.abc --service_parameters=testliste1
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=juergen@example.abc,getDetails,testliste1)
0
    _custom_
        0
            _key_
                'vorname'
            _value_
                'Jürgen'
    _gecos_
        'Jürgen Schmidt'
    _reception_
        'mail'
    _subscribeDate_
        '1654087972'
    _updateDate_
        '1654087972'

But after setCustom "nachname" is "vorname" broken again:

# sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=setCustom --proxy_vars=USER_EMAIL=juergen@example.abc --service_parameters=testliste1,nachname,Schmidt
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=juergen@example.abc,setCustom,testliste1,nachname,Schmidt)
0
    '1'
# sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=juergen@example.abc --service_parameters=testliste1
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=juergen@example.abc,getDetails,testliste1)
0
    _custom_
        0
            _key_
                'vorname'
            _value_
                'Jürgen'
        1
            _key_
                'nachname'
            _value_
                'Schmidt'
    _gecos_
        'Jürgen Schmidt'
    _reception_
        'mail'
    _subscribeDate_
        '1654087972'
    _updateDate_
        '1654087972'
Bakaras commented 2 years ago

In addition: Gecos are set by setDetails with correct characters, and is not broken later.

sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no
--soap_url=https://listen.gkd.re/sympasoap --service=setDetails
--proxy_vars=USER_EMAIL=juergen@example.abc
--service_parameters=testliste1,Jürgen\ Schmidt
ikedas commented 2 years ago

@Bakaras , what is shown in the log?

Bakaras commented 2 years ago

@Bakaras , what is shown in the log?

Jun  1 14:42:44 i2 sympasoap[13033]: notice Sympa::WWW::SOAP::authenticateRemoteAppAndRun(soaplistengkdre, USER_EMAIL=juergen@domain.abc, setCustom, testliste1,vorname,Jürgen)
Jun  1 14:42:44 i2 sympasoap[13033]: info Sympa::WWW::SOAP::setCustom() key=vorname:utf8=1; value=Jürgen:utf8=
Jun  1 14:42:58 i2 sympasoap[13033]: notice Sympa::WWW::SOAP::authenticateRemoteAppAndRun(soaplistengkdre, USER_EMAIL=juergen@domain.abc, setCustom, testliste1,nachname,Schmidt)
Jun  1 14:42:58 i2 sympasoap[13033]: info Sympa::WWW::SOAP::setCustom() key=nachname:utf8=1; value=Schmidt:utf8=1
ikedas commented 2 years ago

Thanks @Bakaras !

info Sympa::WWW::SOAP::setCustom() key=vorname:utf8=1; value=Jürgen:utf8= info Sympa::WWW::SOAP::setCustom() key=nachname:utf8=1; value=Schmidt:utf8=1

It is probably a bug in SOAP::Lite (but probably different from the one @racke referred to).

What is the version of SOAP::Lite you use? I use 1.10.

Anyways workaround would be as below:

diff --git a/src/lib/Sympa/WWW/SOAP.pm b/src/lib/Sympa/WWW/SOAP.pm
index ba4caf9..6fc986b 100644
--- a/src/lib/Sympa/WWW/SOAP.pm
+++ b/src/lib/Sympa/WWW/SOAP.pm
@@ -1501,6 +1501,10 @@ sub setCustom {
     #if(! defined $list->{'admin'}{'custom_attribute'}{$key} ) {
     #  return SOAP::Data->name('result')->type('boolean')->value(0);
     #}
+
+    Encode::_utf8_off($key);
+    Encode::_utf8_off($value);
+
     if ($value eq '') {
         undef $newcustom{$key};
     } else {
Bakaras commented 2 years ago

Thanks @Bakaras !

info Sympa::WWW::SOAP::setCustom() key=vorname:utf8=1; value=Jürgen:utf8= info Sympa::WWW::SOAP::setCustom() key=nachname:utf8=1; value=Schmidt:utf8=1

It is probably a bug in SOAP::Lite (but probably different from the one @racke referred to).

What is the version of SOAP::Lite you use? I use 1.10.

# dpkg -l | grep -i soap
ii  libsoap-lite-perl                   1.27-1                             all          Perl implementation of a SOAP client and server
ikedas commented 2 years ago

@Bakaras , if you have no objection, I'll merge the PR above, and it will be included in the next release.

Bakaras commented 2 years ago

OK, Jürgen is happy.

sympa_soap_client.pl --trusted_application=soaplistengkdre --trusted_application_password=Uazee5no --soap_url=https://listen.gkd.re/sympasoap --service=getDetails --proxy_vars=USER_EMAIL=juergen@example.abc --service_parameters=testliste1
calling authenticateRemoteAppAndRun( soaplistengkdre, Uazee5no, USER_EMAIL=juergen@example.abc,getDetails,testliste1)
0
    _custom_
        0
            _key_
                'vorname'
            _value_
                'Jürgen'
        1
            _key_
                'nachname'
            _value_
                'Schmidt'
        2
            _key_
                'anrede'
            _value_
                'Herr'
    _gecos_
        'Jürgen Schmidt'
    _reception_
        'mail'
    _subscribeDate_
        '1654761809'
    _updateDate_
        '1654761809'

And what about sympa_soap_client.pl?

         if (ref($r->result) =~ /^ARRAY/) {
             #printf "R: $r->result\n";
             @ret = @{$r->result};
+        }elsif (ref( $r->result) =~ /^HASH/) {
+            @ret = ( $r->result );
         } elsif (ref $r->result) {
             print "Pb " . ($r->result) . "\n";
             return undef;
ikedas commented 2 years ago

OK, Jürgen is happy.

Thanks!

And what about sympa_soap_client.pl?

         if (ref($r->result) =~ /^ARRAY/) {
             #printf "R: $r->result\n";
             @ret = @{$r->result};
+        }elsif (ref( $r->result) =~ /^HASH/) {
+            @ret = ( $r->result );
         } elsif (ref $r->result) {
             print "Pb " . ($r->result) . "\n";
             return undef;

I'd overlooked it. Thanks for the reminder!