ERROR: Response of login call to /login_sid.lua in initSID was not xml-formatted.

[ArnaudFeld]

Used the actual Version from Github & get the following Error

ERROR: Response of login call to /login_sid.lua in initSID was not xml-formatted.

FritzBox 7390 FritzOS: 06.30

With the same Script and a 7490 with 06.51 it works :/

[rbu]

I have the same issue with a 7490 on 06.51. A shortened log below:

~/carddav2fb $ php carddav2fb.php 
carddav2fb.php - CardDAV to FRITZ!Box conversion tool
Copyright (c) 2012-2015 Karl Glatz, Martin Rost, Jens Maus

Retrieving VCards from all CardDAV server(s):
 https://example.com/addressbook.vcf
  123 VCards retrieved
PHP Notice:  Undefined index: group_vip in /path/to/carddav2fb/carddav2fb.php on line 217
PHP Notice:  Undefined index: group_vip in /path/to/carddav2fb/carddav2fb.php on line 217
[repeating]
...
Done.
Converting VCards to FritzBox XML format:
...
 VCard: A�bcd, Vorname
  WARNING: no phone entry found. VCard will be ignored.
 VCard: G�bdla, Vorname
  Added phone: +491234567 (mobile)
  Added email: webmaster@gmail.com (home)
  Added photo: Gaebdla_Vorname.jpg
...
Done.
Upload data to FRITZ!Box @ 192.168.1.1
...
 Uploading Phonebook XML
  ERROR: Response of login call to /login_sid.lua in initSID was not xml-formatted.
Done.

Note that there's a mixture of issues. Formatting error seems to be #12? There's also an indexerror that may not be critical. Versions:

PHP 5.4.16 (cli) (built: Jun 23 2015 21:17:27) 
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies

[jens-maus]

As for the first warning message (Undefined index: group_vip) please show your config file. My guess is that you have simply commented out the group_vip setting will it should stay in there.

The second issue (ERROR: Response of login call to /login_sid.lua) might be related to some login problems. So please also check your config file for the fritzbox_user and fritzbox_pw setting and make sure you have configured the Fritz!Box user accordingly in your fritzbox. So please see the following section in the documentation as how to correctly setup your Fritz!Box users:

https://github.com/jens-maus/carddav2fb#configuration

[ArnaudFeld]

For the second Issue: i checked it multiple Times.

The User has all the rights which are needed & the Fritzbox is configured like you have written. Is there a way to debug?

[ArnaudFeld]

hi @jens-maus, i try now the new Version (the new merged Version). Have the Same Problem :(

Upload data to FRITZ!Box @ fb.mydomain.de Uploading Phonebook XML ERROR: Response of login call to /login_sid.lua in initSID was not xml-formatted. Done.

[ArnaudFeld]

i try something (with my local FritzBox, not the remote one) When i try the "IP-Adresse" it fails. When i try "fritz.box" it works.

I hope this helps you and @holzhannes

[ArnaudFeld]

is there a way how i can debug why it isn't working @jens-maus @holzhannes when i try it remote? i'm not the only one with this problem

[rbu]

Thank you for the help. The VIP setting indeed was missing from my configuration (which I based on the readme, instead of the example). I expected the tool to survive removing unwanted options.

Regarding the login error (this bug): I've tried all combinations of host names/ip addresses. When setting the $config['fritzbox_ip'] to an IP address, it fails with:

Uploading Phonebook XML
  ERROR: Response of login call to /login_sid.lua in initSID was not xml-formatted.

The same happens when I set it to the hostname of the box (cleverhostname). When using the generic host name (fritz.box), it fails with:

Uploading Phonebook XML
  ERROR: ERROR: Login failed with an unknown response.

Note that the FTP upload before succeeds (I even see the image files in the NAS browser), so the login information should be correct. For what it's worth, the fritzbox is IPv6 enabled and resolves its name locally to an IPv6 as well:

$ host fritz.box
fritz.box has address 10.0.0.1
fritz.box has IPv6 address fd00::abc:abc:abc:1
fritz.box has IPv6 address 2001:abc::1

[ArnaudFeld]

i think it is a problem of the FRITZ!Box-API-PHP

[ArnaudFeld]

The Problem is not solved :(

@cmaoling @jens-maus @rbu

[cmaoling]

@arnaudFeld can you provide the log of the merged version. Does the DNS lookup for th3 given IP work at all?

[jens-maus]

@ArnaudFeld yes, a detailed log would be great. In addition, please provide your config file so that we can see how you have configured carddav2fb. It might be that some special setting causes this.

[ArnaudFeld]

Where i found the log? Or did you mean the screen-output in my terminal-window?

@jens-maus @cmaoling

[cmaoling]

The screen-output would be great. Especially after thr FTP upload is done. Beside thr actuallconfig might be helpful.

[ArnaudFeld]

Screen-Output:

Upload data to FRITZ!Box @ fb.MYDOMAIN.de INFO: Given value (fb.MYDOMAIN.de) is no valid IP address Uploading Phonebook XML to fb.MYDOMAIN.de ERROR: Response of login call to /login_sid.lua in initSID was not xml-formatted. Done.

Config.php: `<?php

// CONFIG

// DNS name of Fritz!Box or IP address $config['fritzbox_ip'] = 'fb.MYDOMAIN.de'; $config['fritzbox_ip_ftp'] = 'fb.MYDOMAIN.de';

// user name/password to access Fritz!Box $config['fritzbox_user'] = 'kontakte'; $config['fritzbox_pw'] = 'MYPASSWORD';

// number of the internal phone book and its name // 0 - main phone book // 1..n - additional phone books $config['phonebook_number'] = '1'; $config['phonebook_name'] = 'sync';

// optional: write output to file instead of sending it to the Fritz!Box //$config['output_file'] = 'phonebook.xml';

// optional: import only contacts of the given groups //$config['group_filter'] = array('Arzt','Familie','Freunde','Friseur','Geschäftlich','Hotline','Notruf','Restaurant','Shops');

// group name of 'important' callers $config['group_vip'] = 'VIP';

// base path of USB storage of Fritz!Box under which the path 'FRITZ\fonpix' could be found // '' -> use internal fritzbox storage $config['usb_disk'] = '';

// multiple carddav adressbooks could be specified and will be merged together.

// first $config['carddav'][0] = array( // URL of first CardDAV address book on cloud storage 'url' => 'https://MY-OWNCLOUD.com/remote.php/carddav/addressbooks/tht/kontakte', // user name/password for CardDAV access 'user' => 'tht', 'pw' => 'MYPASSWORD', // vcf extension 'extension' => '.vcf' );

// second //$config['carddav'][1] = array( // 'url' => 'https://raspserver/owncloud/remote.php/carddav/addressbooks/fritzbox/fb_contacts_second', // 'user' => 'oc_username', // 'pw' => 'oc_password', // 'extension' => '.vcf' //); `

[cmaoling]

Ok, so my workaround was focused on the case a IP address is given. As described: "When i try the "IP-Adresse" it fails. When i try "fritz.box" it works. " So the code now does a DNS lookup of the IP and uses the given hostname instead. This DOES NOT address the issue when using a non-IP address or registered domain like fb.MYDOMAIN.de.

@jens-maus: I think there are three cases: 1.) generic hostname (fritz.box) => worked prior 2.) IP address given => should be fixed w/ the code I provided 3.) registered domain given (like fb.MYDOMAIN.de or cleverhostname) => Fritzbox API response is not XML formatted. My usage of carddav2fb does has or need the third one. So I have no way to debug or think about a solution currently. Sorry, but this is out of my scope.

[ArnaudFeld]

Sorry my fault. When i opened the Issuse i try to do it via VPN to another Box. But it is the same Problem when i try it via DynDNS.

@cmaoling IP address given => do you mean the local IP e.g.: 10.10.10.1 ?

[cmaoling]

Not sure whether I can answer this in a generic way. I would assume you have a cloud-based owncloud and would like to update the phonebook @home. Correct? Were is carddav2fb running? Wouldn't you fritzbox be connected to a 192.x Or 172.x Local network, rather then a 10.x.? From all I know regarding VPN and IP spaces this really depends on your network topology and firewall setting. Incl. Whether your corresponding port of the Fritzbox is forwarded in the DMZ or not.... So give it a try, but for me it sounds like your Fritzbox declines to talk to carddav2fb because of missing port forward/firewall rules.

[cmaoling]

One additional thought. Have you tried to do a curl from the same server carddav2fb is running on, using the 10.10.10.1 IP adress? I used this to validate whether I have a basic issue. Or carddav2fb misbehaves. Might help to understand if your IP/Frirzbox reponse at all at all to HTTP requests rather than FTP once....

[ArnaudFeld]

my owncloud is running on my webspace. carddav2fb is running von a raspberryPi.

10.x is like 192.x or 172.x. (look rfc1918)

[ArnaudFeld]

The Problem is now only that when i try to go from carddav2fb (running on a raspberry pi) to the Fritzbox which i only can connect via dyndns. I hope it is now much clear whats my problem.

Which port forward/firewall rules i should make?

[ArnaudFeld]

When i try a curl with https://fb.MYDOMAIN.de (the dyndns-adress of the FritzBox) then i get this answer:

"<!DOCTYPE html>

<body> <p>Ihr Browser unterstützt keine XHTML-Frames.</p> <p>Sie können die <a href="/logincheck.lua">FRITZ!Box Benutzeroberfläche</a> aber trotzdem ohne Einschränkung nutzen.</p> </body> "

[jens-maus]

Does this problem still exit?

[mokkin]

The problem still exists. I tried today with a 7390 and 6.51:

Upload data to FRITZ!Box @ 192.168.2.1
PHP Warning:  ftp_mkdir(): /FRITZ/fonpix: File exists. in /var/www/carddav2fb/carddav2fb.php on line 693
 FTP-Upload 'xxxxxxxxxxxxxx.jpg'...PHP Warning:  in_array() expects parameter 2 to be array, boolean given in /var/www/carddav2fb/carddav2fb.php on line 706
 ok.
PHP Warning:  Invalid argument supplied for foreach() in /var/www/carddav2fb/carddav2fb.php on line 724
 FTP-Upload yyyyyyyyyyy.jpg'...PHP Warning:  in_array() expects parameter 2 to be array, boolean given in /var/www/carddav2fb/carddav2fb.php on line 706
 ok.
PHP Warning:  Invalid argument supplied for foreach() in /var/www/carddav2fb/carddav2fb.php on line 724
 WARNING: Unable to get hostname for IP address (192.168.2.1) <192.168.2.1<
 Uploading Phonebook XML to 192.168.2.1
  ERROR: Response of initialization call /login_sid.lua in initSID was not xml-formatted.
Done.

The box is connect with my local network via Box-to-Box-VPN.

If I specify the remote hoste name of the box as 'example.com' it doesn't work too.

[cmaoling]

It would help me to better understand your setup, if you please share the output of the following two commands: nslookup 192.168.2.1 and traceroute 192.168.2.1

Thanks

[mokkin]

nslookup 192.168.2.1
Server:     192.168.10.1
Address:    192.168.10.1#53

** server can't find 1.2.168.192.in-addr.arpa: NXDOMAIN

(both boxes are reachable via dyndns

traceroute 192.168.2.1
traceroute to 192.168.2.1 (192.168.2.1), 30 hops max, 60 byte packets
 1  fritz.box (192.168.10.1)  0.593 ms  0.702 ms  1.321 ms
 2  192.168.2.1 (192.168.2.1)  51.978 ms  53.345 ms  54.390 ms

[cmaoling]

Thanks. Did you try to enter the hostname on Dyndns into your config rather than the IP? Can you also try a traceroute from a host in the 192.168.2.x network toward 192.168.10.1?

@jens-maus for me this remains to be caused by the fritzbox handling on incoming requests and the gateway setups in these VPN like setup. It still bothers my technical understanding that it makes a difference whether you connect via hostname or IP. This is odd. In any case I would suspect some kind of backtrace being done by the fritzbox. Which fails in all cases where VPNs are used and the gateway are not set up in a way that the remote IPs are transperantly forwarded. So in this case the 192.168.10.x network and 192.168.2.x network reach each other (2nd command) but are seperated in the namespace.

[mokkin]

traceroute 192.168.10.1
traceroute to 192.168.10.1 (192.168.10.1), 30 hops max, 60 byte packets
 1  fritz.box (192.168.2.1)  1.016 ms  1.006 ms  1.285 ms
 2  192.168.10.1 (192.168.10.1)  52.251 ms  53.725 ms  54.162 ms

This is what happens if I define the remote dyndns hostname:

$config['fritzbox_ip'] = 'example.com';
$config['fritzbox_ip_ftp'] = 'example.com';

php carddav2fb.php home.php
.
.
.
Upload data to FRITZ!Box @ example.com
PHP Warning:  ftp_set_option() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 685
PHP Warning:  ftp_login() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 686
PHP Warning:  ftp_pasv() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 687
PHP Warning:  ftp_nlist() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 691
PHP Warning:  ftp_mkdir() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 693
 FTP-Upload 'xxxxxxxxxxxxxx.jpg'...PHP Warning:  in_array() expects parameter 2 to be array, null given in /var/www/carddav2fb/carddav2fb.php on line 706
PHP Warning:  ftp_put() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 708
 retrying... PHP Warning:  ftp_login() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 713
PHP Warning:  ftp_pasv() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 714
PHP Warning:  ftp_put() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 715
 ERROR: while uploading file xxxxxxxxxxxxxx.jpg
PHP Warning:  Invalid argument supplied for foreach() in /var/www/carddav2fb/carddav2fb.php on line 724
 FTP-Upload 'yyyyyyyyyyyyyyyy.jpg'...PHP Warning:  in_array() expects parameter 2 to be array, null given in /var/www/carddav2fb/carddav2fb.php on line 706
PHP Warning:  ftp_put() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 708
 retrying... PHP Warning:  ftp_login() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 713
PHP Warning:  ftp_pasv() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 714
PHP Warning:  ftp_put() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 715
 ERROR: while uploading file yyyyyyyyyyyyy.jpg
PHP Warning:  Invalid argument supplied for foreach() in /var/www/carddav2fb/carddav2fb.php on line 724
PHP Warning:  ftp_close() expects parameter 1 to be resource, boolean given in /var/www/carddav2fb/carddav2fb.php on line 740
 Uploading Phonebook XML to example.com
  ERROR: Response of initialization call /login_sid.lua in initSID was not xml-formatted.
Done.

[jens-maus]

Please update to the latest version available in git and retry. Then show the output here again.

[mokkin]

The picture upload has disappeared now but login error persists:

Upload data to FRITZ!Box @ example.com
 WARNING: Secure connection to FTP-server 'example.com' failed, retrying without SSL.
 ERROR: couldn't connect to FTP server 'example.com'.
 Uploading Phonebook XML to example.com
  ERROR: Response of initialization call /login_sid.lua in initSID was not xml-formatted.
Done.

[jens-maus]

Well this output shows that your fritzbox doesn't support FTP transfers. However, I still don't understand what is causing the login problem during phonebook upload.

[mokkin]

I can't confirm this. I tried to login via FTP ohne both fritzboxes (192.168.10.1 and 192.168.2.1) with my laptop and both are working. The first error

 ERROR: couldn't connect to FTP server 'example.com'.

I could resolve. It happened because internet acces via ftp wasn't activated but the login_sid.lua error persists.

Done.
Upload data to FRITZ!Box @ example.com
 FTP-Upload 'xxxxxxxxxxxxx.jpg'... already exists.
 FTP-Upload 'yyyyyyyyyyyyyy.jpg'... already exists.
 Uploading Phonebook XML to example.com
  ERROR: Response of initialization call /login_sid.lua in initSID was not xml-formatted.
Done.

[jens-maus]

Ok, great that you solved your FTP problems. However, I still don't know why you are receiving the "initSID" problem. I really tried hard here to reproduce the problem but I am not able to get the same error here so I can't really debug the issue and would need someone else to debug this situation, I am afraid.

[std68]

It seems to depend on the used hostname. I had the same problem uploading the address book when I tried to use my internal hostname "router.mydomain.de". This name is is resolved by an internal nameserver. When i tried "fritz.box" everything works as expected. So maybe there is a clue in dealing with internal/external authentication?

By the way: Thank you for that nice piece of work!

[jplitza]

Let me guess: You are all using HTTPS, right?

I pinpointed this error to the setting of the certificate paths:

* Hostname was found in DNS cache
*   Trying 192.168.178.1...
* Connected to 192.168.178.1 (192.168.178.1) port 443 (#0)
* error setting certificate verify locations:
  CAfile: /etc/ssl/certs/192.168.178.1.pem
  CApath: /etc/ssl/certs
* Closing connection 0

Of course, /etc/ssl/certs/192.168.178.1.pem doesn't exist, and that's what annoyed curl. If you are using a self-signed certificate, it makes sense you have to put it somewhere so the script can verify it (although I consider that a strange place, especially if you are running the script as a normal user), but if you have a "real" certificate signed by some trusted CA, you can simply patch away that additional path:

diff --git a/lib/fritzbox_api_php/fritzbox_api.class.php b/lib/fritzbox_api_php/fritzbox_api.class.php
index 0e2720e..bf4296e 100644
--- a/lib/fritzbox_api_php/fritzbox_api.class.php
+++ b/lib/fritzbox_api_php/fritzbox_api.class.php
@@ -158,7 +158,6 @@ class fritzbox_api {
                // set SSL-options and path to certificate
                curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 2);
                curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
-               curl_setopt($ch, CURLOPT_CAINFO, '/etc/ssl/certs/'.$ssl_cert_fritzbox.'.pem');
                curl_setopt($ch, CURLOPT_CAPATH, '/etc/ssl/certs');

                // support for pre FRITZ!OS 5.50 remote config

Anyway, even if this isn't your problem, try applying this patch to enable verbose output from curl:

diff --git a/lib/fritzbox_api_php/fritzbox_api.class.php b/lib/fritzbox_api_php/fritzbox_api.class.php
index 0e2720e..00c3d03 100644
--- a/lib/fritzbox_api_php/fritzbox_api.class.php
+++ b/lib/fritzbox_api_php/fritzbox_api.class.php
@@ -143,6 +143,7 @@ class fritzbox_api {
       }   
       curl_setopt($ch, CURLOPT_URL, $this->config->getItem('fritzbox_url') . '/cgi-bin/webcm');
     }
+    curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
     curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
     curl_setopt($ch, CURLOPT_POST, 1);
     if ( $this->config->getItem('enable_remote_config') == true )

[HendrikF]

For me the solution was to force login via plain http:

$config['fritzbox_force_local_login'] = true;

The problem is, that my fb does not listen on 443, so PHPs curl always returned false, which is of course not XML-formatted.

See fritzbox_api.class.php#L84

[holzhannes]

Problem

@jplitza already mentioned that it is might be a problem with TLS because the connection is closed because it is untrusted. I had the same issue ERROR: Response of initialization call /login_sid.lua in initSID was not xml-formatted. on uploading the phonebook to a remote FRITZ!Box. In my config I did set the $config['fritzbox_ip'] = 'XYZ.myfritz.net:45370';. I added curl_setopt($ch, CURLOPT_VERBOSE, TRUE); for debuging within in the API like mentioned by @jplitza and the result was that the connection was closed because it is not trusted.

* SSL certificate problem: self signed certificate
* Closing connection 0
  ERROR: Response of login call to /login_sid.lua in initSID was not xml-formatted.

Solution 1

You need to add the Certificate of your FRITZ!Box to the trusted Certs of the machine the carddav2fb is running on.

how to?

1. Download the Cert from your FRITZ!Box (Internet > Freigaben > Tab: FRITZ!Box-Dienste > Zertifikat herunterladen)
2. Install apt-get install ca-certificates on the machine carddav2fb is running on as root.
3. Copy the downloaded cert to /usr/share/ca-certificates as well as root.
4. Run dpkg-reconfigure ca-certificates as root
5. At the config window select the ask option, scroll to your certificate (at the end), mark it for inclusion and select ok. Afterwards the certs will be reconfigured. Be patience it will need some time.

how to is passed on

Solution 2

You could also get a valid Let’s Encrypt certificate on your FRITZ!Box by following this instruction and autoupdate it with this script.