Aypac / Arduino-TR-064-SOAP-Library

Arduino library for the TR-064 protocol, most commonly used by the Fritz!Box router API
Other
63 stars 21 forks source link

HTTP<Error> #45

Closed JimKnopfIoT closed 2 years ago

JimKnopfIoT commented 3 years ago

Since i updated my Fritz!Box 7590 to 7.25 i have a problem with the tr064 stuff. I used the script published in c't and it was working well. In addition to the tr064 call, i use a telegram bot to message me if the doorbell rings. Anyway, since the mentioned update to 7.25 firmware, i noticed that it takes some time till i get the telegram msg after the doorbell rings. The tr064 calls a phone in the basement, so i didn't notice that it stops working because i'm not that often there. Today i startet an investigation, updated my Arduino libs and reflashed my Klingel.ino to the ESP 12E that i'm using. It seems, that something is wrong with http request from the Fritz!Box. Maybe AVM did some changes to not allow unsafe http requests? I tried to connect to port 49443 instead of 49000 but get the same error. I tried the script from roehrenkramladen.de but it creates the same error. Only the tr064 stuff stoped working. I receive the telegram notification with a delay because of the non working tr064 trials. I swapped the order in the ino file to first receive telegram notification before the internal phone call.

I do have a dedicated user/pwd and i can reach https://:49443/tr64desc.xml (http://:49000/tr64desc.xml is not working).

What changes are needed in the tr064.h and tr064.cpp (and of course in the ino file) to get the tr064 stuff working again? I tried to insert in tr064.cpp

WIFIClient client; just before the line HTTPClient http; and

http.begin(client, ip, _port, url); instead of http.begin(ip, _port, url);

But it didn't fix the problem.

This are my include files:

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <ESP8266WiFiMulti.h>
#include <ESP8266HTTPClient.h>
ESP8266WiFiMulti WiFiMulti;
#include <tr064.h>
#include <ESP8266TelegramBOT.h>
#include <PubSubClient.h>

This is the tr064 stuff from my ino file:

const char* fuser = "<user>";
const char* fpass = "<pwd>";
const char* IP = "<router-ip>";
const int PORT = 49000;
TR064 tr064_connection(PORT, IP, fuser, fpass);
String tr064_service = "urn:dslforum-org:service:X_VoIP:1";
String call_params[][2] = {{"NewX_AVM-DE_PhoneNumber", "**610"}};
String req[][2] = {{}};
String params1[][2] = {{}};
tr064_connection.action(tr064_service, "X_AVM-DE_DialNumber", call_params, 1);
tr064_connection.action(tr064_service, "X_AVM-DE_DialHangup");
This is the serial output before swapping the order for telegram notification (click to show) Connecting to ............... WiFi verbunden. IP-Adresse: Hostname: ESP-Klingel Klingel online. - - - - - - - - - - - - - - - - - - - [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Giving up the request [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Giving up the request Es klingelt an der Tür![HTTPS] Received payload telegram:
This is the serial output after swapping the order for telegram notification (click to show) Connecting to .......... WiFi verbunden. IP-Adresse: Hostname: ESP-Klingel Klingel online. - - - - - - - - - - - - - - - - - - - Es klingelt an der Tür![HTTPS] Received payload telegram: [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Giving up the request [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] nonce/realm request not successful! [action] Retrying in 5s [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Giving up the request
JimKnopfIoT commented 3 years ago

Today i successfully tried the curl script:

#!/bin/bash
credentials="<user>:<pwd>"
FB="fritz.box:49443"
Rufnummer="**611";

curl -4 -k --anyauth -u "${credentials}"                                    \
     "https://${FB}/upnp/control/x_voip"                                     \
     -H 'Content-Type: text/xml; charset="utf-8"'                           \
     -H 'SoapAction: urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' \
     -d '<?xml version="1.0" encoding="utf-8"?>
         <s:Envelope s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
                     xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
            <s:Body>
               <u:X_AVM-DE_DialNumber xmlns:u="urn:dslforum-org:service:X_VoIP:1">
                 <NewX_AVM-DE_PhoneNumber>'${Rufnummer}'</NewX_AVM-DE_PhoneNumber>
               </u:X_AVM-DE_DialNumber>
            </s:Body>
         </s:Envelope>' > /dev/null 2>&1

netcat -4 fritz.box 10

The phone (this time **611) rang immediately. In the ino file the value "IP" is of type char. In tr064.cpp and in ESPHTTPClient.cpp it is type String. I noticed that ESPHTTPClient.cpp will inspect the url String if it'S http or https. So instead of using the ip as a value of "IP", i inserted "https://fritz.box" as the value for "IP" in my ino file. I removed the "HTTP" from line 400 (String TR064 ... starts at line 399). Now line 399/400 of tr064.cpp looks like:

String TR064::httpRequest(String url, String xml, String soapaction, bool retry) {
deb_println("[HTTP] Prepare HTTP request to URL: " + _ip + ":" + _port + url + "'.", DEBUG_INFO);
I set debug level to verbose. This is the output (click to show) ```------------------Start----------------------------------- Connecting to ........... WiFi verbunden. IP-Adresse: 192.168.10.9 Hostname: ESP-Klingel Klingel online. - - - - - - - - - - - - - - - - - - - Es klingelt an der Tür![HTTPS] Received payload telegram: [init] Started initialization of the TR-064 library. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443/tr64desc.xml'. [HTTP] GET... [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443/tr64desc.xml'. [HTTP] GET... [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [initServiceURLs] Router name: [initServiceURLs] Router model: [initServiceURLs] Detected Services: [action] Call with parameters/arguments to be passed. [action] Attempt 1 of calling X_AVM-DE_DialNumber on urn:dslforum-org:service:X_VoIP:1 [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064**611 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064**611 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Response status: [action] Got an empty reply (e.g. http error). Trying again in 10s. [action] Attempt 2 of calling X_AVM-DE_DialNumber on urn:dslforum-org:service:X_VoIP:1 [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064**611 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064**611 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Response status: [action] Got an empty reply (e.g. http error). Trying again in 10s. [action] Attempt 3 of calling X_AVM-DE_DialNumber on urn:dslforum-org:service:X_VoIP:1 [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064**611 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064**611 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Response status: [action] Got an empty reply (e.g. http error). Trying again in 10s. [action] Attempt 4 of calling X_AVM-DE_DialNumber on urn:dslforum-org:service:X_VoIP:1 [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064**611 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064**611 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialNumber' [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Response status: [action] Got an empty reply (e.g. http error). Trying again in 10s. [action] Giving up the request. [action] simple [action] Call with parameters/arguments to be passed. [action] Attempt 1 of calling X_AVM-DE_DialHangup on urn:dslforum-org:service:X_VoIP:1 [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialHangup' [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialHangup' [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Response status: [action] Got an empty reply (e.g. http error). Trying again in 10s. [action] Attempt 2 of calling X_AVM-DE_DialHangup on urn:dslforum-org:service:X_VoIP:1 [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialHangup' [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialHangup' [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Response status: [action] Got an empty reply (e.g. http error). Trying again in 10s. [action] Attempt 3 of calling X_AVM-DE_DialHangup on urn:dslforum-org:service:X_VoIP:1 [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialHangup' [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialHangup' [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Response status: [action] Got an empty reply (e.g. http error). Trying again in 10s. [action] Attempt 4 of calling X_AVM-DE_DialHangup on urn:dslforum-org:service:X_VoIP:1 [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialHangup' [HTTP] Failed, message: 'connection failed' [HTTP] Trying again in 1s. [HTTP] Prepare HTTP request to URL: https://fritz.box:49443'. [HTTP] Printing the XML to be posted (between lines): --------------------------------- tr064 --------------------------------- [HTTP] POSTing the XML to SOAPACTION: 'urn:dslforum-org:service:X_VoIP:1#X_AVM-DE_DialHangup' [HTTP] Failed, message: 'connection failed' [HTTP] Giving up. [action] Response status: [action] Got an empty reply (e.g. http error). Trying again in 10s. [action] Giving up the request. **611 wurde benachrichtigt, gehe in den DeepSee --------------End------------------```

After some atempts, it gives up. The phone did not rang. The last line in the logfile is just the last printf message before the ESP12E goes into DeepSleep.

Any idea how to setup a proper https request? Seems http is disabled for requests like this on a Fritzbox.

PhilKaBoom commented 3 years ago

Same Issue at my setup

saak2820 commented 3 years ago

moin,

there is probably an error on loading the initial XML. as you can see the URL String has no Service "https://fritz.box:49443" This should be : http://fritz.box:49000/upnp/control/x_voip

you can further Debug:

If you use wireshark you will probably see the correct httpResponse with the XML https://www.it-techblog.de/fritzbox-und-wireshark-wlan-router-von-avm-monitoren-teil-1/11/2017/

you can debug HTTPClient as well. use HTTPClient in step 2

https://github.com/esp8266/Arduino/blob/master/doc/Troubleshooting/debugging.rst

Aypac commented 3 years ago

@JimKnopfIoT , @PhilKaBoom were either of you able to solve your issue yet?

JimKnopfIoT commented 3 years ago

@Aypac No. i didn'T solve the issue. I'm not a programmer. For now i disabled tr064 in my ESP doorbell. I only receive telegram msg's at the moment.

Aypac commented 2 years ago

@saak2820 does your PR solve this issue?

RoSchmi commented 2 years ago

@JimKnopfIoT @Aypac "Wählhilfe" in Fritzbox must be activated. https://service.avm.de/help/de/FRITZ-Box-Fon-WLAN-7490/017p1/hilfe_fon_waehlhilfe

saak2820 commented 2 years ago

@saak2820 does your PR solve this issue?

should be. if not there will be proper error messages

Aypac commented 2 years ago

I consider this issue closed then. Please re-open or open new issue and refer to this.