Generic HTTP Advanced [TESTING] with token

[popaserge]

Equipment used Robotdyn Wifi D1 mini and Sonoff 4 CH Pro R2

Last release used on Wifi D1 mini ESP_Easy_mega-20190107_dev_ESP8266_4096.bin

2 questions and problem

Security scheme I would like to use a token instead of login + password to connect to a monitoring server. I had a general look at the C011.ino code to try to understand. This Http advance controller seems to work only according to the user + password security scheme. I tried to use it without user and password but with a token in the URI field The server seems to receive that information but not the content of the Body field and gives an error. Any possibility to improve the security choice? Then where to write that token as the http request should be:

http://server_ip_or_hostname:port/token?content=body ... body has to be xml or json string (see hereafter) token can be as long as 64 characters body can be as long as 512 characters limitations about that values?

Problem with parsing json body text : quote " replaced by tick '

Example as written on controller page HTTP Body:

{"result": [{"Channel": "%vname1%","Value":%val1%},{"Channel":"%vname2%","Value":%val2%}]}

Example as Obtained on log result:

14673232: HTTP before parsing: 14673232: {'result': [{'Channel': '%vname1%','Value':%val1%},{'Channel':'%vname2%','Value':%val2%}]} 14673234: HTTP after parsing: 14673234: {'result': [{'Channel': '%vname1%','Value':%val1%},{'Channel':'%vname2%','Value':%val2%}]} 14673242: HTTP after replacements: 14673242: {'result': [{'Channel': 'Eau','Value':19.25},{'Channel':'Air','Value':19.94}]}

[TD-er]

Not entirely sure if it is a feature request or a bug. (but I am sure I may need some sleep first ;) )

[popaserge]

Hope you had time to sleep! Ok here more detailed information

If I use firefox to send following request

http://192.168.2.150:5050/25178B03-5869-4D92-B650-F22EB65984E9?content={"prtg":{"result":[{"Channel":"Eau","Unit":"Temperature","Float":1 ,"Decimalmode":2,"Value":19.87},{"Channel":"Air","Unit":"Temperature","Float":1,"Decimalmode":2,"Value":20.69}]}}"

the server I test is taking the values in account and is recording following txt file,

Command: GET Uri: /25178B03-5869-4D92-B650-F22EB65984E9 Remote IP: 192.168.2.152 Form Params: Query Params: content={%22prtg%22:{%22result%22:[{%22Channel%22:%22Eau%22,%22Unit%22:%22Temperature%22,%22Float%22:1%20,%22Decimalmode%22:2,%22Value%22:19.87},{%22Channel%22:%22Air%22,%22Unit%22:%22Temperature%22,%22Float%22:1,%22Decimalmode%22:2,%22Value%22:20.69}]}} Version: HTTP/1.1

the quotes " from the json string are well received as url encoded %22

If I try to send the same with ESPeasy with following controller

The server is recording:

Command: GET ContentType: Application/Json Uri: /25178B03-5869-4D92-B650-F22EB65984E9 Remote IP: 192.168.2.177 Form Params: Query Params: Version: HTTP/1.1

I thought the body was not recorded because of the parsing error reported yesterday ' or %27 in place of " or %22 which does not give a valid json string but there seems to be other issues as the log gives the following

Case 1 without "?content=" inside the URI

26385: EVENT: Clock#Time=Wed,18:52 26430: EVENT: Clock#Time=Wed,18:52 Processing time:45 milliSeconds 33938: WD : Uptime 1 ConnectFailures 0 FreeMem 12400 33939: UDP : Send Sysinfo message 34827: LoopStats: shortestLoop: 106 longestLoop: 1011925 avgLoopDuration: 180.87 loopCounterMax: 283018 loopCounterLast: 161448 countF 34828: Scheduler stats: (called/tasks/max_length/idle%) 161444/1826/10/75.10 62784: GET /25178B03-5869-4D92-B650-F22EB65984E9 HTTP/1.1^^Host: 192.168.2.150:5050^^User-Agent: ESP Easy/20103/Jan 7 2019 03:20:08^^ 62786: HTTP after replacements: 62786: GET /25178B03-5869-4D92-B650-F22EB65984E9 HTTP/1.1^^Host: 192.168.2.150:5050^^User-Agent: ESP Easy/20103/Jan 7 2019 03:20:08^^ 62788: HTTP before parsing: 62788: {'prtg':{'result':[{'Channel':'%vname1%','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':%val1%},{'Channel':'%vname2%', 62791: HTTP after parsing: 62791: {'prtg':{'result':[{'Channel':'%vname1%','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':%val1%},{'Channel':'%vname2%', 62800: HTTP after replacements: 62801: {'prtg':{'result':[{'Channel':'Eau','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':19.69},{'Channel':'Air','Unit':'Tem 62880: HTTP : C011 written to client (130/386) 62884: HTTP/1.1 411 Length Required^ 62884: HTTP : C011 Error: HTTP/1.1 411 Length Required^ 62884: GET /25178B03-5869-4D92-B650-F22EB65984E9 HTTP/1.1^^Host: 192.168.2.150:5050^^User-Agent: ESP Easy/20103/Jan 7 2019 03:20:08^^ 62885: Connection: close^ 62886: Content-Type: text/html; charset=ISO-8859-1^ 62886: Content-Length: 0^ 62886: ^ 62887: HTTP : C011 closing connection 63938: WD : Uptime 1 ConnectFailures 0 FreeMem 11424

Case 2 with "?content=" inside the URI

33912: WD : Uptime 1 ConnectFailures 0 FreeMem 12400 33913: UDP : Send Sysinfo message 34800: LoopStats: shortestLoop: 106 longestLoop: 1008880 avgLoopDuration: 181.20 loopCounterMax: 283018 loopCounterLast: 161165 countF 34800: Scheduler stats: (called/tasks/max_length/idle%) 161161/1829/10/75.10 42356: EVENT: Clock#Time=Wed,19:05 42403: EVENT: Clock#Time=Wed,19:05 Processing time:47 milliSeconds 62759: GET /25178B03-5869-4D92-B650-F22EB65984E9?content= HTTP/1.1^^Host: 192.168.2.150:5050^^User-Agent: ESP Easy/20103/Jan 7 2019 0 62762: HTTP after replacements: 62762: GET /25178B03-5869-4D92-B650-F22EB65984E9?content= HTTP/1.1^^Host: 192.168.2.150:5050^^User-Agent: ESP Easy/20103/Jan 7 2019 0 62764: HTTP before parsing: 62764: {'prtg':{'result':[{'Channel':'%vname1%','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':%val1%},{'Channel':'%vname2%', 62766: HTTP after parsing: 62766: {'prtg':{'result':[{'Channel':'%vname1%','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':%val1%},{'Channel':'%vname2%', 62777: HTTP after replacements: 62777: {'prtg':{'result':[{'Channel':'Eau','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':19.81},{'Channel':'Air','Unit':'Tem 62856: HTTP : C011 written to client (139/395) 62861: HTTP/1.1 411 Length Required^ 62861: HTTP : C011 Error: HTTP/1.1 411 Length Required^ 62861: GET /25178B03-5869-4D92-B650-F22EB65984E9?content= HTTP/1.1^^Host: 192.168.2.150:5050^^User-Agent: ESP Easy/20103/Jan 7 2019 0 62862: Connection: close^ 62863: Content-Type: text/html; charset=ISO-8859-1^ 62864: Content-Length: 0^ 62864: ^ 62864: HTTP : C011 closing connection 63910: WD : Uptime 1 ConnectFailures 0 FreeMem 11440

It seems that it remains several problems:

1/ ticks ' in place of quotes " in the json string 2/ the strings before and after parsing and replacements are truncated (reality or just incomplete output of the log function limited to 128 bytes) 3/ it seems that the C011 plugin knows the Length of the strings that are parsed in the 2 cases: 386 and 395 which differ by 9 which is the length of ?content= But in the 2 cases only a part of the strings is transmitted 130 and 139. 4/ it seems that the server asks for a Content-Length and sends an error 411 : HTTP/1.1 411 Length Required^ 5/ if I put a Content-Length:0 line in the Header, the server sends a 200 OK (see herafter) and again only 155/411 are send to the server

Case 3 with header Content-Length:0

33907: WD : Uptime 1 ConnectFailures 0 FreeMem 12384 33908: UDP : Send Sysinfo message 34794: LoopStats: shortestLoop: 106 longestLoop: 1013362 avgLoopDuration: 180.81 loopCounterMax: 283018 loopCounterLast: 161505 countF 34795: Scheduler stats: (called/tasks/max_length/idle%) 161501/1827/10/75.30 54352: EVENT: Clock#Time=Wed,19:38 54394: EVENT: Clock#Time=Wed,19:38 Processing time:42 milliSeconds 62755: GET /25178B03-5869-4D92-B650-F22EB65984E9?content= HTTP/1.1^^Host: 192.168.2.150:5050^^User-Agent: ESP Easy/20103/Jan 7 2019 0 62759: HTTP after replacements: 62759: GET /25178B03-5869-4D92-B650-F22EB65984E9?content= HTTP/1.1^^Host: 192.168.2.150:5050^^User-Agent: ESP Easy/20103/Jan 7 2019 0 62760: HTTP before parsing: 62760: {'prtg':{'result':[{'Channel':'%vname1%','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':%val1%},{'Channel':'%vname2%', 62763: HTTP after parsing: 62763: {'prtg':{'result':[{'Channel':'%vname1%','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':%val1%},{'Channel':'%vname2%', 62773: HTTP after replacements: 62773: {'prtg':{'result':[{'Channel':'Eau','Unit':'Temperature','Float':1 ,'Decimalmode':2,'Value':19.81},{'Channel':'Air','Unit':'Tem 62845: HTTP : C011 connecting to 192.168.2.150:5050 62851: HTTP : C011 written to client (155/411) 62864: HTTP/1.1 200 OK^ 62864: HTTP : C011 Success! HTTP/1.1 200 OK^ 62864: HTTP : C011 closing connection 63905: WD : Uptime 1 ConnectFailures 0 FreeMem 11864

Hope this helps understanding and doing a more universal C011 plugin or give me suggestions .

[popaserge]

Oups

[popaserge]

@TD-er Do you have enough info to define if it is a bug or a feature request. Many thanks

[popaserge]

@TD-er Hello I tried again to see if the issue was related to a bug or not. Therefore I used Wireshark on the server receiving the stream from ESPeasy. Content_Length : 191 is well sent by Espeasy but it seems that there is an arror just before, which may be the cause for the "Content-Length required" sent by the server:

62884: HTTP/1.1 411 Length Required^ 62884: HTTP : C011 Error: HTTP/1.1 411 Length Required^ (see more details on previous post on the issue)

The error given by wireshark is: "Leading CRLF previous message in the stream may have an extra CRLF" see screenshots.

I tried to look to the _C011.ino file but I am not a specialist of coding. Nevertheless maybe the error occurs in the area after line 168. There was already a correction made 2 month ago in the same area.

Is it possible that from other parts of the code other \r\n are remaining or maybe line 182 is involved. In that case possible solution could be to remove all \r\n existing at the end of the string and then add only the 2 \r\n that seem to be necessary to allow a good communication.

Complete Wireshark file available if required.

Thanks for help.