search

fhessel / esp32_https_server

Alternative ESP32 Webserver implementation for the ESP32 Arduino Core, supporting HTTPS and HTTP.
MIT License
342 stars 125 forks source link

Exception when decoding URL parameters #53

Closed vincent-bruel closed 4 years ago

vincent-bruel commented 5 years ago

First: thanks for your great work. Problem is: ESP32 restart with Backtrace in console when pushing a button with a formaction URL like this one: GET /coucou/Supprimer/2?

Viewing the backtrace (decoded, see below) It seems that there is a problem trying to get a parameter that doesn't exist (after the character '?'), so it should be a simple out of index problem, but I may be wrong.

Steps to reproduce:

  1. Serve the following file via a handleRoot 
    
<!DOCTYPE html>
<html>
<body>

Basic HTML Table

Firstname Lastname Age
Smith 50
Eve Jackson 94
John Doe 80 


3. On the client, click on the XXX button

4. See error in console:

23:54:28.701 -> [HTTPS:I] New connection. Socket FID=59 23:54:28.701 -> [HTTPS:I] Request: GET /posologie/Supprimer/2? (FID=58) 23:54:28.736 -> abort() was called at PC 0x401811bf on core 1 23:54:28.736 -> 23:54:28.736 -> Backtrace: 0x4008c454:0x3ffb1740 0x4008c685:0x3ffb1760 0x401811bf:0x3ffb1780 0x40181206:0x3ffb17a0 0x4016ebd3:0x3ffb17c0 0x401705ef:0x3ffb17e0 0x400dd82e:0x3ffb1a90 0x400dd9c3:0x3ffb1ab0 0x400ddf16:0x3ffb1ae0 0x400dbe05:0x3ffb1c30 0x400dd532:0x3ffb1df0 0x400d76d5:0x3ffb1e20 0x400d707e:0x3ffb1e40 0x400e4909:0x3ffb1fb0 0x40088b9d:0x3ffb1fd0 23:54:28.770 -> 23:54:28.770 -> Rebooting...


Backtrace decoded below:

Decoding stack results 0x4008c454: invoke_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c line 155 0x4008c685: abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c line 170 0x401811bf: cxxabiv1::terminate(void ()()) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc line 47 0x40181206: std::terminate() at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc line 57 0x4016ebd3: cxxabiv1::cxa_throw(void, std::type_info, void ()(void)) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_throw.cc line 87 0x401705ef: std::__throw_out_of_range_fmt(char const, ...) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/src/c++11/functexcept.cc line 104 0x400dd82e: std::vector , std::allocator >, std::allocator , std::allocator > > >::_M_range_check(unsigned int) const at c:\users\vb\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0\bits/stl_vector.h line 803 0x400dd9c3: httpsserver::ResourceParameters::setUrlParameter(unsigned char, std::cxx11::basic_string , std::allocator > const&) at c:\users\vb\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0\bits/stl_vector.h line 824 0x400ddf16: httpsserver::ResourceResolver::resolveNode(std::cxx11::basic_string , std::allocator > const&, std::__cxx11::basic_string , std::allocator > const&, httpsserver::ResolvedResource&, httpsserver::HTTPNodeType) at C:\Users\vb\Documents\Arduino\libraries\libraries\esp32_https_server-master\src\ResourceResolver.cpp line 126 0x400dbe05: httpsserver::HTTPConnection::loop() at C:\Users\vb\Documents\Arduino\libraries\libraries\esp32_https_server-master\src\HTTPConnection.cpp line 426 0x400dd532: httpsserver::HTTPServer::loop() at C:\Users\vb\Documents\Arduino\libraries\libraries\esp32_https_server-master\src\HTTPServer.cpp line 122 0x400d76d5: ServerWeb::loop() at C:\Users\vb\AppData\Local\Temp\arduino_build_9243\sketch\ServerWeb.cpp line 36



**ESP32 Module**
Please provide specifications of your module
Regular dev module like LOLIN32
- RAM/PSRAM: no PSRAM
- Flash Size: 4Mo
- Other special properties: 

**Software (please complete the following information if applicable)**
 - IDE and Version: tested both on VSCODE+Platformio and ArduinoIDE 1.8.10
 - OS: Windows 10 (updated)
 - Client used to access the server: google chrome

**Additional context**
EspExceptionDecoder to decode the stacktrace.

_Edit: Added syntax highlighting to your code_
fhessel commented 5 years ago

That looks indeed like an index problem. In ResourceResolver.cpp, it is assumed that there are actually parameters when the ? is present. Thanks for the report!

I'll see what I can do, but it may take until the latest PR is merged until the fix gets to the master branch.

fhessel commented 5 years ago

Do you have an additional wildcard route defined? Like /coucou/Supprimer/* or something similar? If I take the Static Page example from the library and just replace handleRoot(...) like shown as follows, I cannot reproduce the problem:

void handleRoot(HTTPRequest * req, HTTPResponse * res) {
  // Status code is 200 OK by default.
  // We want to deliver a simple HTML page, so we send a corresponding content type:
  res->setHeader("Content-Type", "text/html");
  // The response implements the Print interface, so you can use it just like
  // you would write to Serial etc.
  res->print("<!DOCTYPE html>\n");
  res->print("<html>\n");
  res->print("<body>\n");
  res->print("\n");
  res->print("<h2>Basic HTML Table</h2>\n");
  res->print("<form>\n");
  res->print("<table style=\"width:100%\">\n");
  res->print("  <tr>\n");
  res->print("    <th>Firstname</th>\n");
  res->print("    <th>Lastname</th> \n");
  res->print("    <th>Age</th>\n");
  res->print("  </tr>\n");
  res->print("  <tr>\n");
  res->print("    <td><input type=\"submit\" value=\"XXX\"  formaction=\"coucou/Supprimer/2\"></td>\n");
  res->print("    <td>Smith</td>\n");
  res->print("    <td>50</td>\n");
  res->print("  </tr>\n");
  res->print("  <tr>\n");
  res->print("    <td>Eve</td>\n");
  res->print("    <td>Jackson</td>\n");
  res->print("    <td>94</td>\n");
  res->print("  </tr>\n");
  res->print("  <tr>\n");
  res->print("    <td>John</td>\n");
  res->print("    <td>Doe</td>\n");
  res->print("    <td>80</td>\n");
  res->print("  </tr>\n");
  res->print("</table>\n");
  res->print("</form>\n");
  res->print("</body>\n");
  res->print("</html>\n");
}

However, I only have Chrome and Firefox on Linux for testing, so I cannot preclude that the cause is somehow related to that. Interestingly, only Chrome creates that additional question mark, Firefox doesn't.

vincent-bruel commented 5 years ago

Hi, Yes, sorry I forgot to mention that: ResourceNode nodeSupprimerLigne = new ResourceNode("//Supprimer/*", "GET", &ServerWeb::handleSupprimerLigne);

vincent-bruel commented 5 years ago

Previous message is wrong, cause I've forgot to put the code into backquotes so the first star disapeared. URL pattern is: slash star slash Supprimer slash star ResourceNode *nodeSupprimerLigne = new ResourceNode("/*/Supprimer/*", "GET", &ServerWeb::handleSupprimerLigne);

vincent-bruel commented 5 years ago

My project is too big to cut it into pieces. Anyway, to reproduce the bug in a very similar environment, I've started from the Self signed certificate example, and added the following code line 121 just before secureServer->start() in order to create my own Wifi Access point. "Root" node generates an html page but the X button generating the URL. "S" node generates the same html code with another title. And I cannot reproduce the bug neither with the following code:

self-signed-certificates.ino

```c++ /** * Example for the ESP32 HTTP(S) Webserver * * IMPORTANT NOTE: * To run this script, you need to * 1) Enter your WiFi SSID and PSK below this comment * * This script will install an HTTPS Server on your ESP32 with the following * functionalities: * - Show simple page on web server root * - 404 for everything else * * In contrast to the other examples, the certificate and the private key will be * generated on the ESP32, so you do not need to provide them here. * (this means no need to run create_cert.sh) */ // TODO: Configure your WiFi here #define WIFI_SSID "your ssid" #define WIFI_PSK "your pwd" // We will use wifi #include // Includes for the server #include #include #include #include // The HTTPS Server comes in a separate namespace. For easier use, include it here. using namespace httpsserver; SSLCert * cert; HTTPSServer * secureServer; // Declare some handler functions for the various URLs on the server void handleS(HTTPRequest * req, HTTPResponse * res); void handleRoot(HTTPRequest * req, HTTPResponse * res); void handle404(HTTPRequest * req, HTTPResponse * res); void setup() { // For logging Serial.begin(115200); delay(3000); // wait for the monitor to reconnect after uploading. Serial.println("Creating a new self-signed certificate."); Serial.println("This may take up to a minute, so be patient ;-)"); // First, we create an empty certificate: cert = new SSLCert(); // Now, we use the function createSelfSignedCert to create private key and certificate. // The function takes the following paramters: // - Key size: 1024 or 2048 bit should be fine here, 4096 on the ESP might be "paranoid mode" // (in generel: shorter key = faster but less secure) // - Distinguished name: The name of the host as used in certificates. // If you want to run your own DNS, the part after CN (Common Name) should match the DNS // entry pointing to your ESP32. You can try to insert an IP there, but that's not really good style. // - Dates for certificate validity (optional, default is 2019-2029, both included) // Format is YYYYMMDDhhmmss int createCertResult = createSelfSignedCert( *cert, KEYSIZE_2048, "CN=myesp32.local,O=FancyCompany,C=DE", "20190101000000", "20300101000000" ); // Now check if creating that worked if (createCertResult != 0) { Serial.printf("Cerating certificate failed. Error Code = 0x%02X, check SSLCert.hpp for details", createCertResult); while(true) delay(500); } Serial.println("Creating the certificate was successful"); // If you're working on a serious project, this would be a good place to initialize some form of non-volatile storage // and to put the certificate and the key there. This has the advantage that the certificate stays the same after a reboot // so your client still trusts your server, additionally you increase the speed-up of your application. // Some browsers like Firefox might even reject the second run for the same issuer name (the distinguished name defined above). // // Storing: // For the key: // cert->getPKLength() will return the length of the private key in byte // cert->getPKData() will return the actual private key (in DER-format, if that matters to you) // For the certificate: // cert->getCertLength() and ->getCertData() do the same for the actual certificate data. // Restoring: // When your applications boots, check your non-volatile storage for an existing certificate, and if you find one // use the parameterized SSLCert constructor to re-create the certificate and pass it to the HTTPSServer. // // A short reminder on key security: If you're working on something professional, be aware that the storage of the ESP32 is // not encrypted in any way. This means that if you just write it to the flash storage, it is easy to extract it if someone // gets a hand on your hardware. You should decide if that's a relevant risk for you and apply countermeasures like flash // encryption if neccessary // We can now use the new certificate to setup our server as usual. secureServer = new HTTPSServer(cert); // Connect to WiFi Serial.println("Setting up WiFi"); WiFi.begin(WIFI_SSID, WIFI_PSK); while (WiFi.status() != WL_CONNECTED) { Serial.print("."); delay(500); } Serial.print("Connected. IP="); Serial.println(WiFi.localIP()); // For every resource available on the server, we need to create a ResourceNode // The ResourceNode links URL and HTTP method to a handler function ResourceNode * nodeRoot = new ResourceNode("/", "GET", &handleRoot); ResourceNode * nodeS = new ResourceNode("/*/Supprimer/*", "GET", &handleS); ResourceNode * node404 = new ResourceNode("", "GET", &handle404); // Add the root node to the server secureServer->registerNode(nodeRoot); secureServer->registerNode(nodeS); // Add the 404 not found node to the server. secureServer->setDefaultNode(node404); Serial.println("Setting up WiFi"); WiFi.softAP("MYSSID"); // TODO: AFFICHER DANS MATRIX LED Serial.print("Wifi SoftAP IP="); Serial.println(WiFi.softAPIP()); Serial.println("Starting server..."); secureServer->start(); if (secureServer->isRunning()) { Serial.println("Server ready."); } } void loop() { // This call will let the server do its work secureServer->loop(); // Other code would go here... delay(1); } void handleS(HTTPRequest * req, HTTPResponse * res) { // Status code is 200 OK by default. // We want to deliver a simple HTML page, so we send a corresponding content type: res->setHeader("Content-Type", "text/html"); // The response implements the Print interface, so you can use it just like // you would write to Serial etc. res->print("\n"); res->print("\n"); res->print("\n"); res->print("\n"); res->print("

SUPPRIMER

\n"); res->print("
\n"); res->print("\n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print("
FirstnameLastnameAge
Smith50
EveJackson94
JohnDoe80
\n"); res->print("
\n"); res->print("\n"); res->print("\n"); } void handleRoot(HTTPRequest * req, HTTPResponse * res) { // Status code is 200 OK by default. // We want to deliver a simple HTML page, so we send a corresponding content type: res->setHeader("Content-Type", "text/html"); // The response implements the Print interface, so you can use it just like // you would write to Serial etc. res->print("\n"); res->print("\n"); res->print("\n"); res->print("\n"); res->print("

Basic HTML Table

\n"); res->print("
\n"); res->print("\n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print(" \n"); res->print("
FirstnameLastnameAge
Smith50
EveJackson94
JohnDoe80
\n"); res->print("
\n"); res->print("\n"); res->print("\n"); } void handle404(HTTPRequest * req, HTTPResponse * res) { // Discard request body, if we received any // We do this, as this is the default node and may also server POST/PUT requests req->discardRequestBody(); // Set the response status res->setStatusCode(404); res->setStatusText("Not Found"); // Set content type of the response res->setHeader("Content-Type", "text/html"); // Write a tiny HTTP page res->println(""); res->println(""); res->println("Not Found"); res->println("

404 Not Found

The requested resource was not found on this server.

"); res->println(""); } ```

Edit: Again, added syntax highlighting to your code

vincent-bruel commented 5 years ago

Hi, I've made a patch to HTTPConnection.cpp, adding this code after line 374 to remove the trailing '?'

if (_httpResource.find('?') == _httpResource.size() - 1) {
  Serial.println("Removing trailing '?' at the end of the request");
  _httpResource = _httpResource.substr(0, _httpResource.size() - 1);
}

After that my URL /contacts/Supprimer/3 should match a ResourceNode defined with "/contacts/Supprimer/*"

But still an exception raised at line 118 in ResourceResolver.cpp, just after seeing "Testing parameter match on /contacts/Supprimer/*" in the console:

Decoding stack results
0x4008c454: invoke_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c line 155
0x4008c685: abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c line 170
0x40181b33: __cxxabiv1::__terminate(void (*)()) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc line 47
0x40181b7a: std::terminate() at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc line 57
0x4016f547: __cxxabiv1::__cxa_throw(void*, std::type_info*, void (*)(void*)) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_throw.cc line 87
0x40170f63: std::__throw_out_of_range_fmt(char const*, ...) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/src/c++11/functexcept.cc line 104
0x400ddd72: std::vector   , std::allocator  >, std::allocator   , std::allocator  > > >::_M_range_check(unsigned int) const at c:\users\vb\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0\bits/stl_vector.h line 803
0x400ddf07: httpsserver::ResourceParameters::setUrlParameter(unsigned char, std::__cxx11::basic_string  , std::allocator  > const&) at c:\users\vb\appdata\local\arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf\include\c++\5.2.0\bits/stl_vector.h line 824
0x400de46a: httpsserver::ResourceResolver::resolveNode(std::__cxx11::basic_string  , std::allocator  > const&, std::__cxx11::basic_string  , std::allocator  > const&, httpsserver::ResolvedResource&, httpsserver::HTTPNodeType) at C:\Users\vb\Documents\Arduino\libraries\libraries\esp32_https_server-master\src\ResourceResolver.cpp line 118
0x400dc212: httpsserver::HTTPConnection::loop() at C:\Users\vb\Documents\Arduino\libraries\libraries\esp32_https_server-master\src\HTTPConnection.cpp line 435
0x400dda76: httpsserver::HTTPServer::loop() at C:\Users\vb\Documents\Arduino\libraries\libraries\esp32_https_server-master\src\HTTPServer.cpp line 122
0x400d7711: ServerWeb::loop() at C:\Users\vb\AppData\Local\Temp\arduino_build_346644\sketch\ServerWeb.cpp line 36

Edit: Syntax highlighting

vincent-bruel commented 5 years ago

The bug line 118 of ResourceResolver.cpp is also activated if I define the resource node with a star:

ResourceNode *nodeSupprimerLigneContacts = new ResourceNode("/contacts/Supprimer/*", "GET", &ServerWeb::handleSupprimerLigne);

And activate it with an URL like /contacts/Supprimer/3

Edit: Highlighting

fhessel commented 4 years ago

I did a complete rewrite of the resource resolving. Could you give a try to this branch and let me know if that solves your problem? https://github.com/fhessel/esp32_https_server/tree/issue-53

I think it had to do with the usage of capacity and size in ResourceParameters::setUrlParameter. But I rewrote the whole thing, because I did only consider URL parameters at the end (so /route/*/* but not /*/action/*) before, and that's no longer the case. You can now put the placeholder anywhere you like, as long as it's enclosed by slashes or at the end of the pattern. So /*/Supprimer/* should be perfectly fine.

vincent-bruel commented 4 years ago

Hi, that's great news :) ! I will give it a try ASAP and let you know the result here. Thanks very much anyway for your help and your library.

vincent-bruel commented 4 years ago

It seems to work very well. I've tested the following URLs in a ResourceNode with success: "/*/xxx" "/*/yyy/*" Thanks

fhessel commented 4 years ago

Thanks for the quick testing. I'll queue the change as a PR and include it in the next release. You should be able to work with the branched version of the lib until then, as the fix is the only difference to master/develop.

vincent-bruel commented 4 years ago

OK, that's exactly what I've planned to do.