search

espressif / arduino-esp32

Arduino core for the ESP32
GNU Lesser General Public License v2.1
13.37k stars 7.37k forks source link

http client add.header("Authorization", ... ) broken? #4951

Closed ElToberino closed 3 years ago

ElToberino commented 3 years ago

Hardware:

Board: DOIT ESP32 DEVKIT V1 Core Installation version: 1.0.5 IDE name: Arduino IDE Flash Frequency: 80Mhz PSRAM enabled: ?no? ?yes? Upload Speed: ?115200? Computer OS: Windows 10

Description:

I'm running a programm calling the Spotify API. The sketch works fine with Core V 1.0.4. - but with V 1.0.5 the code posted below doesn't work any more. The server call always results in a "http error: -1" message. I'm far from being an expert but I doubt there's something wrong with the code because it worked fine with earlier core versions. Thanks in advance for your feedback.

Edit: Perhaps is could also be related to the Base64 encoding - I read about problems with it in other issues. But I really don't know why it stopped working, so any ideas are welcome.

Sketch:


const char* clientID = "XXXXXX";
const char* clientSecret ="XXXXXX";
const char* fingerprint = "XXXXXX";
String refreshtoken ="XXXXXX";                                                                           
String auth;
String accesstoken;
uint16_t expiretime;

void callSpotify(){
    auth = "Basic " + base64::encode(String(clientID) + ":" + String(clientSecret));           
    Serial.println("Calling Spotify for accesstoken");

    HTTPClient http;

    if(http.begin("https://accounts.spotify.com/api/token", fingerprint)){
      http.addHeader("Content-Type", "application/x-www-form-urlencoded"); 
      http.addHeader("Authorization", auth);

      String url = "grant_type=refresh_token&refresh_token=" + refreshtoken;
      int httpCode =  http.POST(url);
       Serial.println(httpCode);

      if(httpCode == 200) {
        DynamicJsonDocument doc(400);
        DeserializationError error = deserializeJson(doc, http.getStream());
        if (error) {
            Serial.println("deserializeJson() failed: ");
            Serial.println(error.c_str());
            return;
        }
        const char* newtoken = doc["access_token"];
        expiretime = doc["expires_in"];

        accesstoken = "Bearer " + String(newtoken);     
       } else {
        Serial.printf("http error: %i\n", httpCode);
        }
    } else {
        Serial.println("Unable to connect to Spotify");
    }
    http.end();
}
me-no-dev commented 3 years ago

can you please select the ESP32 Dev Board in the board menu and set the Debug level to Verbose, then try again and post the logs from Serial here?

ElToberino commented 3 years ago

Thanks for your answer. Here the debug output, as you can see, my sketch makes 5 connection attempts:

Debug Messages:

Calling Spotify for accesstoken
[V][HTTPClient.cpp:239] beginInternal(): url: https://accounts.spotify.com/api/token
[D][HTTPClient.cpp:287] beginInternal(): host: accounts.spotify.com port: 443 url: /api/token
[D][HTTPClient.cpp:573] sendRequest(): request type: 'POST' redirCount: 0

[V][ssl_client.cpp:59] start_ssl_client(): Free internal heap before TLS 239700
[V][ssl_client.cpp:65] start_ssl_client(): Starting socket
[V][ssl_client.cpp:104] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:113] start_ssl_client(): Setting up the SSL/TLS structure...
[V][ssl_client.cpp:129] start_ssl_client(): Loading CA cert
[E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected
[E][WiFiClientSecure.cpp:127] connect(): start_ssl_client: -8576
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[D][HTTPClient.cpp:1112] connect(): failed connect to accounts.spotify.com:443
[W][HTTPClient.cpp:1411] returnError(): error(-1): connection refused
[D][HTTPClient.cpp:394] disconnect(): tcp is closed

[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][HTTPClient.cpp:239] beginInternal(): url: https://accounts.spotify.com/api/token
[D][HTTPClient.cpp:287] beginInternal(): host: accounts.spotify.com port: 443 url: /api/token
[D][HTTPClient.cpp:573] sendRequest(): request type: 'POST' redirCount: 0

[V][ssl_client.cpp:59] start_ssl_client(): Free internal heap before TLS 239396
[V][ssl_client.cpp:65] start_ssl_client(): Starting socket
[V][ssl_client.cpp:104] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:113] start_ssl_client(): Setting up the SSL/TLS structure...
[V][ssl_client.cpp:129] start_ssl_client(): Loading CA cert
[E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected
[E][WiFiClientSecure.cpp:127] connect(): start_ssl_client: -8576
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[D][HTTPClient.cpp:1112] connect(): failed connect to accounts.spotify.com:443
[W][HTTPClient.cpp:1411] returnError(): error(-1): connection refused
[D][HTTPClient.cpp:394] disconnect(): tcp is closed

[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][HTTPClient.cpp:239] beginInternal(): url: https://accounts.spotify.com/api/token
[D][HTTPClient.cpp:287] beginInternal(): host: accounts.spotify.com port: 443 url: /api/token
[D][HTTPClient.cpp:573] sendRequest(): request type: 'POST' redirCount: 0

[V][ssl_client.cpp:59] start_ssl_client(): Free internal heap before TLS 239088
[V][ssl_client.cpp:65] start_ssl_client(): Starting socket
[V][ssl_client.cpp:104] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:113] start_ssl_client(): Setting up the SSL/TLS structure...
[V][ssl_client.cpp:129] start_ssl_client(): Loading CA cert
[E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected
[E][WiFiClientSecure.cpp:127] connect(): start_ssl_client: -8576
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[D][HTTPClient.cpp:1112] connect(): failed connect to accounts.spotify.com:443
[W][HTTPClient.cpp:1411] returnError(): error(-1): connection refused
[D][HTTPClient.cpp:394] disconnect(): tcp is closed

[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][HTTPClient.cpp:239] beginInternal(): url: https://accounts.spotify.com/api/token
[D][HTTPClient.cpp:287] beginInternal(): host: accounts.spotify.com port: 443 url: /api/token
[D][HTTPClient.cpp:573] sendRequest(): request type: 'POST' redirCount: 0

[V][ssl_client.cpp:59] start_ssl_client(): Free internal heap before TLS 238776
[V][ssl_client.cpp:65] start_ssl_client(): Starting socket
[V][ssl_client.cpp:104] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:113] start_ssl_client(): Setting up the SSL/TLS structure...
[V][ssl_client.cpp:129] start_ssl_client(): Loading CA cert
[E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected
[E][WiFiClientSecure.cpp:127] connect(): start_ssl_client: -8576
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[D][HTTPClient.cpp:1112] connect(): failed connect to accounts.spotify.com:443
[W][HTTPClient.cpp:1411] returnError(): error(-1): connection refused
[D][HTTPClient.cpp:394] disconnect(): tcp is closed

[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][HTTPClient.cpp:239] beginInternal(): url: https://accounts.spotify.com/api/token
[D][HTTPClient.cpp:287] beginInternal(): host: accounts.spotify.com port: 443 url: /api/token
[D][HTTPClient.cpp:573] sendRequest(): request type: 'POST' redirCount: 0

[V][ssl_client.cpp:59] start_ssl_client(): Free internal heap before TLS 238464
[V][ssl_client.cpp:65] start_ssl_client(): Starting socket
[V][ssl_client.cpp:104] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:113] start_ssl_client(): Setting up the SSL/TLS structure...
[V][ssl_client.cpp:129] start_ssl_client(): Loading CA cert
[E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected
[E][WiFiClientSecure.cpp:127] connect(): start_ssl_client: -8576
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[D][HTTPClient.cpp:1112] connect(): failed connect to accounts.spotify.com:443
[W][HTTPClient.cpp:1411] returnError(): error(-1): connection refused
[D][HTTPClient.cpp:394] disconnect(): tcp is closed

[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
[V][ssl_client.cpp:265] stop_ssl_socket(): Cleaning SSL connection.
me-no-dev commented 3 years ago

Your fingerprint is not formatted correctly :) [E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected

ElToberino commented 3 years ago

I'm really sorry, but I don't know how the right format should be. Until the update to V 1.0.5. it worked fine with the following definition of fingerprint: const char* fingerprint = "B9:79:6B:CE:FD:61:21:97:A7:02:90:EE:DA:CD:F0:A0:44:13:0E:EB";

I searched the web and tried some variants, but I always get the same error message as above. Could you please give me a hint how fingerprint must be formatted correctly.

Thank you and best regards!

lbernstone commented 3 years ago

@me-no-dev There needs to be some way in HTTPClient to set WiFiClientSecure::setInsecure.

ElToberino commented 3 years ago

Meanwhile I found out something interesting: In my code I use the same fingerprint (!) formated as described above for another request:

HTTPClient http;
  if(http.begin("https://accounts.spotify.com/api/token", fingerprint)) {

    http.addHeader("Content-Type", "application/x-www-form-urlencoded");
    String completeAuth = "grant_type=authorization_code&code=" + oneTimeCode + "&redirect_uri=" + String(redirectUri) + "&client_id=" + String(clientID) + "&client_secret=" + String(clientSecret);
    int httpCode =  http.POST(completeAuth);
      Serial.println(httpCode);

    if(httpCode == 200) {                     
      DynamicJsonDocument doc(500);                                                 
      DeserializationError error = deserializeJson(doc, http.getStream());
      if (error) {
          Serial.println("deserializeJson() failed: ");
          Serial.println(error.c_str());
      return;
      }
      const char* initialAccesstoken = doc["access_token"];                      
      const char* initialRefreshtoken = doc["refresh_token"];                        
      accesstoken = "Bearer " + String(initialAccesstoken);
      refreshtoken = initialRefreshtoken;
      expiretime = doc["expires_in"];
      auth = "Basic " + base64::encode(String(clientID) + ":" + String(clientSecret));
      authSuccess = true;
      } else {
        Serial.printf("http error: %i\n", httpCode);
      } 
    } else {
        Serial.println("Unable to connect to Spotify");
    }
  http.end();

This http request works fine, with the same the fingerprint!

Once again - to spare you scrolling - the non-working code:

void callSpotify(){
    auth = "Basic " + base64::encode(String(clientID) + ":" + String(clientSecret));           
    Serial.println("Calling Spotify for accesstoken");

    HTTPClient http;

    if(http.begin("https://accounts.spotify.com/api/token", fingerprint)){
      http.addHeader("Content-Type", "application/x-www-form-urlencoded"); 
      http.addHeader("Authorization", auth);

      String url = "grant_type=refresh_token&refresh_token=" + refreshtoken;
      int httpCode =  http.POST(url);
       Serial.println(httpCode);

      if(httpCode == 200) {
        DynamicJsonDocument doc(400);
        DeserializationError error = deserializeJson(doc, http.getStream());
        if (error) {
            Serial.println("deserializeJson() failed: ");
            Serial.println(error.c_str());
            return;
        }
        const char* newtoken = doc["access_token"];
        expiretime = doc["expires_in"];

        accesstoken = "Bearer " + String(newtoken);     
       } else {
        Serial.printf("http error: %i\n", httpCode);
        }
    } else {
        Serial.println("Unable to connect to Spotify");
    }
    http.end();
}

I really can't see any reason why this doesn't work. As I wrote above, everything (both calls) worked without any issues with core V1.0.4

lbernstone commented 3 years ago

The issue is that previous to 1.0.5, TLS could be established with no CA certificate. In 1.0.5, unsecured connections are not allowed by default. There needs to be a method in HTTPClient to pass through the setInsecure to WiFiClientSecure which runs in the background.

ElToberino commented 3 years ago

Thanks for your answer! I don't want to get on your nerves - but could you please explain me the differnce between these two http client calls. I'm not able to see the difference between the working and the non-working one.

me-no-dev commented 3 years ago

@lbernstone http.begin(url, const char *) expects the const char * to be the root certificate. https://github.com/espressif/arduino-esp32/blob/master/libraries/HTTPClient/src/HTTPClient.h#L160

If NULL is given, it will run with WiFiClientSecure::setInsecure. I am not sure what that fingerprint is supposed to be but the API is expecting root certificate.

me-no-dev commented 3 years ago

@ElToberino could you please run your code with the very latest master instead from the board manager version 1.0.5? That way we will be on the same page and also some hotfixes have been added after 1.0.5 that could be influencing the result

ElToberino commented 3 years ago

@me-no-dev I gave it a try, set up a new portable arduino install and was able to install the current master branch. Unfortunately with the same result - the http call fails with [E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected

I did some further investigation and must state that in all calls where the fingerpint is added with this pattern, it doesn't work any longer. const char* fingerprint = "B9:79:6B:CE:FD:61:21:97:A7:02:90:EE:DA:CD:F0:A0:44:13:0E:EB";

I followed the debug output of the working V 1.0.4. and found a warning message in the Debug output:

[V][HTTPClient.cpp:235] beginInternal(): url: https://accounts.spotify.com/api/token
[D][HTTPClient.cpp:276] beginInternal(): host: accounts.spotify.com port: 443 url: /api/token
[V][ssl_client.cpp:56] start_ssl_client(): Free internal heap before TLS 219400
[V][ssl_client.cpp:58] start_ssl_client(): Starting socket
[V][ssl_client.cpp:93] start_ssl_client(): Seeding the random number generator
[V][ssl_client.cpp:102] start_ssl_client(): Setting up the SSL/TLS structure...
[I][ssl_client.cpp:156] start_ssl_client(): WARNING: Use certificates for a more secure communication!
[V][ssl_client.cpp:180] start_ssl_client(): Setting hostname for TLS session...
[V][ssl_client.cpp:195] start_ssl_client(): Performing the SSL/TLS handshake...
[V][ssl_client.cpp:216] start_ssl_client(): Verifying peer X.509 certificate...
[V][ssl_client.cpp:225] start_ssl_client(): Certificate verified.
[V][ssl_client.cpp:240] start_ssl_client(): Free internal heap after TLS 178440
[D][HTTPClient.cpp:1025] connect():  connected to accounts.spotify.com:443
[V][ssl_client.cpp:279] send_ssl_data(): Writing HTTP request...
[V][ssl_client.cpp:279] send_ssl_data(): Writing HTTP request...
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'HTTP/1.1 200 OK'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'date: Wed, 17 Mar 2021 14:35:39 GMT'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'content-type: application/json'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Length: 252'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'set-cookie: __Host-device_id=AQCVrb-BrJemyLT6st2JDW_VX-1P3HZ50m_6iqh0oLrXaYc3IhzOs-q0KLmIRweC9PalUgjLvFD6xb682TuqHqEvlpErClj5WuM;Version=1;Path=/;Max-Age=2147483647;Secure;HttpOnly;SameSite=Lax'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'sp-trace-id: dea20fa8a1f4c0e5'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'strict-transport-security: max-age=31536000'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'x-content-type-options: nosniff'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'vary: Accept-Encoding'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'server: envoy'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Via: HTTP/2 edgeproxy, 1.1 google'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Alt-Svc: clear'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: ''
[D][HTTPClient.cpp:1158] handleHeaderResponse(): code: 200
[D][HTTPClient.cpp:1161] handleHeaderResponse(): size: 252
[D][HTTPClient.cpp:368] disconnect(): tcp keep open for reuse

[V][ssl_client.cpp:248] stop_ssl_socket(): Cleaning SSL connection.
[V][ssl_client.cpp:248] stop_ssl_socket(): Cleaning SSL connection.
lbernstone commented 3 years ago

Can you remove the fingerprint entirely and see if it works?

ElToberino commented 3 years ago

[@lbernstone : insecure call works, but I think that is not the problem]

Meanwhile I changed my code according to the library example "BasicHttpsClient". That means, instead of fingerprint, I defined the root certificate and changed the code above:

const char* clientID = "XXXXXX";
const char* clientSecret ="XXXXXX";
const char* rootCACertificate = R"===(
-----BEGIN CERTIFICATE-----
MIIGEDCCBPigAwIBAgIQBS8G4gPhQtIePtEv2M7pnzANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E
aWdpQ2VydCBTSEEyIFNlY3VyZSBTZXJ2ZXIgQ0EwHhcNMjAwNjIzMDAwMDAwWhcN
MjEwOTAyMTIwMDAwWjBOMQswCQYDVQQGEwJTRTESMBAGA1UEBxMJU3RvY2tob2xt
MRMwEQYDVQQKEwpTcG90aWZ5IEFCMRYwFAYDVQQDDA0qLnNwb3RpZnkuY29tMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA6HWzVXakqZHjqPgKpyBCffVx
j9Vnki7IiTrKDn4AFU2TCkNEj7BjhUg2tNwytK726zpZ3PcRJ7EEyVEkhKCTDieC
hHawaUL0B3Xh7chgphPNEq39kw/neWAZ/gPWl+HaDB5CBrK95/z4vkIVIko1a+tl
LBqWFcHLEhjkdq5tWnsbJgQjCxwSCQxC5U9jg8i5he4HCPNMj0LW+05pqcfdin7E
bmkAlBxST6nHHgRSgqvH61StUx4/gEBsaeI9yET+xnj7CuL4V5LEYVXlWw94ZLtN
XAQjBrihrBkA4uiTkwMxEfFKHNbaebILpqJ+JiGH5ovYwpI72U7ghREEwd5xKwID
AQABo4IC6TCCAuUwHwYDVR0jBBgwFoAUD4BhHIIxYdUvKOeNRji0LOHG2eIwHQYD
VR0OBBYEFMHE9fNvSFaGJ8CcWqfO6HRQ9/4dMCUGA1UdEQQeMByCDSouc3BvdGlm
eS5jb22CC3Nwb3RpZnkuY29tMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggr
BgEFBQcDAQYIKwYBBQUHAwIwawYDVR0fBGQwYjAvoC2gK4YpaHR0cDovL2NybDMu
ZGlnaWNlcnQuY29tL3NzY2Etc2hhMi1nNi5jcmwwL6AtoCuGKWh0dHA6Ly9jcmw0
LmRpZ2ljZXJ0LmNvbS9zc2NhLXNoYTItZzYuY3JsMEwGA1UdIARFMEMwNwYJYIZI
AYb9bAEBMCowKAYIKwYBBQUHAgEWHGh0dHBzOi8vd3d3LmRpZ2ljZXJ0LmNvbS9D
UFMwCAYGZ4EMAQICMHwGCCsGAQUFBwEBBHAwbjAkBggrBgEFBQcwAYYYaHR0cDov
L29jc3AuZGlnaWNlcnQuY29tMEYGCCsGAQUFBzAChjpodHRwOi8vY2FjZXJ0cy5k
aWdpY2VydC5jb20vRGlnaUNlcnRTSEEyU2VjdXJlU2VydmVyQ0EuY3J0MAwGA1Ud
EwEB/wQCMAAwggEEBgorBgEEAdZ5AgQCBIH1BIHyAPAAdgD2XJQv0XcwIhRUGAgw
lFaO400TGTO/3wwvIAvMTvFk4wAAAXLhDERlAAAEAwBHMEUCIASIo9eFKEVLL4cD
xlBhAGSo82I0TRd66jvwFhx6Se79AiEAhL0I6dqaIZTsorV0XwSObs8gbh48ba5h
XqQesmvVjH0AdgBc3EOS/uarRUSxXprUVuYQN/vV+kfcoXOUsl7m9scOygAAAXLh
DESTAAAEAwBHMEUCIQDnnsP6nLnwtiUHORgRCBqPqFTHsPaTA9FxbmRw0gxGfAIg
HLukydDmJy6a3f4ZXa5eJ63kUB1iQc/oai3aXSHbP6MwDQYJKoZIhvcNAQELBQAD
ggEBAHfGtU9XU+pUAMgZoc0NenTV2fQfZl5MM5EPG/Jf0Fp4YnIIz92vReVnUvco
MxO73E/mP15nxaXQtknQTpCn4ZBSD4OF5oMKM1M98iSGC+ZvKlk0UektOv9zrRTn
A2JLJH695cul3nyaoqhuAWtzfFft8y6fm1Bclt1q6OvEGkNtWJ07aQFBXeaIzr4W
kZR8sdw+EkiGk1bN/C1CMU0xBnkQE19OycfE2Ax5Qag81KwLlcdkgMtXkMie2ZV1
ytyF8FFgyCakrFkDeDOnc5YAOpjlkD8J0uNTZIPv1rF/nA67EgAW9NFxulCqqrXb
AFkCjRN8xiv10mrVbcQ86LwZdGA=
-----END CERTIFICATE-----
)===";
String refreshtoken ="XXXXXX";                                                                           
String auth;
String accesstoken;
uint16_t expiretime;

void callSpotify(){
    auth = "Basic " + base64::encode(String(clientID) + ":" + String(clientSecret));           
    Serial.println("Calling Spotify for accesstoken");
    WiFiClientSecure *client = new WiFiClientSecure;
    if(client) {
    client -> setCACert(rootCACertificate);

    HTTPClient http;

    if(http.begin(*client , "https://accounts.spotify.com/api/token"){
      http.addHeader("Content-Type", "application/x-www-form-urlencoded"); 
      http.addHeader("Authorization", auth);

      String url = "grant_type=refresh_token&refresh_token=" + refreshtoken;
      int httpCode =  http.POST(url);
       Serial.println(httpCode);

      if(httpCode == 200) {
        DynamicJsonDocument doc(400);
        DeserializationError error = deserializeJson(doc, http.getStream());
        if (error) {
            Serial.println("deserializeJson() failed: ");
            Serial.println(error.c_str());
            return;
        }
        const char* newtoken = doc["access_token"];
        expiretime = doc["expires_in"];

        accesstoken = "Bearer " + String(newtoken);     
       } else {
        Serial.printf("http error: %i\n", httpCode);
        }
    } else {
        Serial.println("Unable to connect to Spotify");
    }
    http.end();
    delete client;
 }
}

But this results in the following error:

[V][ssl_client.cpp:115] start_ssl_client(): Loading CA cert
[V][ssl_client.cpp:180] start_ssl_client(): Setting hostname for TLS session...
[V][ssl_client.cpp:195] start_ssl_client(): Performing the SSL/TLS handshake...
[E][ssl_client.cpp:33] _handle_error(): [start_ssl_client():199]: (-9984) X509 - Certificate verification failed, e.g. CRL, CA or signature check failed
[E][WiFiClientSecure.cpp:132] connect(): start_ssl_client: -9984
[V][ssl_client.cpp:248] stop_ssl_socket(): Cleaning SSL connection.
[D][HTTPClient.cpp:1018] connect(): failed connect to accounts.spotify.com:443
[W][HTTPClient.cpp:1318] returnError(): error(-1): connection refused
[D][HTTPClient.cpp:383] disconnect(): tcp is closed

The certificate is copied from the browser and should be ok.

lbernstone commented 3 years ago

Look at your log from 1.0.4. It was unsecure there as well. I have no idea where you got that root cert from. Here's the correct digicert CA:

const char* digicert_CA_pem = R"literal(-----BEGIN CERTIFICATE-----
MIIElDCCA3ygAwIBAgIQAf2j627KdciIQ4tyS8+8kTANBgkqhkiG9w0BAQsFADBh
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD
QTAeFw0xMzAzMDgxMjAwMDBaFw0yMzAzMDgxMjAwMDBaME0xCzAJBgNVBAYTAlVT
MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxJzAlBgNVBAMTHkRpZ2lDZXJ0IFNIQTIg
U2VjdXJlIFNlcnZlciBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
ANyuWJBNwcQwFZA1W248ghX1LFy949v/cUP6ZCWA1O4Yok3wZtAKc24RmDYXZK83
nf36QYSvx6+M/hpzTc8zl5CilodTgyu5pnVILR1WN3vaMTIa16yrBvSqXUu3R0bd
KpPDkC55gIDvEwRqFDu1m5K+wgdlTvza/P96rtxcflUxDOg5B6TXvi/TC2rSsd9f
/ld0Uzs1gN2ujkSYs58O09rg1/RrKatEp0tYhG2SS4HD2nOLEpdIkARFdRrdNzGX
kujNVA075ME/OV4uuPNcfhCOhkEAjUVmR7ChZc6gqikJTvOX6+guqw9ypzAO+sf0
/RR3w6RbKFfCs/mC/bdFWJsCAwEAAaOCAVowggFWMBIGA1UdEwEB/wQIMAYBAf8C
AQAwDgYDVR0PAQH/BAQDAgGGMDQGCCsGAQUFBwEBBCgwJjAkBggrBgEFBQcwAYYY
aHR0cDovL29jc3AuZGlnaWNlcnQuY29tMHsGA1UdHwR0MHIwN6A1oDOGMWh0dHA6
Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RDQS5jcmwwN6A1
oDOGMWh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RD
QS5jcmwwPQYDVR0gBDYwNDAyBgRVHSAAMCowKAYIKwYBBQUHAgEWHGh0dHBzOi8v
d3d3LmRpZ2ljZXJ0LmNvbS9DUFMwHQYDVR0OBBYEFA+AYRyCMWHVLyjnjUY4tCzh
xtniMB8GA1UdIwQYMBaAFAPeUDVW0Uy7ZvCj4hsbw5eyPdFVMA0GCSqGSIb3DQEB
CwUAA4IBAQAjPt9L0jFCpbZ+QlwaRMxp0Wi0XUvgBCFsS+JtzLHgl4+mUwnNqipl
5TlPHoOlblyYoiQm5vuh7ZPHLgLGTUq/sELfeNqzqPlt/yGFUzZgTHbO7Djc1lGA
8MXW5dRNJ2Srm8c+cftIl7gzbckTB+6WohsYFfZcTEDts8Ls/3HB40f/1LkAtDdC
2iDJ6m6K7hQGrn2iWZiIqBtvLfTyyRRfJs8sjX7tN8Cp1Tm5gr8ZDOo0rwAhaPit
c+LJMto4JQtV05od8GiG7S5BNO98pVAdvzr508EIDObtHopYJeS4d60tbvVS3bR0
j6tJLp07kzQoH3jOlOrHvdPJbRzeXDLz
-----END CERTIFICATE-----)literal";
ElToberino commented 3 years ago

@lbernstone You're absolutely right - I took the wrong certifcate... So far, following the library example, I got the secure connection working. For investigation purposes, I also tried the original code with the certificate instead of the fingerprint: if(http.begin("https://accounts.spotify.com/api/token", digicert_CA_pem)) This works, but it's also a insecure connection. So I suppose it's necessary to setup the WiFiClientSecure manually to setup a secure connection.

@me-no-dev and @lbernstone Thanks a lot for your help and your patience. You really helped me and I think I've learned a lot...

me-no-dev commented 3 years ago

http.begin("https://accounts.spotify.com/api/token", digicert_CA_pem) will connect using the certificate, while http.begin("https://accounts.spotify.com/api/token") will be insecure

ElToberino commented 3 years ago

@me-no-dev @lbernstone I think I got behind it and want to thank you for your help an patience with me. My sketch is even better than before (the fingerprint solution wasn't a proper solution as I know now) and I even managed to read the certificate from a file on Spiffs, so that I can easily change the certificate when it has expired. Best regards and thanks for your work!

me-no-dev commented 3 years ago

Everything now working as expected @ElToberino ? :)

ElToberino commented 3 years ago

@me-no-dev Yes, everything is fine now. Thank you!

AlfonsoAlejandre commented 3 years ago

Really helpful issue, thank you very much