HTTPS / 443

[victor7376]

I'm trying to make another API client for the clock, but the API client requires the 443 connection.

I've tried to add in the WiFiClientSecure and setting a fingerprint, but the connection always fails.

I can make the API call in a basic sketch with the WiFiClientSecure, but as soon as I try to port it over to the clock it just fails to work.

I've even tried changing the openweathermap to use the 443 and WiFiClientSecure but that fails too.

Is there something I'm missing??

[k7bbr]

The esp8266 lacks the memory to handle https. Running a simple sketch with nothing else probably works, but as soon as you try to run it with the marquee code, there aren't enough resources to make it work on that chip.

If you were to take the significant time to add esp32 capability to the marquee code, that chip might be able to do ssl. I'm not sure.

[victor7376]

I'm looking at some code for the ESP32 which makes an HTTPS request to spotify.

I've also looked at some code for another clock that has esp32 & ESP8266 both coded in.

[victor7376]

Hopefully tomorrow I'll take a closer look at the Marquee to see if I can make it for both esp8266 & ESP32 as I've just looked at the 2nd clock that has them both coded in and tried with a simple sketch and got that working for both esp32 & esp8266.

[victor7376]

Well I had a go and it didn't go well lol, but I've found 2 other 'simple' clocks that I might be able to work into the https. Found them on youtube, they didnt have the up to date ArduinoJson or ESP8266 coding, but between 2 youtube users they have updated the code to the latest.

The two clocks are from the same youtuber, one is a small simple 1x4 Max7219 display with weather scrolling, the 2nd is a double height (2x4 Max7219 displays) which has the options of making the clock full size taking up the double height.

[victor7376]

Finally got it to work, after a lot of research and coding I've got the Marquee working perfectly with an API that requires HTTPS and a fingerprint code.

I have removed a lot from the code from features that I dont use - Pihole, News & Bitcoin so now only uses 50 - 55% of the ESP8266

[brianmiller1956]

Well done, does this mean you will post this as a fork? I am still testing the day suffix, so far so good.

Sent from my iPad Pro

On 10 Feb 2021, at 16:27, victor7376 notifications@github.com wrote:

 Finally got it to work, after a lot of research and coding I've got the Marquee working perfectly with an API that requires HTTPS and a fingerprint code.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[victor7376]

More than likely, but just a warning the site that I'm using is an adult website lol

[brianmiller1956]

Well I am 64!

Sent from my iPad Pro

On 10 Feb 2021, at 17:17, victor7376 notifications@github.com wrote:

 More than likely, but just a warning the site that I'm using is an adult website lol

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[victor7376]

Well I am 64! … Sent from my iPad Pro On 10 Feb 2021, at 17:17, victor7376 @.***> wrote:  More than likely, but just a warning the site that I'm using is an adult website lol — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

'beatles music playing'

Hopefully I'll be able to upload the code tonight.

[victor7376]

After a lot of testing yesterday I've had to remove OctoPrint from the clock too, now it only has timedb, weather and the other api on it. Seemed the clock couldnt handle both of them and kept crashing.

Today has been more stable so I've uploaded the new code I've done to show the changes.

[david-nguyen]

How on earth did you get this to work? Even adding setting the fingerprint the connection fails with the initial commit.

[victor7376]

How on earth did you get this to work? Even adding setting the fingerprint the connection fails with the initial commit.

Which site are you trying to get to work? You’ve got to remember to include the wificlientsecure library for it to work.

[david-nguyen]

I'm assuming it's included in the core package I do not see any compile errors unless you are saying it's external. It's just my own website (https://importbible.com/)

[victor7376]

if you've made a new section which connects to your site you need to include it in the .h section

#include WiFiClientSecure.h

if you can post what info your trying to get I could have a look.

[david-nguyen]

Sure thing thank you for the help.

CPP:

/** The MIT License (MIT)

Copyright (c) 2018 David Payne

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/

#include "IBClient.h"

#define arr_len( x )  ( sizeof( x ) / sizeof( *x ) )

IBClient::IBClient() {
}

void IBClient::updateData() {
  WiFiClientSecure modelClient;
  modelClient.setInsecure();
  modelClient.setFingerprint("9B 42 85 E8 39 C4 DF 88 49 BE D0 BB 20 49 D4 F8 E1 F2 6F 63");
  // modelClient.setInsecure();

  if (!modelClient.connect("importbible.com", 443)) {
    statData.root = "connection failed";
    return;
  }

  modelClient.println("GET /xxx HTTP/1.0");
  modelClient.println(F("Host: importbible.com"));
  modelClient.println(F("User-Agent: Arduino"));
  modelClient.println(F("Connection: close"));

  char status[32] = {0};
  modelClient.readBytesUntil('\r', status, sizeof(status));

  if (strcmp(status, "HTTP/1.1 200 OK") != 0) {
    statData.root = status;
    return;
  }

  modelClient.find("\r\n\r\n");
  const size_t bufferSize = JSON_OBJECT_SIZE(12) + 238;
  DynamicJsonBuffer jsonBuffer(bufferSize);
  JsonObject& root = jsonBuffer.parseObject(modelClient);
  if (!root.success()) {
    statData.root = "Bitcoin Data Parsing failed!";
    return;
  }

  statData.dailyOrders = (const char*)root[String("dailyOrders")];
  statData.dailyTotal = (const char*)root[String("dailyTotal")];
  statData.ytdOrders = (const char*)root[String("ytdOrders")];
  statData.ytdTotal = (const char*)root[String("ytdTotal")];

  Serial.println();
}

String IBClient::getDailyOrders() {
  return statData.dailyOrders;
}

String IBClient::getDailyTotal() {
  return statData.dailyTotal;
}

String IBClient::getYTDOrders() {
  return statData.ytdOrders;
}

String IBClient::getYTDTotal() {
  return statData.ytdTotal;
}

String IBClient::getRoot() {
  return statData.root;
}

h:

/** The MIT License (MIT)

Copyright (c) 2018 David Payne

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/

#pragma once
#include <ESP8266WiFi.h>
#include "libs/ArduinoJson/ArduinoJson.h"
#include <WiFiClientSecure.h>

class IBClient {

  private:

    typedef struct {
      String root;
      String dailyOrders;
      String dailyTotal;
      String ytdOrders;
      String ytdTotal;
    } stats;

    stats statData;

    const char* servername = "importbible.com";  // remote server we will connect to

  public:
    IBClient();
    void updateData();

    String getRoot();
    String getDailyOrders();
    String getDailyTotal();
    String getYTDOrders();
    String getYTDTotal();
};

[victor7376]

straight away I can see one issue, the url that you're trying to connect to is (https://importbible.com/xxx) visiting that url doesn't show any json data.

[victor7376]

heres an example of the json data that you need for this to work; https://api.coindesk.com/v1/bpi/currentprice.json

visiting the page above is what you need to connect to so you can gather the json data. if your site has an api that spits out json data you need to connect to that.

[david-nguyen]

right I changed the url since it's being posted, the connections fails on this line

 if (!modelClient.connect("importbible.com", 443)) {

Not sure if it's just my wifi connection is failing on the device itself but it shows 100% signal strength.

[victor7376]

I will admit I removed a lot of things from the marquee for it to work with https.

does your site give out any json data? have you also used the https://arduinojson.org/v5/assistant/ ?

[victor7376]

lol I believe I just found your issue. remove the following:

modelClient.setInsecure();

you're trying to connect to a secure site while having the setting as insecure.

[david-nguyen]

I tried removing that too with no luck, I'll keep cracking away at it thanks!

[victor7376]

I’ll take a look at the coding tomorrow if you haven’t already fixed it.

[victor7376]

I've just been looking at the code and just doing the basic code has given me the following:

Connecting to importbible... Connected to importbible Sending request to importbible... Response Header: HTTP/1.1 301 Moved Permanently Unexpected HTTP statusHTTP/1.1 301 Moved Permanently

So my code is working perfectly with your site, the reason for the error is that the page that you've posted doesnt give out any JSON data. The one difference between your code and my code is that I changed the Fingerprint to lowercase which makes me believe that the fingerprint is case sensitive:

#define IMPORTBIBLE_FINGERPRINT "9b 42 85 e8 39 c4 df 88 49 be d0 bb 20 49 d4 f8 e1 f2 6f 63"

Try this fingerprint and you should be able to connect to your site.

Hope this helps you.

[david-nguyen]

finally got it working, I was missing a println after modelClient.println(F("Connection: close")); thanks for the help!