helium / blockchain-etl

Blockchain follower that follows and stores the Helium blockchain
Apache License 2.0
64 stars 37 forks source link

Different responses from https://api.helium.io/v1 #162

Closed APshenkin closed 3 years ago

APshenkin commented 3 years ago

We faced with issue that api.helium.io return different responses when we call it from different places. When we call

curl --location --request GET 'https://api.helium.io/v1/accounts/132eiufNkm7DedxkVYnzEwFgj96McokvV7RYgzKVgxhdrbvNpKA/activity' \
--header 'Content-Type: application/json'

locally, the response is

{
    "data": [],
    "cursor": "eyJtaW5fYmxvY2siOjgwNzU2MiwiYmxvY2siOjgwNzgwMCwiYW5jaG9yX2Jsb2NrIjo4MDc4MDB9"
}

but when we do this call from our server the response looks so

{"data":[]}

So cursor is missing for some reason. How it could be and how to solve this?

APshenkin commented 3 years ago

Seems like request from server goes through some cache

* Connected to api.helium.io (199.232.26.133) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 594 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
*    server certificate verification OK
*    server certificate status verification SKIPPED
*    common name: api.helium.io (matched)
*    server certificate expiration date OK
*    server certificate activation date OK
*    certificate public key: RSA
*    certificate version: helium/blockchain-core#3
*    subject: CN=api.helium.io
*    start date: Sun, 28 Mar 2021 23:45:54 GMT
*    expire date: Sat, 26 Jun 2021 23:45:54 GMT
*    issuer: C=US,O=Let's Encrypt,CN=R3
*    compression: NULL
* ALPN, server accepted to use http/1.1
> GET /v1/accounts/132eiufNkm7DedxkVYnzEwFgj96McokvV7RYgzKVgxhdrbvNpKA/activity HTTP/1.1
> Host: api.helium.io
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Connection: keep-alive
< Content-Length: 11
< Content-Type: application/json; charset=utf-8
< Access-Control-Allow-Origin: *
< Cache-Control: max-age=86400
< Accept-Ranges: bytes
< Date: Sat, 17 Apr 2021 17:47:31 GMT
< Via: 1.1 varnish
< Age: 75306
< X-Served-By: cache-dub4327-DUB
< X-Cache: HIT
< X-Cache-Hits: 2
< X-Timer: S1618681652.604454,VS0,VE0
< Vary: Accept-Encoding
< Strict-Transport-Security: max-age=300
<
* Connection #0 to host api.helium.io left intact

But locally it doesn't

*   Trying 199.232.82.133...
* TCP_NODELAY set
* Connected to api.helium.io (199.232.82.133) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=api.helium.io
*  start date: Mar 28 23:45:54 2021 GMT
*  expire date: Jun 26 23:45:54 2021 GMT
*  subjectAltName: host "api.helium.io" matched cert's "api.helium.io"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7f9d68810a00)
> GET /v1/accounts/132eiufNkm7DedxkVYnzEwFgj96McokvV7RYgzKVgxhdrbvNpKA/activity HTTP/2
> Host: api.helium.io
> User-Agent: curl/7.64.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< content-type: application/json; charset=utf-8
< access-control-allow-origin: *
< cache-control: max-age=60
< accept-ranges: bytes
< date: Sat, 17 Apr 2021 17:48:58 GMT
< via: 1.1 varnish
< age: 0
< x-served-by: cache-mrs10530-MRS
< x-cache: MISS
< x-cache-hits: 0
< x-timer: S1618681737.356513,VS0,VE735
< vary: Accept-Encoding
< strict-transport-security: max-age=300
< content-length: 99
<
* Connection #0 to host api.helium.io left intact
{"data":[],"cursor":"eyJtaW5fYmxvY2siOjgwNzU2MiwiYmxvY2siOjgwNzgwMCwiYW5jaG9yX2Jsb2NrIjo4MDc4MDB9"}* Closing connection 0
madninja commented 3 years ago

@APshenkin the api responses are cached by Fastly. In the response you can see that the x-cache was a MISS meaning that Fastly actually asked the backend api for the data.

I don't know why you were getting completely empty response there. Was that data in the same minute as the account being created?

And does the problem still happen right now?

APshenkin commented 3 years ago

For now it's showing correct information about address.

Account was created long time ago and we got this response for one day. Cache was cleared after some time and now we get correct information