Compress data over internet to improve weak cell network connections

[brianmathews]

Problem: when in remote areas where cell phone reception is week, it can be difficult to download the "recent spots" list from the server API. Reducing the amount of data required through compression can help.

Proposal:

• Check to see if the SOTA and POTA API servers are already allowing compressed (gzip) data downloads.
• Check to see if the client Javascript code is requesting gzip encoding if it is available. Validate it works.
• If the SOTA and/or POTA servers don't support gzip encoding, even when requested by the client, then create a new API endpoint on the SOTAMAT server as a relay / proxy service. This means the client can negotiate a gzip compressed stream from SOTAMAT-server and the SOTAMAT-server can grab the data from the SOTA / POTA API uncompressed (over a high speed reliable fiber network).

[jeffkowalski]

It doesn't appear that api2.sota.uk.org will respect compression requests.

For example, consider:

curl --compressed -i https://api2.sota.org.uk/api/spots/24/all

The --compressed flag will add Accept-Encoding: gzip header to the request.

The server responds with these headers:

HTTP/1.1 200 OK
date: Fri, 28 Jun 2024 04:21:45 GMT
content-type: application/json; charset=utf-8
server: Kestrel
transfer-encoding: chunked

followed, of course by the content.

Regrettably, it is missing the crucial Content-Encoding: gzip header. And so, the content arrives uncompressed. The server ignored our request for compression.

---

Now, different case for POTA.

curl --compressed -i https://api.pota.app/spot/activator

responds with headers that include

content-encoding: br

That's great news - Brotli compression, even better than gzip.

---

So, I'll change the fetches to request compression. We won't get it yet for SOTA, but perhaps someday. We'll get it immediately for POTA. It won't degrade performance, and we'll be set up for compression, and even improved compression when it arrives. It'll all be handled transparently by inclusion of the Accept-encoding header, as in

const response = await fetch(`${baseurl}/${summitCode}`, {
  headers: {
    'Accept-Encoding': 'gzip'
  }
});

[brianmathews]

Sounds great.