API v2

[Grayda]

This issue will be for the discussion of v2 of the API. v2 is still in the planning / early development stages, but feedback is sought

Reasoning:

v1 of the API is good, but is slow, inflexible and lacks many pieces of data other sites have. v2 aims to speed things up with more modular code and making use of PHP and browser tactics.

Ideas / planned features:

These ideas are NOT final, and are in the planning stages. Feedback welcome.

• Requests will now be made via POST, instead of GET, and requests will need to be a valid JSON object
  • This allows users to request specific pieces of data, and to pass options through, so you could, for example, request current Kp AND archived Kp data (previously, this would require 2 separate requests)
• Make use of PHP and browser features to speed up the site
  • Examples would include:
    • Frequent automatic caching of data which would potentially shave multiple seconds from requests
    • Closing the connection after outputting data, but continuing processing (e.g. writing logs, processing cached items
    • Using alternative data sources (e.g. using OpenWeatherMap would eliminate the need to use Nominatim for geolocation, as OWM includes this info in the results)
• More information available to users
  • For example, custom-made charts showing Bz history over the last hour
• A bigger push for WeatherCam and other homebrew aurora hunting instruments
• Smarter aurora alerts based on more data
  • Options could include weather and time-based alerts (e.g. only alert if cloud < 50% or time in local timezone is between 8pm and 8am)
  • Other suggestions welcomed for this.

These changes will eventually culminate in v2 of the Auroras.live app, rewritten from the ground up. There's no timeline for either of these yet, but I'd rather it be sooner than later, as there's a need for better aurora hunting data that the current API doesn't easily make available.

[Grayda]

Over a month on, and I think it's time for a status update.

v2 of the API is ongoing. Right now the following data is implemented:

• SWPC alerts (with timezone substitution, meaning any dates shown within an alert will be in your timezone)
• Archives (with the max amount of data in one hit bumped up to 7 days)
• Bz, Bt, Density, Speed and Kp (current, 3 day, 7 day and 27 day)
• Weather (Provided by OWM), Sunrise, sunset, moon rise, moonset and phase (now calculated locally so requests for sunrise will be instantaneous)
• Custom charts, including current Kp, 3 day Kp forecast, 7 & 27 day Kp forecasts, and current conditions (as percentage so they're easier to read for new hunters. For example Kp 0 = 0%, while Kp 9 = 100%)
  • More charts coming soon, including historical charts (e.g. last 2 days of Kp, Bz, Speed, Density)
• More images available, with their own dedicated URLs. This means you can call api.auroras.live/v2/images/yellowknife.jpg to get an image. You can also swap formats, so yellowknife.png and yellowknife.jpg will return the image in their respective formats.

This is all still in development, and will change, expand and shrink as it goes on. No ETA, but I hope to have some documentation written up in the near future.

[Photogad]

By any chance can you provide weather via the Dark Sky API? I've found it to be more accurate than OWM, faster, and it's probably cheaper for you too. The first 1,000 calls per day on dark sky are free and then remaining calls are only 0.001 per call

[Grayda]

I've coded support into v2 as an optional thing, and I'm currently emailing Dark Sky to find out some finer points of their terms and services before making that an enabled feature.

I'm a little hesitant when it comes to paid services, because I had a DDoS at the end of last year that coloured my world map a nice shade of green, and I shudder to think what that would have been like at $0.0001 per call. That's why api.met.no was the data source initially, and then OWM (because of it's 60 calls a minute thing).

So once I hear back from Dark Sky, I'll let you know. Best case scenario, there'll be limits on how often the DS source can be accessed in the API, worst case, you'll need to sign up for your own API key and use that.

[Photogad]

It was just a suggestion, trying to help you out. :) you can manually limit daily calls with day sky too in event if ddos

On Aug 27, 2017 4:33 AM, "David Gray" notifications@github.com wrote:

I've coded support into v2 as an optional thing, and I'm currently emailing Dark Sky to find out some finer points of their terms and services before making that an enabled feature.

I'm a little hesitant when it comes to paid services, because I had a DDoS at the end of last year that coloured my world map a nice shade of green, and I shudder to think what that would have been like at $0.0001 per call. That's why api.met.no was the data source initially, and then OWM (because of it's 60 calls a minute thing).

So once I hear back from Dark Sky, I'll let you know. Best case scenario, there'll be limits on how often the DS source can be accessed in the API, worst case, you'll need to sign up for your own API key and use that.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Auroras-live/issues/issues/20#issuecomment-325184788, or mute the thread https://github.com/notifications/unsubscribe-auth/ARpg5GrkbZBHAiUjo5N7LmGwJKbm2G2hks5scSnQgaJpZM4MxBhG .

[Grayda]

Oh, I completely missed that setting in the dashboard. In which case, it'll be enabled in v2. It'll still be opt-in, as OWM is "close enough" for most people, but you'd just need to add in "provider": "darksky" to the request and it'll use that.

I'll also leave in the "key": "abcd" bit so if a user wants to use their own key, they can.

[Grayda]

Time for another update:

I've been rewriting v2 from scratch. Everything is now separated into modules (code-wise) to make updates and management easier. I'm aiming for a July release, but that's dependent on a whole bunch of factors, but programmatic and personal.

So far the following has been completed:

• Bt and Bz modules
• Speed and density
• Product alerts
• Kp
• Weather (with OpenWeatherMap, Dark Skies, and met.no as providers)

In progress:

• Archives

Upcoming:

• Nowcast "front-end" (a.k.a aurora sighting probabilities)
• Images (e.g. skycams, charts etc.)
• Custom images (e.g. Auroras.live branded charts, weather summary images etc.)

Future:

• Updating of API documentation
• Re-working of push notifications to support smarter notifications and more push options (e.g. browser-based notifications, webhooks and such)
• Migration and rewriting of CRON jobs (i.e. scripts that collect and archive data, including Nowcast data)
• Rewritten mobile app to use the new API source

I'll provide more information as the API progresses.

[Photogad]

Thank you for the update and all of your hard work! You still need to take my recommendation of charging a monthly subscription for your API.

Question: When you release v2, will v1 still be live? If you can leave v1 live for a few months after v2 is released, that would be great so it's a smooth transition without any interruption.

[Photogad]

also this probably isn't the right place to ask so feel free to delete this but -

have you noticed any issues with getting moonrise and moonset times for certain locations? For some reason I am sometimes getting no moonrise or moonset times (or sometimes just one) from the API. Usually it's the moonset time. It seems to be happening the most in Canada.

What weather service do you use for moonrise and moonset times?

[Grayda]

@photog0411: v1 will remain active for quite a while. There'll be adequate notice before it's deactivated, and will probably only be deactivated if the amount of resources it consumes gets to be too much (though most of the resources in use are from background scripts, and those are being rewritten and migrated to v2 in a way that won't affect users).

Moonrise and moonset come from met.no, but v2 will do local calculations via a fork of the SunCalc library (the moon and sun times can now be obtained without having to get the weather).

As for your issue with no moonrise or moonset times appearing, you might get partial data if the moon doesn't rise and / or set at all on that particular day. If that happens again, check with another site to ensure what met.no is reporting, is correct.

[Grayda]

I'm closing this issue in anticipation of a (beta) v2 release coming soon (and as a clean up thing). The API has been completely rewritten since the last update, and all the useful things such as Darksky / OWM integration have been added