search

opennem / opennem-fe

An Open Platform for National Electricity Market Data
https://opennem.org.au
MIT License
65 stars 12 forks source link

add time period to the URL #4

Closed charliedotau closed 2 years ago

charliedotau commented 5 years ago

This one is feature request.

Given the new version of the site offers more options for the chart time periods, I think it makes sense to have these represented in the URL. So for instance, if you were to change the region to Victoria and select a time period of one year, the url would be something like /#/regions/vic/year or better still have the actual dates in the URL.

The benefits of adding the timeframe to the URL may include:

  1. better for sharing. If I tweet the aforementioned link for instance, the people who click on my tweet see on the website the same as what I was seeing). This would be particularly true if you put actual dates in the URL.

  2. possibly better browser caching. Given browser caching works at the page level (and the browser defines a page via its URL), more granular URLs may result in more accurate browser caching

  3. slightly better user experience - because the state of the app (which region and region time period is selected) is reflected not only in the app's UI but also via its URL

  4. better search engine ranking. Google tends to like site's that have a sitemap (inferred from the URL structure) that is more than 1 level deep.

  5. More granular reporting in Google Analytics. Because out of the box reporting in GA works via URL page views, having the time period in the URL would enable you to, for instance, determine which time periods generate the most page views.

There are some technical considerations here. For instance, you may wish to update the Twitter Cards metadata, so that as the user chooses a different region or time period, the Twitter Card metadata updates accordingly (so that if someone tweets that URL, the tweet's auto generated metadata is consistent with the selected region/time-period).

ta Charlie

aleith commented 5 years ago

What is Twitter Card('s) metadata?

aleith commented 5 years ago

In some future instances OpenNEM might also display modelled data not just historical/regional data, any URL schema would need to considered extensibility beyond the constructs of historical NEM full and regional time periods. For instance a WA SWIS model that is using historical weather data but a model for 50% RE (or something).

simonhac commented 5 years ago

i'm a big fan of this request — i'd like to be able to share a view by sending someone the URL, so, to the greatest extent possible, every piece of configurability for the page should be able to be put in the URL.

i'd also like to get the # out of the URL.

aleith commented 5 years ago

This would effectively be the bookmark FR sorted too. (maybe it would need some additional UI sugar)

charliedotau commented 5 years ago

What is Twitter Card('s) metadata?

Twitter Card metadata is the code you add to your HTML pages, so that your website is optimised for sharing on Twitter. In the case of OpenNem the Twitter card code ensures that when the site's URL is tweeted, the tweet comes with some bells and whistles such as a thumbnail image, a title and a short description. See the attached screenshot, where I've highlighted in green the bits of the tweet automatically generated from the Twitter Card metadata.

Twitter provides a tool for previewing your site's Twitter Card metadata here (you need to be logged into twitter).

As I understand it, the Twitter Card metadata for opennem is static: its the same irrespective of the URL that is tweeted. Ideally the Twitter Card metadata would update with the URL. screen shot 2019-01-13 at 10 26 01 am

charliedotau commented 5 years ago

I've given some thought to the URL design. To my mind the only approach that really makes sense is to use query parameters, like so:

https://opennem.org.au/regions/vic?startDate=<<dateFormat>>&endDate=<<dateFormat>>

The proposed date format would be iso8061 so the full example would be

https://opennem.org.au/regions/vic?startDate=2018-01-31&endDate=2018-12-31

The benefits of this design are

  1. dates are human readable (as opposed to a date format that isn't such as Unix time)
  2. supports reporting on any time period, rather than just predetermined ranges. Finally you'll be able to know what the NEM was doing over the 2016 Olympics!
  3. various benefits of iso8061 dates as described here
  4. conventional - this is precisely what query parameters were designed for
charliedotau commented 5 years ago

It's probably worth outlining the scenarios that the URL design should support

the primary variables for a given region are

  1. the moment in time (e.g a single point on x-axis on chart)
  2. the reporting period (e.g 1 day, 3 day, 7 day...all)
  3. the chart interval (e.g 5 or 10 minutes in single day view)

Moment in time

The units of the x-axis can be one of the following

Clearly then startDate and endDate is insufficient. How best to capture all of this in a URL?

charliedotau commented 5 years ago

If we were to focus to the URLS around the Moment in time, we might design the URLs like so

The benefits of this are:

What to do with Financial Year and Season?

These are outliers, so not sure how best to handle these.

What about Reporting Period and Chart interval?

Potentially these could be captured as query strings, e.g

Note in examples above - because we know the Moment in Time is a timeless date, its implicit that the chart interval is one day

Other examples

aleith commented 5 years ago

Just to throw a curve ball, I'm talking with Steven about the prospect of using OpenNEM to display energy modelling results, not just historical data, so if we were to extend this FR to that context, we would need some way to identify the model in questions, which could I suppose just be a variation on the /regions/sa e.g.

aleith commented 5 years ago
  1. various benefits of iso8061 dates as described here

opennem already uses iso-8601 dates in the JSON data for the graph.

chienleng commented 5 years ago

i think i probably not add the week num param as it makes the parsing more complicated and it's not consistent with the rest. The period query should be enough there to cover the week num. /regions/sa/2018/W02/01.

@aleith: for the models, it can be included as a query string instead. i.e. ?model=xxx

charliedotau commented 5 years ago

Hi @chienleng. I see where you're coming from. There's something there that doesn't sit right with me though, and I think it probably comes back to the related conversation around server vs client side.

When I designed those URLs, I wanted to ensure we were delivering distinct, canonical pages. But what does that mean exactly? It's more than just a unique URL.

Canonical Page

To my mind, a canonical page, that is easily indexed by Google and readily shared on the Socials comprises as much of the following as possible:

  1. a unique URL (whether that's with or without a parameter query)
  2. a unique Page Title and Meta Description (Google hates nothing more than duplicate titles) 3, some distinct page content ( ideally a unique H1 tag and some unique copy.
  3. a distinct, representative thumbnail, to aide sharability with Twitter etc.
  4. all of the above accessible by web crawlers, including not just Google but also Twitter (see below).

Server side vs Client Side

If we strive for the above, this inevitably raises the server-side vs client-side question. Whilst Google is getting better at crawling client side, there are other considerations. Twitter for instance, cache's the metadata it gleans from the website that it uses to render the preview (what they call the Twitter Card):

quote: "Our web crawlers re-index the Card tag information on your page roughly every week."

This needs to be factored in, when considering the best implementation. (not to mention the challenge that you've already raised around generating those thumbnails in the first place, server side).

Back to URL design

This brings us back to the URL design. @chienleng: let's take your suggestion to @aleith about models. If using a query parameter as you've suggested, to distinguish historical data from model data can be achieved, whilst meeting the canonical page criteria I set out above, then great. However If using a query parameter necessitates a particular implementation that can't easily achieve those criteria, then I'd suggest we steer away from it, and look at a different way.

I'd say the same question applies regarding your suggestion for handling weeks.

In other words, its hard to appraise a specific URL design without also considering the implementation.

VueJS

@chienleng Is there a simple solution, whereby we essentially have one static index.html per distinct URL (route?)?

sorry for the essay.

chienleng commented 5 years ago

Re implementation, I think the important thing is to have a sensible URL design that works as a way to get to the time period when people share the link. I don’t think at this stage we are able to implement the server side component that can generate/save a user-defined (date, chart, user defined description etc) image output to be served for Twitter, and still unconvinced we need to do that yet.

The canonical pages for OpenNEM are the regions (/nem, /sa, etc), with different sections (upcoming) that connects them /nem/energy, /nem/generators.

I like @charliedotau‘s idea of having years (and possibly months) in the params as they can be useful in certain context.

But we will still need to use query strings rather than params as they allow users to select a custom range that’s not defined by the param structure. With these three, we can cover almost all the use cases:

I am looking into using Nuxt.js (vuejs framework + static generated pages) to create static pages per route. So this hopefully should help with some of the issues of SPA. It will not create a static file for dynamic routes though.

charliedotau commented 5 years ago

Hi @chienleng

With your approach @chienleng, does that mean the pages you've identified as canonical can have a unique page title and description? This is really essential for SEO.

chienleng commented 5 years ago

@charliedotau yep.

chienleng commented 2 years ago

not quite the right solution, but we now have range, interval and filter for the energy url. i.e.https://opennem.org.au/energy/nem/?range=all&interval=1M&filter=february

alex-harvey-z3q commented 2 years ago

@chienleng Your original proposal seems to be the right one. I don't think this feature request should be closed. It's a real limitation of this tool not be able to query historical data.

aleith commented 1 year ago

yes, I'd really like to see historical data queries with high resolution data (period= hourly or 5minute data) made possible. if the url schema allowed for automatic or url-based time-period data resolution switching we could even make a little online tool GUI for users to generate specific time period, region, data-resolution and technology grouping and selection requests using a simplified GUI dedicated to the task, that generates the appropriate URL link. For those who don't know the secrets of zooming into graphs and filtering technologies etc, it would make it more explicit and more discoverable.

I just put some related comments on this thread https://github.com/opennem/opennem-fe/issues/147

currently the URL doesn't reflect "zooming in" on data. And Zooming in on data doesn't regenerate data at a more appropriate resolution for the period (see below).

e.g. June 2019 QLD looks like this (see below) and the URL doesn't reflect the zoomed state. https://opennem.org.au/energy/qld1/?range=all&interval=1M ← this shows all months on record in the database.

Screenshot 2023-07-19 at 3 38 31 pm