oembed / embed some of our pages on external sites

[scd]

I am motivated by the best way to get a competition onto a gym's WordPress site.

I am just thinking out aloud, but it seems that oembed (returning rich html) should do the trick.

A couple of issues come up in thought land:

1. How to get WordPress to trigger a oembed call on thecrag - one line needs to be added to the theme's function file OR we could write a very small plugin (seems to be the standard).
2. How do we implement with least effort? (see discussion comments below)
3. Are there any conceptual problems with all the links taking people to thecrag rather than the on the gyms wordpress site.
4. If we are linking back to thecrag then the links in the template should be fully qualified.
5. Is there any issue with Google seeing duplicate content? Surely there is a tag we can include (note to self to do some research).

thecrag.com/oembed end point is implemented. It is pretty simple for something like a photo, however for a more complex embed like a competition or a facet then we don't want to duplicate code.

For competitions/facets the www.thecrag.com/embed endpoint could make a call to www.thecrag.com/competition/1234?oembed=1 The oembed=1 parameter could tell the template to only include the template components that are required for the embed (ie not header stuff). This parameter could also set a global variable to tell templates to use fully qualified links.

Besides the benefit of competitions being embedded, it seems to me that facets could be a good target for rich html embedding. This would be an awesomely powerful tool for a blogger.

[brendanheywood]

Wordpress already supports oembed in core and a bunch of other add on plugins, there should be zero code, just a a bit of config to point it at our oembed endpoint:

http://codex.wordpress.org/Embeds

[brendanheywood]

or maybe a one liner:

http://codex.wordpress.org/Function_Reference/wp_oembed_add_provider

[scd]

Yup I have already caught up with this research. The issue is whether we want gyms to edit their Functions file to add a line of code. The standard seems to be that providers are writing a whole plugin which implements this one line of code, so gyms would install 'theCrag oEmbed' plugin.

[scd]

I have done some more work on this, to demonstrate we can embed photos on WordPress.

Oembed endpoint works for our photos:

http://www.thecrag.com/oembed?url=http%3A//www.thecrag.com/photo/521192313

(note that if the submittor is a private account then no author_url is returned. If the photo is under a publisher copyright then unotherised 401 is returned until we work out the rules).

I have created my first bitbucket repository for a word press plugin - 'WP theCrag oEmbed'. It was great to go through the full end to end process for the first time. I like this architecture.

I have updated howto the article on our site and published the wordpress plugin zip file:

http://www.thecrag.com/article/oEmbed

Interestingly wordpress do their oembed server side, but they cache their content. It looks like urls are mapped to an md5 key which returns the cached content. The thing with this is that a) dynamic content won't be any good and b) if you disable the plugin the cached content sticks around (it looks like it will eventually time out).

If we are going to do competitions then we must use iframes to get around this cached content problem.

If it works I am thinking ascent logbook and maybe other facetted searches????

[scd]

I have come across a little bit of a difficulty setting the height of the iframe to include all the content of the containing document. There seems to be a mess of solutions out there, but this one seems to be promisingly simple:

http://yysource.com/2011/08/source-code-iframe-with-100-height/

I am not sure, but maybe this one might be something we should use:

https://github.com/davidjbradshaw/iframe-resizer

Also an interesting discussion here:

http://stackoverflow.com/questions/153152/resizing-an-iframe-based-on-content

[brendanheywood]

Yeah this is notoriously ugly and the solutions keep changing every year as browser security models tighten. That git project looks pretty good

[scd]

Ok thanks. It is good to get your feedback on this one which makes me a lot more confident that I am going the right direction.

[scd]

Looking into iframe-resizer I have looked at David's example code which brings up immediate questions for me to investigate further:

• [x] Will this work cross domain. The example is not cross domain and everything I have read about the nastiness of this issue is cross domain. There is a crossOrigin property which implies the thing will work cross domain, but this needs to be demonstrated.
• [ ] Do I have to learn anything about polyfil - there is a script at the top of his example regarding MDN polyfil. Do we need this?
• [ ] If we polyfil stuff then then what implications are there on a WordPress site?
• [x] How to load jquery in Word Press site. Several articles about this, as a version of jquery comes with WordPress. Which brings up a whole heap more questions.
• [x] Does iframe-resize work with the Word Press version of jquery
• [x] Word Press recommends to use jquery in no confict mode, which is no good for us because we don't control this in the iframe-resize. So what are the implications of not using no conflict mode.
• [x] How to safely include third party scripts in word press.

[scd]

Hmmm, am I overthinking this. Cross browser issue means that the child page cannot communicate to the parent. However the parent can check the child size, which means that the parent size can be known on load. This will solve 90% of the problem for now, as the content within the page does not change size with UI interaction except for browser resizes. As far as I am concerned that would be ok. This leaves a fairly simple script to be implemented???? I'm going to the gym, and will look at this avenue when I get back.

[scd]

I looks like I will have to use iframe-resize after all as it looks like the main page cannot read the scrollHeight of the iframe page. I'm not 100% sure about this conclusion because there are so many things going on.

[scd]

FYI a demo of photo embeding on WordPress.

http://www.simon-dale.com/demo-of-thecrag-photo-embeding-on-wordpress/

[brendanheywood]

Nice

[scd]

And now for something really cool

http://www.simon-dale.com/test-iframe-resizer/

It's not yet oembed, but i have written a wrapper WordPress plugin to test iframe-resizer. Which works awesome. There are other issues, like making the competition tables responsive.

What is the etiquette, should I contact David Bradshaw and let him know that I have started some work that he may interested in pulling back into his main project?

Also I think I am going with '/embed' at the end of urls, which tells the system that the page is to be embedded. I have also made a generic variable $app->{embed} which can be tested in case various places need to output differently. Look at the competition template as an example. (note that I need to create an embed header template.

[scd]

oembed competitions now working on wordpress:

http://www.simon-dale.com/demo-of-thecrag-competitions-embeding-of-wordpress/

[brendanheywood]

When you do the embed header / footer, would be good to get something like this going - this is what an embedded gist from github looks like on a remote site:

I'm note sure the header needs much in it, certainly not the navigation or the tabs, possibly just a thinner style logo line next to the h1, and then some more details in the footer with links back to the original page.

This pretty well solves #838 as well, but in a much more elegant way than inside the photo data.

[brendanheywood]

Another good embeding concept for inspiration (and for the help article too):

http://blog.codepen.io/documentation/features/embedded-pens/

[scd]

And now for something really cool - my ascents logbook in a wordpress site:

http://www.simon-dale.com/demo-of-thecrag-ascents-logbook-embeding-in-wordpress/

I am actually excited about this, because ascents (and other facets) could potentially be really awesome for blogging.

@brendanheywood , what do you think? Is it worthwhile asking on Facebook for any takers?

[brendanheywood]

I find the format a little cluttered as an embed. If it was on my blog I would probably want this in a side bar, not as a main content page, and even if it was a full page embed I wouldn't want to see my name on every single tick, and a lot of the other stuff like crag is repetitious. I'd probably want to see something more like the tick column on the profile page:

But this has nothing to do with the actual embed which is great, more just with the logbook format itself. This actually idea crosses over a lot with what Garth was talking about with his use of the trips page. I'll write that up #1682

In terms of people to talk to, Lee comes to mind but there are heaps of people with blogs added as their website url.

Some climbers who have linked their blogs: http://www.thecrag.com/climber/monty http://www.thecrag.com/climber/verticalnomad http://www.thecrag.com/climber/wesleydouglas http://www.thecrag.com/climber/jjobrien http://www.thecrag.com/climber/manacubus

[scd]

Agree about the clutter in the current ascents facet, but as per #1682 we are going to sort this out.

Should we contact these guys individually or do a Facebook announcement.

The thing which excites me about this is that the embed makes url's behave as remote apps, for almost no work.

[brendanheywood]

I'd probably get some private feedback first. General rule of thumb is getting 5 eyeballs on something will find 90% of the quirks

[scd]

I have just looked at these blogs and only one is wordpress. Given this I think we need a different tactic. Maybe update our oembed article, asking people to help us.

I still think a call for volunteers who use WordPress may be useful.

[brendanheywood]

Most of the work is really in the ?embed or /embed logic. The oembed is really just a metadata layer on top with a consistent api. The obvious candidate for general purpose embed without a heap of technical skill is a JS wrapper around the lot, similar to how the FB like button works, or a gist embed works. ie a html snippet they cut and paste that JS loads a scripts from our server which then generates an iframe and then calls the oembed url and/or the /embed version of a page depending on what it is doing.

A good test use case might be worth talking to the canberra guys about seeing if what we do here could more easily replace what they did with the 'recent new routes' embed which is fairly crusty and inefficient:

http://www.canberraclimbing.org.au/routes--guides/new-routes.aspx

They are using some MS asp cms by the looks of it. It would also solve a lot issue they had, like trying to replicate and keep up to date with css styles etc.

[scd]

this is a good next step. The simplest embed is an inclusion of the js file for resizing the iframe and an iframe to the embed url.

[brendanheywood]

I'd probably make it even simpler so we can change the implementation easier later if we need to. Something like this:

<script src="https://www.thecrag.com/embed.js" async defer
   data-embed="/climbing/australia/gara-gorge/upper-gara-gorge/routes?sortby=when-fa,desc"></script>

This way we could change from an iframe to a seamless iframe, or direct embed, or whatever later. One thing to note is that the .js file can't really be updated, so it can't be far future cached. Doing js embeds can be a little hairy, there are some good boiler plates our there for helping manage it, see also:

http://lightningjs.com/ - look super solid but too heavy for my liking (it's a full requirejs lib)

There was another boiler plate I remember reading about which has a simple event api to how google analytics's does it but can't remember of the top of my head.

[scd]

So I am gathering that it is not a simple as loading our iframe-resizer javascript file and then creating an iframe on the fly?

[brendanheywood]

Yes it is that simple - I just want to retain the flexibility to change the implementation later without the other website changing the embed code. So the embed.js file I mentioned would initially do exactly what you said, but would just have a generic name without mentioning iframes or resizing. It would also have a relatively short expires header, say 1-2 weeks so that, and not be versioned, so we could later convert how it embeds to something else if needed.

[brendanheywood]

Oh, sorry I think I did miss something - yes the embedding process is more complicated that you think. In particular the way you have embedded in that wordpress plugin is open to a single point of failure issue. Lets say that thecrag.com is down, which you can simulate by hacking your hosts file or with a SPOF chrome plugin, then go to your wordpress page and load it. It never finishes loading it just hangs.

We don't want to negatively impact the other site so we have to be very careful with how and when we load scripts to not block either loading, or rendering of their page. There have been a couple highly publicized cases of the old facebook like button bring down vast sections of the net when facebook has had an outage, and with many pages now using multiple 3rd party the scripts the risks are multiplied if any of them go down, so doing it right is important.

And depending on how old the browsers are that you want to support, the way to do it is a little hairy. html5 solves this fairly well with the async attribute but this isn't supported in old IE. This is why snippets like GA and FB like buttons are always a chunk of JS and never just a script tag.

According to our GA stats, 15% of users browsers don't yet support to async tag so we probably need a snippet with deeper support, that script tag above isn't good enough

http://caniuse.com/#feat=script-async

[scd]

This is good to know. Still some more hurdles to jump. In summary to do this properly we need a javascript snippet for embedding similar to facebook.

BTW the wordpress integration is probably even more complex because it has it's own oembed implementation constraints. The oembed result associated with the url is cached and if we are down when the article is saved then it will be cached undefined.

This problem seems to also be endemic with oembed which typically uses iframes for video embeds.

To me this means that it is accepted risk for a lot of use cases.

So maybe where single point of failure only effects a single page (eg utube embed or thecrag competition embed) then maybe we don't really have to worry about doing anything too sophisticated and what we are currently doing is suffice. These embeds are typically on a destination page, so it does not matter too much if they temporarily fail.

But if we are embedding something that operates throughout a third party site (eg similar to Facebook Like button) then we should do this properly. Because it does matter if we effect a whole site.

[brendanheywood]

hey just a heads up that as part of #1179 I'm also gonna start refactoring some of the new embed logic, ie the header and footer so it's more generic. Should be done shortly

[scd]

ta.

[brendanheywood]

Ok finished the embed cleanup, check it out:

http://www.simon-dale.com/demo-of-thecrag-ascents-logbook-embeding-in-wordpress/

[scd]

sweet, it is coming together. Let's keep embedding in mind as we fix up all the facets.

[brendanheywood]

hey just tried to make a mockup for the canberra guys here:

http://www.simon-dale.com/canberra-new-routes-embed-tests/

It doens't work, I've added the extra regex for the url match to wordpress, is there something on the server side to also do? I can see a HEAD request coming through but no GET for the actual embed page

[scd]

I don't think I have hooked up all the pieces for anything but ascents facet at the moment. Shall we just do all the facets this way?

[brendanheywood]

ah yeap just found the regex in OEmbed.pm - yeah I think we should make all the facets handle it the same way. Is i just that regex or other stuff to change?

[scd]

There should be three things to do

1. Update regex in the plugin
2. Update regex in the OEmbed.pm page
3. Facet template handles embed flag correctly (I think you fixed this across all facets in your last update?)

Do you want to leave this with me?

[brendanheywood]

yeah sure thanks

[scd]

try it now - good demo btw.

I think we need to show the actual urls used by wrapping them in a <code>

[brendanheywood]

From #1682 / @oliverstory:

The criteria we're currently using for the Canberra new routes page is 'routes with an FA or FFA date of 1998 or later in areas [11743723 11743795 11743747 11743771 11743819 11743843 11743867 11831941 391992096 23373237 510681699]'. The idea is to show all new routes since the last guidebook was published.

The data comes from the URL you guys helped create: api/area/ids?id=[comma-separated list of area IDs]&show=routeswithhistorysince,karma&since=[date]&key=lykevgk8qlltq2o9

Ok so that's equivalent to this page: (also added top rope, there are 3 of them) totalling 448 new routes which is way way more than what I thought was on that page which I'd estimate at 250-300.

http://dev.thecrag.com/climbing/australia/new-south-wales-and-act/australian-capital-territory/routes/with-gear-style/trad+sport+top%20rope+aid/between/1988+2015/?sortby=at,asc

Anyway, next question @oliverstory: this currently work using the oembed standard, I'm not sure what CMS you are using but looks like some custom asp stuff? Would it be able to use oembed?

But we also think oembed will probably be too hard for most non-technical people so we're thinking about making a little embed code that can go into just about any html page which hides away all the complexity (like a FB button js snippet)

[oliverstory]

Looks like you put '1988' instead of '1998' as the starting FA year. 1998-2015 gives 186 routes, which is about right.

We are using Umbraco CMS. Looks like there are a couple of plugins that support oembed for Umbraco, but also looks like they only support providers hosted on embed.ly : http://our.umbraco.org/projects/tag/oEmbed. I like the idea of a javascript snippet. That way you as the authors don't have to worry about all the different CMSs out there.

If you do register with embed.ly, it looks like the plugins allow content editors to add oembed references from the rich text editor interface

Some comments on blog post sample:

1) I like the layout of the 'sorted by area' view

2) No concern with the links going back to the relevant thecrag page. Personally I don't like links opening in a new window by default though - this breaks the 'back button' as the way to get back where I was a moment ago.

3) When I mouse over a route name, the pop-up has lots of ' ' in it (using Chrome)

4) In the bottom right of the iframe I'd write 'View source page' rather than 'View original page'. Implication is more seamless.

5) The 'Crag' column seems to be the first level from the top with type 'Crag'. Many of these I'd characterise more as a 'collection of crags'. For example, all of Orroral Ridge, River Crags and 'Gibraltar and Corin Road' are set as type 'Crag' but only contain other 'Crags'. As an alternative, you could show here the crag that's closest to the route in the hierarchy. For example for 'Moth' the crag would be Gibraltar Peak.

That's enough from me tonight!

[scd]

I am closing as this issue is substantially done. Anything remaining should be broken out into it's own issue.