Sharing my RESTful code

[adam-s]

I built 954live on top of RESTful-Drupal. The repository is at https://github.com/adam-s/954live. Instead of building a headless drupal instance, I used RESTful internally as a query engine instead of Views. The implementation is super clean, no clutter, very little configuration is in the database, and easy to extend to solve problems keeping things organized. The more I used RESTful the less I used it to format data and wrap it with HTML. The livesearch module uses AngularJS embedded in Drupal.

[e0ipso]

This is so awesome!

I'd love to see a more in-depth article about the experience of using RESTful as the query engine. Do you plan on writing something like a blog post?

[theodorosploumis]

I just added a new section on the Wiki about real apps using the RESTful module. If the showcase list is too long we can break it to a new Wiki page.

[e0ipso]

@theodorosploumis I'm not sure I see the value of having a show case for a back-end technology like RESTful. I'd rather have there a compendium of blog posts explaining what you liked about RESTful, what could have been better and what are the interesting edge cases you had to solve (and how).

Maybe @amitaibu has some input about this too.

[amitaibu]

Yeah, I agree that links to /api don't add much. I think only proper blog posts should be added there

[amitaibu]

(but of course thank you for taking the time to work on it :wink: )

[e0ipso]

(but of course thank you for taking the time to work on it :wink: )

Indeed! I forgot to thank @theodorosploumis because he is like RESTful-module-family.

[amitaibu]

:+1:

[theodorosploumis]

I 'd like to add a blog post about RESTful. Mostly I wanted to share the Docs and the code. Furthermore, I am preparing something bigger here in Thessaloniki which is a presentation about RESTful options with Drupal 7.x and 8.x and of course this module will be there.

I am not a developer actually, I am a front end developer. The README file lies about the audience!

[jeff-h]

@adam-s Thanks for sharing. Nice site!

I'd love it if you had the time to add a wiki page describing some of your lessons learnt doing RESTful-from-inside-Drupal.

For example, you touched on the fact you used RESTful instead of Views. Given the developer skills needed to leverage RESTful in this manner, I'd think a more interesting question is why you chose RESTful as a query builder instead of hand-built EFQ, and what you enjoyed most about this approach.

Another topic would be why you chose to not do the trendy cool-kid thing and built an SPA front-end client, and instead stick with the fork-in-eyeball Drupal theme layer :)

[adam-s]

I very much would like to make a wiki blog post to hopefully inspire more people to use RESTful. Until then one important lesson I learned was not using formatters.

I had major difficulties with using formatters defined on public fields. If you, @e0ipso, do spawn 7.x-3.x which you say you won't soon, I recommend deprecating formatters. Perhaps, because of Drupal 8 core code being consistent it will work better in that version. There were so many edge cases to handle that had little to do RESTful implementation, but rather different implementation of field formatters in core and worse in contributed modules, I just gave up. At first, I used process callback functions and handled the theming of data there. Later, I simple passed entire objects to a class that built a render array. I have a hunch that the reason the problems with formatters haven't been addressed earlier is that developers using RESTful are not using them. If you do get to 7.x-3.x, maybe do a survey of people who use formatters. If it is very few, ask the few who use them if they can handle wrapping code in HTML either in a process callback function or later in the rendering.

@jeff-h I, last year, decided to not use Views on the first iteration of 954live and instead used EFQ. The big lesson was as massive and clunky Views is, it is optimized. I had to decide which approach would get the end result faster, hacking Views or writing custom caching and optimizations to EFQ. In the end, I used Views, but I had to deal with the problem of Views spitting out little parts of data into different templates all over the place. There is wall where the complexity is too much and there is an inability to test.

Why RESTful?

1. RESTful has amazing caching out of the box. <------
2. The API implementation of RESTful is simple, logical, and straight forward.
3. RESTful provides a way to process outbound information at the field level with process callbacks.
4. I can now solve any problem very quickly. It took me 3 weeks to build the site, 2 of which were because I was learning how to use RESTful. The speed I can prototype a project is of upmost important since most startups and projects by any developer fail. If a Drupal project built on RESTful did go viral, it would performant enough to hold out until the amount of traffic justifies millions of dollars in venture capital.
5. If all the information is in one object, it is easy to build a render array.

Why did use Drupal instead of an SPA?

1. The search page is a hybrid like MSNBC and Weather.com, Drupal with AngularJS. The search page is just proof of concept. I can build the entire site as an Angular app. The REST server is already built and so are all the templates. If a person searches for a venue or artist, right now it shows the venue card or artist card inside the search page. That is pointless but I wanted to see how hard it was to be done. I built that in a couple hours. I can wrap information in markup on the server or with Angular on the client, so who cares where it happens as long as I'm being pragmatic?
2. Search engine optimization. This should probably be reason number 1. The website right now only provides information. Why do I need an SPA? It would take me several weeks to do SEO without all the stuff Drupal provides out of the box. Drupal does it out of the box. There is a lot that goes into it. Google has a 20 page document on how to help them help us help them make good looking search results and let the Google algorithm know about the content. The document has cartoons. It's 20 pages. Why mess with it? The website was only created to promote small local bands and when the New Kids on the Block page got 10,000 page views from organic search results because Google put it on page one, I think I figured something out.
3. Right now I'm rebuilding the backend built on MongoDB, Express, AngularJS, and Node.js. The big difference is that artist, venue, and event data will be stored in Drupal accessed through RESTful instead of in MongoDB like it is now. Not only did RESTful make it easy to build pages, it also provides a REST server without any more cost.

Cool-kid thing?

1. My laptop is a $200 Chromebook.
2. Boyd's Law of Iteration.

Boyd decided that the primary determinant to winning dogfights was not observing, orienting, planning, or acting better. The primary determinant to winning dogfights was observing, orienting, planning, and acting faster. In other words, how quickly one could iterate. Speed of iteration, Boyd suggested, beats quality of iteration.

I think those are some good reasons.

[jeff-h]

Thanks for the thorough writeup! I do feel something like this should be promoted somehow outside of the issue queue, but I'm not sure the wiki is the right place as this is somehow more subjective, more of a journey. The wiki feels more black-and-white I guess.

If you do get a chance to put this in a blog post, I'd stick a link to that straight into the wiki! :)

ps Re: cool-kid thing — your Chromebook proves you are a cool-kid lol :)

[e0ipso]

If you, @e0ipso, do spawn 7.x-3.x which you say you won't soon, I recommend deprecating formatters.

That's one of the intentions for 7.x-3.x. That feature, in any case, should live in a separated contrib. restful_token_auth is another case just like that one.

[e0ipso]

@adam-s it's great to get to see your journey! I really enjoyed the post.

[adam-s]

I'm going to continue a little bit here, consider this notes for a future write up on why and how to use RESTful internally.

Google has a Knowledge Graph(aka Rick Snippets) which is Search Engine Optimization on steroids. The way it works is information is wrapped in structured markup using the Schema.org microdata specification. Drupal touches on this with RDFa in Core and in the Microdata contrib module. However, both these implementations fall very short of the Google requirements for defining several different types of data including specific defined event data. Perhaps, I missed something in my research and I would like someone to point that out if it is the case. Once a list of events are rendered with the Views module getting the correct markup to meet the Schema.org specification is a monster hack. I did it but it wasn't pretty. All the required information, including venue address, event date, band names, band genres, venue location, event name, event url links, band url links, venue image, band image, ect. which in the normalized Drupal database are across several tables have to be in the same object at some point to be rendered either wrapped in html or in a JSON-LD object which is preferable.

I had made a mistake of not defining the entry point context to the data in the JSON-LD object. After I corrected the mistake last night and had Google through the Webmasters Tools dashboard re-index the pages, I found Knowledge Graph search results today. It didn't take long. This is what it looks like.

The website data just doesn't render to a HTML page in a browser window, it is rendered to Twitter and Facebook cards and in Rich Snippets in Google search results. Fair enough, the Metatags module handles the Facebook and Twitter cards very well. Don't get me started right now on how using microdata lets Google know exactly the content of your images so the engine is confident displaying the image in image search results. Or, how Google is having me train AI about the content of images with recaptcha. That's another story for another day. Why is Google putting images from 954live, a crappy site at the time, in the top left image position of several artists on the front Google search page? My guess is microdata. I told Google what it was.

What does correctly formatted JSON-LD look like on a webpage? I didn't pretty print the JSON output on the AmericanAirlines Arena(view-source:http://www.954live.com/venues/miami/americanairlines-arena) page so I added the Rock genres(view-source:http://www.954live.com/genres/rock) listing page for a better view. How did I get that markup? It was quite easy. I used RESTful to return a Simple JSON object which I decoded. In the future I might just write another JSON formatter class which is plugable instead of running through all the Simple JSON or JSON API code which is a bit overkill for this use case. Here is the code. I iterate through each event object and create an array which has the keys as defined in the Schema.org Event specification, render it with json_encode(), and then use drupal_add_html_head() to add it to the HTML <head>. As it is now, I query the database once to render the HTML and then I query the database again to render the microdata. I should do it at the same time, it is on my TODO list.

That is why! Now I have a JSON API implementation on top free as a side effect to use in a hybrid Angular / server rendered HTML app -- that is just sugar on top.

[theodorosploumis]

@adam-s I am really interested to see a generic JSON-LD formatter for RESTful module using core's RDFa functionality.