[Maps App] Introduce Support for Torque.js

[senakafdo]

Yet another library from CARTO, makes it possible to plot timeseries data on maps. See here and here for more information.

[jamesscottbrown]

We should consider how much functionality we really want to add to the Maps App. There is an almost unlimited amount of features we could add, many of which might be used only once (or perhaps never).

Currently, I see two distinct use cases for the Maps App are:

• to simply display map tiles quickly and easily, without needing to choose a library and figure out how to use it's API
• to display more complex representations of geographic data: these are currently inconvenient to represent in the HTML App due to some of its limitations

Rather than adding more and more specific features to the Maps App I think that it would be better to increase the power of the HTML App, so that people can use it to create visualizations using whatever libraries they prefer.

Specifically, the most significant limitation of the HTML App is currently that it does not respond to pan/zoom actions initiated by the universal controller (or by POSTing a transform to OVE Core). One way to accommodate this would be for the HTML App to fire an event in response to these, and allow JavaSCript in the page loaded by the app to register a callback function to respond to this.

This would allow me to write a page which creates one map for each oveSectionView, and ensure that each was initially panned and zoomed correctly using the information in window.ove.geometry. I could register a callback that would adjust the zoom level and pan correctly in response to controller actions. I could then use whatever mapping library I wanted, without making feature requests on the Maps App.

We could distribute example projects showing how to do this with some popular libraries, so that people could use these as templates.

This feature is also useful beyond maps: it would also allow me to create things like a large scatter-plot can be zoomed and panned (and in which each oveSectionId only paints a canvas corresponding to the region of the space that it is responsible for).

I also previously suggested enabling the controller to control the HTML app by adjusting the CSS transform on the corresponding iframe, so that 'normal' pages could be panned and zoomed. I think that both modes of controlling the HTML App are useful in different circumstances, and both should be implemented.

[senakafdo]

@jamesscottbrown, below is a lengthy answer. I have broken them down into sections.

Audience So, we have two clear target audiences when it comes to the collection of Apps we have in OVE (or any similar framework) for that purpose:

1. Those who have data and would like to spend less time on developing and maintaining some application that would fulfil some of the straightforward needs they have.
2. Those who would like to develop a visualisation (generally one-off) that is far more superior and would generally require a lot of custom integration.

The Maps app is designed for audience 1 mostly and provides a clear extension point for anyone who would be interested in 2, provided that they would be able to make use of the functionality provided by OVE.

Design Decisions and Limitations In OVE, we promote a way of developing your app as a collection of various components that would fit together rather than developing a single component that would be doing everything. And, there is a messaging layer that lets you communicate from client to server, one type of app to another type of app, or to your peers that are loaded within a single browser instance.

The OVE library is the basis of all of our apps, and it is possible not simply to use the HTML app but to develop a 100% external application and still use OVE to get your app to properly resize and interpret screen geometries accordingly.

We however do not provide good enough tutorials for people to understand how to use these apps and develop visualisations. But, with requests like yours and also what we have been doing so far, we are aiming to get many more platform-level features done to enrich support for apps. So, in a way what you suggest that we must not do to the Maps App is exactly what you suggest to be done for the HTML App.

However, the two scenarios (extending the Maps App vs HTML App) are also not entirely the same. If you know the type of content that you are working with (maps vs videos vs images etc) it would be easier to develop graphical tools and APIs compared to having a single very capable HTML App which then becomes very hard to maintain and manage.

Comparisons While it does sound easy enough to implement what you suggest, if we had one single app instead of the 14 that we have right now, the controllers and also the implementation would not be of this quality of maintainability. At the DSI, we have already had enough of experiences working with monoliths and have a very good understanding of their limitations.

But, at the same time, a collection of isolated apps (or services) are also equally challenging. However, this is a challenge that can be overcome with proper documentation, examples and good thinking. A good example is the moduler design of D3 compared to what it was before.

One limitation in OVE however, is the inability to extract these individual components and bring them together within a single app, if needed. This is what #26 is all about.

Roadmap/Future Work So, if we are to get to a level where we can achieve #26, we have to be working in a model where we are very systematic about how we introduce features and how we focus on extensively testing the individual components that we have. Unless all of these existing apps are thoroughly tested, we wouldn't know which designs work best and which don't. This is the reason behind improving the coverage of some of the apps, so that we can start using them for usecases that are much beyond basic displaying of content. And, that is why we are integrating with CARTO components etc.

Now, in terms of the HTML App, we can also add zoom/pan features as you suggest. But, these would be very implementation-specific. An alternative would be to simply broadcast the events and then let the developer decide how they would like to deal with it. All existing webapps/pages will either have to be replaced or redesigned as a result.

Practical Aspects We also need to think what zoom and pan means in the context of different apps. The reason why you would want OVE to provide zoom/pan capability and not your HTML content is also down to your specific design choices. The WebGL demos that use the HTML App and the SVG demos that uses the SVG App (which is nothing more than the HTML App + Tuoris in reality) are great examples of how it is possible to design a controller for interactive content. These can be moved around, zoomed in or out, or even transformed and animated to suit your preferences.

The HTML App (as-is today) offers what is not available to you as a web developer, which are the ability to automatically scale/resize to suit a variety of large or small screen sizes, a messaging layer to communicate across distributed nodes, and a mechanism to make parallelise functions or distribute client/server-side state.

Zooming and Panning on top of this would be mostly targeting changing the position or dimensions of the content rather than the app being interactive.

The HTML app is not generally designed for someone who is working with a canvas that is as large as a DZI or map made of 256x256 tiles, or not even a large network diagram. It is mostly aimed at someone who would be displaying content at fixed positions but changing what is displayed within those fixed positions based on the changes of their surroundings.

I could be mistaken, but what you seem to be suggesting is to generalise all operations to an extent where you as the developer of the content of what you see within the HTML App need not interpret the changes to external state but for the framework to tell you exactly what changed.

Special Notes I believe your point on the universal controller could be slightly down to the fact that you didn't understand how easy it is to intercept any message that is flowing within the OVE framework and interpret it accordingly. Take a look at the tool I developed to debug messages that are sent back and forth in OVE. With a very minimum amount of code I can listen to messages sent specifically to one type of app or even one sectionId. So, you too could develop such a simple app to react to the universal controller's commands or even adapt to changes in any other app for that purpose.

I would say the reason why this is not so obvious is because we have no good tutorials explaining what you can do with OVE. My suggestion for you would be to start developing some good visualisations, maximise the use of existing OVE functionality, and to discuss the limitations that you come across and find appropriate resolutions. These can then be the basis of such tutorials explaining how to use OVE to solve real-world visualisation requirements, similar to what frameworks such as D3.

[jamesscottbrown]

Having thought about this more, I think that we should leave the HTML App as it is.

However, I still think that we should be hesitant about extending the Maps App by continuing to add libraries such as Torque that are only useful for relatively narrow use-cases.

[senakafdo]

It seems that torque.js is not well maintained.

Only 1 of 5 of their examples for Leaflet actually work. And, even in the one that works, it fails when we have have a non-integer zoom level in Leaflet. I can however confirm that the integration does work when the zoom level is an integer, which can be achieved by manually setting a state. I have reported the issue at CartoDB/torque#301.

The implementation for OpenLayers also doesn't work, and have two bugs that I have reported, CartoDB/torque#302 and CartoDB/torque#303. However, unlike in Leaflet, it works at any zoom level and I have included 2 temporary fixes in the code.