Mapbox viz doesn't work with Druid

[LAlbertalli]

Hi,

Trying to use the new MapBox visualization against Druid, the query fails with the following stack trace:

[Fri Jul 01 06:12:22.021652 2016] [:error] [pid 55345] 2016-07-01 06:12:22,021:ERROR:root:query() got an unexpected keyword argument 'columns'
[Fri Jul 01 06:12:22.021712 2016] [:error] [pid 55345] Traceback (most recent call last):
[Fri Jul 01 06:12:22.021721 2016] [:error] [pid 55345]   File "/usr/lib/python2.7/site-packages/caravel-0.10.0-py2.7.egg/caravel/views.py", line 831, in explore
[Fri Jul 01 06:12:22.021748 2016] [:error] [pid 55345]     payload = obj.get_json()
[Fri Jul 01 06:12:22.021755 2016] [:error] [pid 55345]   File "/usr/lib/python2.7/site-packages/caravel-0.10.0-py2.7.egg/caravel/viz.py", line 294, in get_json
[Fri Jul 01 06:12:22.021762 2016] [:error] [pid 55345]     'data': self.get_data(),
[Fri Jul 01 06:12:22.021768 2016] [:error] [pid 55345]   File "/usr/lib/python2.7/site-packages/caravel-0.10.0-py2.7.egg/caravel/viz.py", line 1800, in get_data
[Fri Jul 01 06:12:22.021775 2016] [:error] [pid 55345]     df = self.get_df()
[Fri Jul 01 06:12:22.021781 2016] [:error] [pid 55345]   File "/usr/lib/python2.7/site-packages/caravel-0.10.0-py2.7.egg/caravel/viz.py", line 156, in get_df
[Fri Jul 01 06:12:22.021788 2016] [:error] [pid 55345]     self.results = self.datasource.query(**query_obj)
[Fri Jul 01 06:12:22.021795 2016] [:error] [pid 55345] TypeError: query() got an unexpected keyword argument 'columns'

From a quick recognition of the code, it seems that the graph relies on the columns feature implemented for Tables but not for Druid Datasources. The way in which the feature has been implemented is not clear to me. But for druid you could consider of checking this: http://druid.io/docs/latest/development/geo.html

[mistercrunch]

@georgeke

[LAlbertalli]

I've played a little bit more with this feature, and I found some additional issues that probably should require a complete rethinking. My first problem (not the biggest one, but definitively an issue for a released feature) is that it requires setting the config param MAPBOX_API_KEY but this is not documented nor it is documented How I could get a key (I have to signup for MapBox). Furthermore, using an SQL backend, where it works, the queries with real production data are incredibly slow (using HP Vertica 7.2 and MemSQL 5.0). Here are the issues I found:

• The query function returns all the data in the timeframe selected with no regards to the shown area. This means in my case returning to the browser in the order of ten million data point for one day. It killed Vertica for few second and crashed my Chrome due to memory pressure (on a beefy Mac).
• The feature ignores the optimized geospatial indexes and data structure of more modern databases (it is pretty complex to pull this out, though)
• The feature also doesn't try to perform some pre-aggregation on the server to reduce the traffic to/from caravel.

As is, I'm going to disable the feature in the 0.10 deployment of Caravel since it is not suitable for production.

[edevil]

Just to make things clearer, at this time the Mapbox visualization does NOT work with Druid using either spatialDimensions or normal dimensions for coordinates, even after #725. Am I right?

[toabey]

Druid doesn't support dimensions other than String datatype as of Now, development is in progress for other datatypes in dimension then it works with same configuration.

Spatial Dimension is supported in Druid , but Mapbox visualization doesnot use it as of now.

[mistercrunch]

My feeling is that this won't be a priority for us at Airbnb for a little while, help is welcomed if someone wants to add the feature!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.