search

okfn / dataproxy

Web application (targeted at appengine) to proxy data from certain data types into a JSON-P data type so that users can create mashups against remote data sets.
http://jsonpdataproxy.appspot.com/
Other
75 stars 24 forks source link

[Requires discussion] Enable type guessing by default. #20

Closed domoritz closed 11 years ago

domoritz commented 11 years ago

Type guessing now has to be disabled by setting guess-types=false.

domoritz commented 11 years ago

Speed tests

With type guessing

http://jsonpdataproxy.appspot.com/?&url=http%3A%2F%2Fafghanistanelectiondata.org%2Fsites%2Fdefault%2Ffiles%2Fdistrict_centerpoints.csv&max-results=1000&type=csv&guess-types=true

Takes about 1.8 seconds

http://jsonpdataproxy.appspot.com/?url=http%3A%2F%2Fckan.net%2Fstorage%2Ff%2Ffile%2F3ffdcd42-5c63-4089-84dd-c23876259973&max-results=1000&guess-types=true&type=csv

Takes about 2.47 seconds

Without type guessing

http://jsonpdataproxy.appspot.com/?&url=http%3A%2F%2Fafghanistanelectiondata.org%2Fsites%2Fdefault%2Ffiles%2Fdistrict_centerpoints.csv&max-results=1000&type=csv

Takes about 800 ms

http://jsonpdataproxy.appspot.com/?url=http%3A%2F%2Fckan.net%2Fstorage%2Ff%2Ffile%2F3ffdcd42-5c63-4089-84dd-c23876259973&max-results=1000&type=csv

Takes about 1.36 seconds

rufuspollock commented 11 years ago

I think the cost of a second is worth it. So shall we put this in and see how we go?

domoritz commented 11 years ago

I think we should try it, since the cost of enabling or reverting it is negligible. Also, we should test it while we are preparing CKAN Beta 2 so that we can test CKAN+Dataproxy together.

rufuspollock commented 11 years ago

Please merge it. I can then deploy ...