search

digitalmethodsinitiative / dmi-tcat

Digital Methods Initiative - Twitter Capture and Analysis Toolset
Apache License 2.0
367 stars 114 forks source link

Validate bounding boxes when creating geotrack bins... #260

Closed mikesname closed 7 years ago

mikesname commented 7 years ago

Currently if you accidentally add a bin with a malformed bounding box, e.g. "51.513586, -0.134843, 51.512915, -0.134594" (second latitude lower than south-west corner), it will cause 406 NotAcceptable errors from the Twitter API.

It would be nicer if the TCAT did a sanity check on these values before creating a geotrack that will break harvesting.

dentoir commented 7 years ago

Hi @mikesname,

It's a good idear to try and prevent TCAT from not running those queries. The problem is we cannot do a simple numeric comparison. If the second latitude is lower than the first, like in this example, it means the geo area queries is very huge (covering most of the Earth). The 406 NotAcceptable is thrown for very large areas; I think it is also thrown when querying all tweets from France for example. So if we want to catch these queries from the front-end, we should calculate the area size in PHP and define our own maximum.

dentoir commented 7 years ago

I've pushed a fix for this issue to the PHP 7 branch, which should be merged into master soon. It currently denies adding huge areas approximately as large as France. The value could be tweaked, though. It should help avoid obvious errors.