ThePatr10t / yall-Qaeda

73 stars 13 forks source link

lat/long precision #42

Closed mistakenmustard closed 3 years ago

mistakenmustard commented 3 years ago

First off, thanks for all your work here!

I'm working on a project to consolidate a handful of sources and I noticed that your KML file has a significant amount of precision for lat/long, more so than the metadata JSON files each entry points to. I assume the metadata JSON files are close to the original data, so where does the precision in the KML file come from?

ThePatr10t commented 3 years ago

Sharp eye!

Basically, the lat/lon precision from the original files was rounded, and because of this, markers were stacking right on top of eachother, forming a grid pattern. To get around this, on early versions of the map, I would add a small random shift to the lat and long points to spread them out a bit.

But then I noticed that whenever I added new videos to the collection, the random number generator would shift them differently and Id never remember which was which. So I found some code on stack overflow that creates a random deviation based on the ID of the video itself. Basically a function that maps any arbitrary string -> random number between 0-1.0

The code that implements this is the following - note that for longitude, I just reverse the video ID string from right to left to get a second random seed : https://github.com/ThePatr10t/yall-Qaeda/blob/8716a6907130131824f7d757a34c5f8da2d691c8/parse.py#L145

Also note : this same random generator also picks the coloring of each point, in a way that is reproducible each time the parser is run.

mistakenmustard commented 3 years ago

Perfect, thanks for clarifying. This is what I assumed was happening. It makes for a really nice map, more so than those "clustering" libraries. Nice work!