Enhance data source interactivity

[joshuagraber]

Comments from @josh-chamberlain

What do you think about having a layer of map icons for agencies without any sources, set to 20% opacity? That way there would be little gray icons showing what we don't yet have. Could be too scary.

What do you think about a little box which shows the agencies that appear on the map, in list form? Clicking them would take you to a search with the appropriate filter for agency, or if you clicked the county you'd get everything from the county. If there are cases with too many agencies, we could truncate to "...and 117 more agencies"

[joshuagraber]

Hey @josh-chamberlain I started looking into this yesterday, a couple of comments:

What do you think about having a layer of map icons for agencies without any sources, set to 20% opacity?

This is possible with Mapbox, but we'd be back to fetching lat/lng via a geolocation API in order to render the markers in the appropriate location, and our usage would put us into the paid tier of most geo APIs (Mapbox included).

What do you think about a little box which shows the agencies that appear on the map, in list form? Clicking them would take you to a search with the appropriate filter for agency, or if you clicked the county you'd get everything from the county. If there are cases with too many agencies, we could truncate to "...and 117 more agencies"

Love this idea. Have been working on sorting data for this, etc. Doing it by county name might be difficult as presently constructed, because we don't currently return county_name from this API endpoint. @mbodeantor do you think we can add county_name to the data returned here?

[josh-chamberlain]

@joshuagraber

lat/lng: we could use the google maps API for this, too—or mapbox. I'm on the mapbox pricing page and I'm not sure exactly which one we'd need, but after the first batch of 20k, we could store them in the db and wouldn't need more than a slow trickle—unless there's some reason I'm missing to fetch them every time. I think the cost will be reasonable and we should go for it.

[mbodeantor]

@josh-chamberlain @joshuagraber If we are just looking for the lat/long for each agency based on zip, there's this: https://gist.github.com/erichurst/7882666

Otherwise we will need a process for finding each agency's address.

[mbodeantor]

@joshuagraber added county_name to the /data_sources_map endpoint: https://github.com/Police-Data-Accessibility-Project/data-sources-app/pull/245

[joshuagraber]

added county_name to the /data_sources_map endpoint

Thank you, @mbodeantor, ✅

If we are just looking for the lat/long for each agency based on zip

@mbodeantor I don't know if that's going to work, given that not all agencies have zip code. I think the best bet is to look up using a geocoding API by place name, then filtering by state/municipality to de-dupe places with the same name in other states, etc.

but after the first batch of 20k, we could store them in the db and wouldn't need more than a slow trickle—unless there's some reason I'm missing to fetch them every time. I think the cost will be reasonable and we should go for it.

@josh-chamberlain Looks like there are approximately 215 sources with no lat/lng data. That's about 90 visits to the map before we reach 20,000 lookups. (It may be discounted if we batch, but nonetheless, it'll rack up quickly...) If the idea is to update DB records with this lookup info, I'd suggest we do that server-side or via scripting.

[josh-chamberlain]

@joshuagraber your strategy sounds good, but if it helps disambiguate: many of the agencies without a ZIP have a jurisdiction_type of county, state, or federal. I would expect the local to have ZIPs, but there are probably still significant gaps. Looks like that might not appear on the API endpoint.

Agreed that we should script this instead of doing it on the front end—sounds brittle/slower for something we won't need to update that often. I made an issue.

[joshuagraber]

Sounds good @josh-chamberlain. Thinking a bit more about it, that zip code data set could be useful (and inexpensive) for backfilling this data, no?

Also, one question on the map sidebar:

Clicking them would take you to a search with the appropriate filter for agency, or if you clicked the county you'd get everything from the county

Could you clarify this a little bit? Is each county and agency meant to be clickable and take the user to the data sources search route? Or when you're talking about filtering for agency, do you mean filtering results on the map and showing more detail in the side bar?

[josh-chamberlain]

@joshuagraber yes, good clarification! A search route. If we did it today they would take you to these:

https://data-sources.pdap.io/search/all/allegheny%20county https://data-sources.pdap.io/search/all/allegheny%20county%20jail

[joshuagraber]

@josh-chamberlain Hmm, I noticed as part of this that not all agencies return data when they are searched for. So as of right now, I'm searching for municipality. Do we want to update the API? Or won't it matter, since search V2 will change everything anyway?

[josh-chamberlain]

@joshuagraber search v2 will change everything, but I imagine we would still want a way to search / display sources by agency. That seems like a bug to me.

[joshuagraber]

search v2 will change everything, but I imagine we would still want a way to search / display sources by agency. That seems like a bug to me.

Totally agreed. FWIW, I tried searching for agency in the search term as well, and same result. Might make sense to add this functionality there (searchTerm) rather than in the location param, no? Wyt, @mbodeantor?

[josh-chamberlain]

I think agency is part of the location param. I think of those as "type of data" and "jurisdiction of data"

[mbodeantor]

Yeah location makes more sense to me

[joshuagraber]

Makes sense to me as well