While working through the upgrade, I discovered 2 other bugs related to houseQuery.
We're using string datatype for address number range fields (LFROMHN, RFROMHN, LTOHN and RTOHN). Unfortunately, this does not perform as expected on ES Range Query since string ranges are calculated lexicographically, not numerically. The result is "1000" < "200" < "30".
The simple fix to just change grasshopper-loader's censusType. However, this is complicated by the fact that not all address numbers are just numeric. Some address ranges have letters and other characters like - as well.
There are probably a few approaches for fixing this, but they'll most likely involve changes to both the loader and CensusGeocoder.
We're assuming that the FROM values will always be less than the TO values . Unfortunately, that is not the case as seen in the example below.
As mentioned on https://github.com/cfpb/grasshopper/pull/211#issue-143093233, I commented out
houseQuery
fromCensusGeocoder.searchAddress()
(see: https://github.com/cfpb/grasshopper/pull/211/commits/6dd5d5834794c5fdbaad67436a8e498084276050#diff-6b3a25548e48dfe8c1891ebf6afef9efR87) in order to get it working post ES 2.2 upgrade.While working through the upgrade, I discovered 2 other bugs related to
houseQuery
.We're using
string
datatype for address number range fields (LFROMHN
,RFROMHN
,LTOHN
andRTOHN
). Unfortunately, this does not perform as expected on ES Range Query since string ranges are calculated lexicographically, not numerically. The result is "1000" < "200" < "30".The simple fix to just change grasshopper-loader's censusType. However, this is complicated by the fact that not all address numbers are just numeric. Some address ranges have letters and other characters like
-
as well.There are probably a few approaches for fixing this, but they'll most likely involve changes to both the loader and
CensusGeocoder
.FROM
values will always be less than theTO
values . Unfortunately, that is not the case as seen in the example below.