The Address Search API Returns No Result

[mzagaja]

When you are in the map view and enter street addresses, the address search API will fail to return a result. However it will respond to a town name. It sends a request to the API but it does not receive a result.

The source of this issue is that the client application hits the MassBuilds API and the MassBuilds API attempts to get information from the Mapzen API and then fails to receive a useful response from Mapzen.

To surmount this issue we need to make sure we receive a latitude and longitude.

[mzagaja]

Working on debugging this locally I think there might be some sort of validation of the "id" parameter somewhere before we hit searchable_resource.rb. I threw some Rails.logger.info statements which get hit when we send it a valid entry:

 curl 'http://api.lvh.me:5000/searchables/Cambridge' \                                                                                                 develop ✭ ✱
-XGET \
-H 'Referer: http://lvh.me:5000/developments/?placeSearch=Cambridge' \
-H 'Origin: http://lvh.me:5000' \
-H 'Accept: application/vnd.api+json' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/602.3.12 (KHTML, like Gecko) Version/10.0.2 Safari/602.3.12' \
-H 'X-CSRF-Token: pjNhkGqRU5HJ3wKqkPPNzsRlLA4kakJUvKdhFSymSD8YMvwtogkKJ0cAOEv3QWLedKHwPnwIg+43p+8C2YN6jw=='
{"data":[{"id":"2","type":"places","links":{"self":"http://api.lvh.me:5000/places/2"},"attributes":{"name":"Cambridge","place-type":"Municipality","geometry":null,"neighborhood-ids":[]}}]}%

Started GET "/searchables/Cambridge" for 127.0.0.1 at 2016-12-27 16:53:12 -0500
Processing by API::V1::SearchablesController#show as API_JSON
  Parameters: {"id"=>"Cambridge"}
About to call PgSearch
  PgSearch::Document Load (5.2ms)  SELECT  "pg_search_documents".* FROM "pg_search_documents" INNER JOIN (SELECT "pg_search_documents"."id" AS pg_search_id, (ts_rank((to_tsvector('simple', coalesce("pg_search_documents"."content"::text, ''))), (to_tsquery('simple', ''' ' || 'Cambridge' || ' ''' || ':*')), 0)) AS rank FROM "pg_search_documents" WHERE (((to_tsvector('simple', coalesce("pg_search_documents"."content"::text, ''))) @@ (to_tsquery('simple', ''' ' || 'Cambridge' || ' ''' || ':*'))))) AS pg_search_ce9b9dd18c5c0023f2116f ON "pg_search_documents"."id" = pg_search_ce9b9dd18c5c0023f2116f.pg_search_id WHERE "pg_search_documents"."searchable_type" = $1  ORDER BY pg_search_ce9b9dd18c5c0023f2116f.rank DESC, "pg_search_documents"."id" ASC LIMIT 5  [["searchable_type", "Development"]]
About to call MapzenSearch
Here are the results!
{"geocoding":{"version":"0.2","attribution":"https://search.mapzen.com/v1/attribution","query":{"text":"\"Cambridge","tokens":["\"Cambridge"],"size":10,"sources":["openstreetmap"],"private":false,"focus.point.lat":42.357,"focus.point.lon":-71.056},"errors":["'macrohood\"' is an invalid layers parameter. Valid options: coarse,address,venue,street,country,macroregion,region,county,localadmin,locality,borough,neighbourhood,continent,dependency,macrocounty,macrohood,microhood,disputed"],"engine":{"name":"Pelias","author":"Mapzen","version":"1.0"},"timestamp":1482875592120},"type":"FeatureCollection","features":[]}
Cambridge
  Place Load (0.9ms)  SELECT  "places".* FROM "places" WHERE (name ILIKE 'Cambridge%')  ORDER BY
      CASE
        WHEN type = 'Municipality' THEN '1'
        WHEN type = 'Neighborhood' THEN '2'
        ELSE '3'
      END
     LIMIT 1
   (0.3ms)  SELECT "places"."id" FROM "places" WHERE "places"."place_id" = $1  ORDER BY
      CASE
        WHEN type = 'Neighborhood' THEN '1'
        WHEN type = 'Municipality' THEN '2'
        ELSE '3'
      END
      [["place_id", 2]]
Completed 200 OK in 192ms (Views: 0.5ms | ActiveRecord: 9.7ms)

But we do not hit these methods when we send the street address:

curl 'http://api.lvh.me:5000/searchables/9%20Banks%20St,%20Cambridge,%20Ma' \                                                                         develop ✭ ✱
-XGET \
-H 'Referer: http://lvh.me:5000/developments/?placeSearch=9%20Banks%20St%2C%20Cambridge%2C%20Ma' \
-H 'Origin: http://lvh.me:5000' \
-H 'Accept: application/vnd.api+json' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/602.3.12 (KHTML, like Gecko) Version/10.0.2 Safari/602.3.12' \
-H 'X-CSRF-Token: d3vfZ5x9APEsdLgV8UcPwUn1kjmiJjGfFoerqsVk+z/JekLaVOVZR6KrgvSW9aDR+TFOCfpE8CWdhyW9MEHJjw=='
{"errors":[{"title":"Invalid field value","detail":"9 Banks St, Cambridge, Ma is not a valid value for id.","id":null,"href":null,"code":103,"source":null,"links":null,"status":"400","meta":null}]}%

Started GET "/searchables/9%20Banks%20St,%20Cambridge,%20Ma" for 127.0.0.1 at 2016-12-27 16:52:27 -0500
Processing by API::V1::SearchablesController#show as API_JSON
  Parameters: {"id"=>"9 Banks St, Cambridge, Ma"}
Completed 400 Bad Request in 4ms (Views: 0.3ms | ActiveRecord: 0.0ms)

Typically I'd expect a relevant method in searchables_controller.rb to ingest the HTTP request (usually show) and then I'd be able to follow it's execution along the codebase. Unfortunately for me it merely inherits from APIController which inherits from JSONAPI::ResourceController which I am unfamiliar with and is thus engaging in some currently not understood magic. It may simply be a matter of sipping some coffee and learning this gem. However it'd be easier if I could find a hint on where in the codebase things start firing.

[mzagaja]

So it turns out that we're hitting the show method in the searchable controller. But it's just inheriting which we can see if I update searchables_controller.rb to contain the following:

module API
  module V1
    class SearchablesController < APIController
      def show
        Rails.logger.info "What's up from Searchables Controller"
        super
      end
    end
  end
end

And hit it with a curl request. The genesis of this error message can be found in the jsonapi-resources gem codebase by an enterprising code archaeologist, giving us hope that we've finally found the source of this issue. Sadly we get our hopes up too soon, as even though we've found how the gem gives us our error message, we do not understand why other than it is complaining of an Invalid Field Value. The next logical question is: who defines what is a valid field value and where? More importantly why are we submitting a street address or city name as an "id" parameter? This defies intuition! id parameters are usually integers.

[mzagaja]

I managed to get the pry-debugger working using some manual launching bundle exec puma -C config/puma.rb -p 5000. I have ascertained that the difference between a working and non-working request is about hitting https://github.com/MAPC/massbuilds/blob/develop/app/resources/api/v1/searchable_resource.rb#L7 where a successful request goes into this world of abstract and returns something useful to the user, where an unsuccessful request comes back into the land of searchable_resource.rb and ends up returning a 400 error. It is not clear or explained why along the way.