Closed blackmad closed 3 months ago
this change seems broken. will keep looking.
[{phrase: "saunders st rego park", query: "6307", id: 420782897, name: "Rego Park",…}] 0: {phrase: "saunders st rego park", query: "6307", id: 420782897, name: "Rego Park",…} geom: {area: 0.000259, bbox: "-73.871429,40.711338,-73.85173,40.73627346", lat: 40.727122, lon: -73.862152} id: 420782897 languageDefaulted: false lineage: [{borough: {id: 421205767, name: "Queens", languageDefaulted: false},…}] name: "Rego Park" phrase: "saunders st rego park" placetype: "neighbourhood" query: "6307"
this seems happier now
Actually, looking back at this work, it's still an interesting base to play with. tokenize2 could probably be called tokenizeStructured or something else smarter. @missinglink equally happy to abandon this or keep working on it, what do you think?
This change is in preparation for experimenting with placeholder doing more of the admin splitting work that parser & libpostal currently do. It adds a "query" to the placeholder responses which is the left most tokens in the query that are not matched by placeholder.
It adds a /tokenize2 endpoint (which I think is for debugging only?) to expose this it adds "query" to the results from /search
internally, it primarily adds a remainder: {before: string, after: string} to the "group" object in the code.
It is a tiny bit based off of this stale PR: https://github.com/pelias/placeholder/pull/37/
demo frontend updated