mosuka / blast

Blast is a full text search and indexing server, written in Go, built on top of Bleve.
Apache License 2.0
1.08k stars 76 forks source link

GEO searches #60

Closed hubyhuby closed 5 years ago

hubyhuby commented 5 years ago

Is it possible to use blast for Geo localized searches ? It is nowadays an important feature in many projects. If yes would it be possible to include an example at one point ? ( mapping, and search request)

I have put a very basic example of Geo search with bleve there : https://github.com/hubyhuby/bleve-search-example/blob/444d18d810064302fef76693f86f07c533552897/main.go#L138

There are two main search I see : Geo box search around a point and Geo Sort search by distance to a point.

I understand it is a WIP, but I am really looking forward to this project. Thanks for this nice project !

mosuka commented 5 years ago

Hi @hubyhuby ,

Unfortunately I haven't tried GEO Search yet. However, Blast supports Bleve's SearchRequest. But it takes a lot of work. Please marshal Bleve's SearchRequest into JSON (or scratch JSON based on Bleve's SearchRequest) and search it as a Blast search request.

This is an example of a Blast search request. https://github.com/mosuka/blast/blob/master/example/search_request.json The above is JSON which marshalled Bleve's SearchRequest.

Please try this method :)

Since this method is very troublesome, I will consider preparing a query builder in the future.

ghost commented 5 years ago

hey @hubyhuby

For Geo maybe use this golang db designed exactly for it: https://github.com/tidwall/tile38

My thinking is that you can run blast and tile38, but you just need a watch system somewhere and so when a mutation happen in the tile38 system, you send a event (with a payload) to Blast, and it will index it for you.

Now maybe this does not suit your use case. Hard to know.

Its also safer approach long term, because you can add more specialised databases as you need to into the "chain". Just like we add libraries to our stack to do things with functions that call them !

What to use to do the pipeline chaining between these databases ? https://github.com/benthosdev

Hope this is not too abstract an answer ?

@mosuka If you do add a Geo Search ( github.com/blevesearch/bleve/mapping) it would be pretty cool. I needed geo and so used tile38 for now and i got the job done.

I suspect it may be worthwhile looking at tile38 a bit as there is probably lots of good lessons learn in there about Geo search too.

hubyhuby commented 5 years ago

Dear @mosuka , Thank you for your answer. For now I have not used blast. I am thinking of using it as a faster easier way to use bleve search. I am not sure how to "marshal Bleve's SearchRequest into JSON" . Could you explain more ? For example how do I "marshall" this query : ` lon, lat := -122.107799, 37.399285

//distance query

distanceQuery := bleve.NewGeoDistanceQuery(lon, lat, "100000mi")

distanceQuery.SetField("geo")`

Not sure either how can I marshal to JSON the geo mapping :
`locationMapping := bleve.NewGeoPointFieldMapping()

locationMapping.IncludeTermVectors = true

locationMapping.IncludeInAll = true

locationMapping.Index = true

locationMapping.Store = true

locationMapping.Type = "geopoint"

breweryMapping.AddFieldMappingsAt("geo", locationMapping)`

@gedw99 , Thanks for your help. This seems a little complex for me. Today using bleve directly inside my go app is easier for me. The bleve library already has basic geo search. My question is more how could we expose bleve geo feature in blast. The approach of micro-service you describe is nice, but it s getting more complex that needed for my projects at least.

ghost commented 5 years ago

no worries, and I agree with you too that more microservices is just more pain.

If blast does enough its awesome

mosuka commented 5 years ago

Hi @hubyhuby @gedw99 , Sorry for the late reply. I added a GEO search example. https://github.com/mosuka/blast/blob/master/README.md#spatialgeospatial-search-example Could you try this out?