Is it possible to use CDQ to perform a query that uses a block?

[matthewsinclair]

I have a situation where I have a list of Stores and each Store has a StoreLocation which is just a lat/lon pair. For simplicity, just consider that each Store has a lat and a lon element.

I want to filter the set of Stores to just those that a "nearby", where nearby means that the result of the Great Circle calculation is less than some value. Note: I know how to do all of the geo stuff, so this bit is irrelevant to my CDQ question, but it's useful background.

I already have a table view controller that is displaying all stores using CDQ and a fetched results controller, but now I want to add in the ability to filter the list down based on proximity.

Here's the method that the TVC uses to get a fetched results controller:

  def fetch_controller
    @entities ||= Store.all.sort_by(:name)
    @fetch_controller ||= NSFetchedResultsController.alloc.initWithFetchRequest(
      @entities.fetch_request,
      managedObjectContext: @entities.context,
      sectionNameKeyPath: nil,
      cacheName: nil
    )
  end

What I want to know is, how can use a CDQ query to make the @entities ||= ... line filter based on application of the Great Circle algorithm? This is what I tried:

    def fetch_controller
      return super if !@nearby
      @entities ||= begin
        cur_loc = App.delegate.session.shopper_location.current_location
        Store.all_nearby_to(cur_loc.coordinate.latitude, cur_loc.coordinate.longitude)
      end
      @fetch_controller ||= NSFetchedResultsController.alloc.initWithFetchRequest(
        @entities.fetch_request,
        managedObjectContext: @entities.context,
        sectionNameKeyPath: nil,
        cacheName: nil
      )
    end

And here's the impl of the all_nearby_to method in the Store CDO:

  def self.all_nearby_to(lat, lon)
    my_loc  = MyApp::GreatCircleLocationImpl.new(lat: lat, lon: lon)
    Store.all.sort_by(:store_location).find_all do |store|
      if store.store_location && store.store_location.lat && store.store_location.lon
        other_loc = MyApp::GreatCircleLocationImpl.new(lat: store.store_location.lat, lon: store.store_location.lat)
        my_loc.nearby?(other_loc)
      end
    end
  end

This filters the CDOs perfectly. However, the problem is that it returns an array, and not whatever it is that CDO returns from Store.all or any version of a CDQ query applied to a CDO entity.

What I think I want to be able to do is the inject a block into a query so that I can apply my algorithm over the set of entities returned from Store.all. But I could have this totally wrong.

Any ideas would be much appreciated. Thanks, M@

[infinitered]

Yeah, so this is a tricky case.  Once you start using regular code to filter, you’ve popped out fetch request land.  There is an NSPredicate that takes a block and applies it as a filter in your fetch request… but it specifically does not work with Core Data when you’re using the SQLite store, which is all CDQ currently supports.  If your dataset is small and fixed, you might consider using XML store instead, but you’d have to set up your NSPersistentStoreCoordinator yourself (or modify CDQ to work with XML :).  

There may be a way to subclass NSFetchedResultsController to do some extra filtering post-fetch, as well.  

--  Ken Miller Co-Founder, InfiniteRed

On June 4, 2014 at 5:35:58 PM, Matthew Sinclair (notifications@github.com) wrote:

I have a situation where I have a list of Stores and each Store has a StoreLocation which is just a lat/lon pair. For simplicity, just consider that each Store has a lat and a lon element.

I want to filter the set of Stores to just those that a "nearby", where nearby means that the result of the Great Circle calculation is less than some value. Note: I know how to do all of the geo stuff, so this bit is irrelevant to my CDQ question, but it's useful background.

I already have a table view controller that is displaying all stores using CDQ and a fetched results controller, but now I want to add in the ability to filter the list down based on proximity.

Here's the method that the TVC uses to get a fetched results controller:

def fetch_controller @entities ||= Store.all.sort_by(:name) @fetch_controller ||= NSFetchedResultsController.alloc.initWithFetchRequest( @entities.fetch_request, managedObjectContext: @entities.context, sectionNameKeyPath: nil, cacheName: nil ) end What I want to know is, how can use a CDQ query to make the @entities ||= ... line filter based on application of the Great Circle algorithm? This is what I tried:

def fetch_controller
  return super if !@nearby
  @entities ||= begin
    cur_loc = App.delegate.session.shopper_location.current_location
    Store.all_nearby_to(cur_loc.coordinate.latitude, cur_loc.coordinate.longitude)
  end
  @fetch_controller ||= NSFetchedResultsController.alloc.initWithFetchRequest(
    @entities.fetch_request,
    managedObjectContext: @entities.context,
    sectionNameKeyPath: nil,
    cacheName: nil
  )
end

And here's the impl of the all_nearby_to method in the Store CDO:

def self.all_nearby_to(lat, lon) my_loc = MyApp::GreatCircleLocationImpl.new(lat: lat, lon: lon) Store.all.sort_by(:store_location).find_all do |store| if store.store_location && store.store_location.lat && store.store_location.lon other_loc = MyApp::GreatCircleLocationImpl.new(lat: store.store_location.lat, lon: store.store_location.lat) my_loc.nearby?(other_loc) end end end This filters the CDOs perfectly. However, the problem is that it returns an array, and not whatever it is that CDO returns from Store.all or any version of a CDQ query applied to a CDO entity.

What I think I want to be able to do is the inject a block into a query so that I can apply my algorithm over the set of entities returned from Store.all. But I could have this totally wrong.

Any ideas would be much appreciated. Thanks, M@

— Reply to this email directly or view it on GitHub.

[matthewsinclair]

Hmmm, that's no fun. But thanks for the reply anyway.

[infinitered]

Yeah, great circle calculations are tricky if you want to stick with SQL.  On a server side app ages ago I cheated and calculated a “great square” instead, but your use case might prevent such degradations.  Wish I had a better answer for you.  :(

--  Ken Miller Co-Founder, InfiniteRed

On June 4, 2014 at 5:52:26 PM, Matthew Sinclair (notifications@github.com) wrote:

Hmmm, that's no fun. But thanks for the reply anyway.

— Reply to this email directly or view it on GitHub.

[matthewsinclair]

How do people do live searching (or progressive search, or whatever it's called)? Wouldn't that need to take some extra input post query to filter the results?

Alternatively, is there a in_set kind of operation in the query syntax somewhere? For example, if I know the ids of the stores that are close, can I use that list to run the query?

For example:

nearby_store_ids = [1, 2, 3]
nearby_stores = Store.where(:store_id).in?(nearby_store_ids)
...

Or something like that?

[matthewsinclair]

Re: the GC calcs, I've got that worked out no probs, and I've even got a great optimisation for it that allows me to pre-calc half of the function in advance, so that the final nearby? is really, really fast. But the problem is injecting that calc into the query chain ...

[infinitered]

Live searching just updates the predicate in the fetch request and re-queries, afaik.  

--  Ken Miller Co-Founder, InfiniteRed

On June 4, 2014 at 5:55:37 PM, Matthew Sinclair (notifications@github.com) wrote:

How do people do live searching (or progressive search, or whatever it's called)? Wouldn't that need to take some extra input post query to filter the results?

Alternatively, is there a in_set kind of operation in the query syntax somewhere? For example, if I know the ids of the stores that are close, can I use that list to run the query?

For example:

nearby_store_ids = [1, 2, 3] nearby_stores = Store.where(:store_id).in?(nearby_store_ids) ... Or something like that?

— Reply to this email directly or view it on GitHub.

[matthewsinclair]

Ok, would it be possible to get the predicate that's in the NSFetchRequest that's created from Store.all.fetch_request and update it or change it somehow?

Or another question, what's going on when I call fetch_request on the value returned from something like Store.all? What about if I could manually construct an appropriate instance of NSFetchRequest? Could that work?

[infinitered]

Well, sure.  But without access to the block predicate, you’re limited in what you can do with that.  The predicates get translated into SQL, essentially.  

Your best bet is probably to subclass NSFetchedResultsController and override fetchedObjects and/or performFetch, since there you have the opportunity to run some code on the results of the real fetch.

--  Ken Miller Co-Founder, InfiniteRed

On June 4, 2014 at 6:00:54 PM, Matthew Sinclair (notifications@github.com) wrote:

Ok, would it be possible to get the predicate that's in the NSFetchRequest that's created from Store.all.fetch_request and update it or change it somehow?

Or another question, what's going on when I call fetch_request on the value returned from something like Store.all? What about if I could manually construct an appropriate instance of NSFetchRequest? Could that work?

— Reply to this email directly or view it on GitHub.

[matthewsinclair]

Ok, I'll give that a go. And there's no way to do an in set kind of operation anywhere?

[infinitered]

There is a way to do ‘in' queries yes:  Store.all.in([list])

--  Ken Miller Co-Founder, InfiniteRed

On June 4, 2014 at 6:08:51 PM, Matthew Sinclair (notifications@github.com) wrote:

Ok, I'll give that a go. And there's no way to do an in set kind of operation anywhere?

— Reply to this email directly or view it on GitHub.

[matthewsinclair]

Ok, that's awesome. I think that'll do what I want!