Implement queries - Githubissues

nolanlawson commented 10 years ago

Got this request from a user, but I have no idea how to implement it.

I'm currently building a node-webkit application using Ember and Ember-Pouch. It works great to retrieve my model and to add data in the IndexedDB.

When it comes to querying though, I'm getting errors I'm not sure where it comes from.

When I retrive the data, I call this.get('store').find('client'); in the route model. It gets all the records fine. However, if I add details, say this.get('store').find('client', { first_name: 'George' });, it always return an empty set even though George exists.

When I want to query for a specific client from the controller, for instance this.get('store').find('client', { phone: phone_num });, it logs a GET app://cbt/clients?phone=5555555555 net::ERR_FILE_NOT_FOUND. Why is it searching for 'clients' plural while everywhere in my models it's 'client' singular?

I'm not very experienced with Ember-Data yet and I was wondering if my issues where with the Adapter or with Ember-Data.

nolanlawson commented 10 years ago

Examples:

rsutphin commented 10 years ago

In the meantime, maybe it would be nice if the adapter overrode findQuery to throw an exception or log an error when you try to do a query like this? The seemingly random XHRs are confusing.

nolanlawson commented 10 years ago

Thanks a bunch, your fix is now in v1.1.1!

srsgores commented 9 years ago

Just experienced this same behaviour. Ember data was making the request localhost:4000/blog?slug=another-test-post. See stackoverflow.

In the meantime, what is the recommended approach to doing queries?

nolanlawson commented 9 years ago

Unfortunately you'll just have to not use queries until someone implements this. :(

srsgores commented 9 years ago

@nolanlawson, is there any update on this? Can you describe the process of implementing this feature, or any relevant resources? I may be willing to help out.

nolanlawson commented 9 years ago

@srgores I think the wisest course of action would be to wait until pouchdb-find is implemented ("new" query language, more intuitive and easy to use than the current map reduce API): https://github.com/nolanlawson/pouchdb-find

srsgores commented 9 years ago

@nolanlawson, I was thinking we could use views in the interim. Perhaps, the user could configure the URL to be used which should correspond to the _design document.

Edit: for example, to retrieve a Post with a given slug property, I might configure my CouchDB instance with a (map) view to filter out the correct post. The request URL would look something like this:

https://seangoresht.iriscouch.com/seangoresht/_design/slug/_view/bySlug?key=%22other-post%22&include_docs=true

And the result:

{"total_rows":4,"offset":3,"rows":[
{"id":"post_2_C5F9F82D-8A2B-2E09-9E2F-5CCE883E6352","key":"other-post","value":null,"doc":{"_id":"post_2_C5F9F82D-8A2B-2E09-9E2F-5CCE883E6352","_rev":"2-93c7869edf40647d4fa1ceb77496e4d6","data":{"title":"Other post","slug":"other-post","body":"# hey there","createdDate":"2014-12-27T12:13:50.408Z","lastModified":"2014-12-27T12:17:09.487Z","isPublished":false,"author":"AFBC04DA-58BF-6F42-A48B-C075BB5E2C03","tags":[],"categories":["F2FAAD60-D011-ECA1-8BF1-13ACAEDFB411","9B0D76D4-9215-0D6E-8CEF-6A8A07888C1F"]}}}
]}

Thus, you might configure the adapter to retrieve data with the following pattern:

https://couchserver.com/_design/{modelName}/_view/{modelProperty}?key={query}

This way, the user could be in control of what each query returns.

@nolanlawson, what are your thoughts?

srsgores commented 9 years ago

@nolanlawson. what is the status on this? I am very eager to assist, if you can point me in the right direction.

nolanlawson commented 9 years ago

We are developing pouchdb-find pretty quickly, so I don't think there's much reason to use map/reduce.

To be precise, I released a beta version of pouchdb-find last weekend: http://nolanlawson.github.io/pouchdb-find/

Based on some recommendations from this thread, I'd say that the query() method in ember-pouch should just directly pass in the Mango-style JSON query. Ember Data has no spec for this, so the more 1-to-1 the correspondence between App.MyModel.query() and pouchdb.find(), the better. So something like:

App.MyModel.query({selector: { name: "tom" }, sort: ["name"]})

So ideally, if you would like to implement this, you could just include pouchdb-find and then basically write glue code. :)

srsgores commented 9 years ago

@nolanlawson, thanks for clarifying. I still don't quite understand what is required to get this functioning, so in the mean time, I am trying to configure my own model to return from the URL structure mentioned above: https://seangoresht.iriscouch.com/seangoresht/_design/slug/_view/bySlug?key=%22other-post%22&include_docs=true

I try setting up my model method to be overridden to return the results from the request, but I understand that there's a lot more involved in parsing the data (indeed from the data attribute in the response):

 import Ember from "ember";
  import ajax from "ic-ajax";
  import config from "../../../config/environment";
  var PostRoute;

  PostRoute = Ember.Route.extend({
    serialize: function(model, params) {
      return {
        post_slug: model.get("slug")
      };
    },
    model: function(params) {
      return ajax({
        type: "GET",
        url: "" + config.db.url + "/_design/slug/_view/bySlug?key=\"" + params.post_slug + "\"&include_docs=true"
      });
    }
  });

  export default PostRoute;

@nolanlawson, can you suggest an approach to customize this adapter to suit my needs? In the meantime, I will continue researching.

stickbyatlas commented 9 years ago

Hi folks,

I stumbled across this issue because I tried to invoke DS.find() in my Ember-CLI app and it pointed me here. So, thanks @rsutphin for including that notification!

I'm wondering if anybody on this thread has figured out the best solution to tackle Mango-style query mapping to the DS methods. I'm building a non-syncing app, so I have no recourse to query a remote server. All I'm trying to do is retrieve a list of records of a certain type, whose boolean flag is set to false, so I can delete them from the database.

I imagine it would be more performant to do this directly using DS.find. However, does it make sense to do DS.filter('type'), then programmatically delete every returned result in the array? Or is there a better way?

Thanks for all the great work on PouchDB and its related libraries. So glad to see the CouchDB / PouchDB ecosystem thriving.

nolanlawson commented 9 years ago

I'm less involved with the Ember side of things these days, so I'm hoping somebody will pick up the slack. :)

pouchdb-find is still in beta, but it's definitely robust enough to handle the use-case you're talking about. I'm probably going to slap a 1.0 sticker on it within the next month, so anybody who wants to start tying it to the Ember way of doing things would be more than welcome.

nolanlawson commented 9 years ago

Update: pouchdb-find is definitely stable enough for anyone who wants to implement this. :) https://github.com/nolanlawson/pouchdb-find

leojpod commented 9 years ago

Hi @nolanlawson

I'ld like to work on that. Should it be done in this addon or within https://github.com/nolanlawson/relational-pouch/ ?

nolanlawson commented 9 years ago

@leojpod I think it should start with a PR on relational-pouch.

stickbyatlas commented 9 years ago

Hey @leojpod , just wondering if you've made any headway with this issue? I don't have an immediate need for it, but it's something I can anticipate needing to use on a project that's coming up. Would hate to duplicate effort if you've already made a start.

leojpod commented 9 years ago

@stickbyatlas no I haven't had time to make anything good. I had just made a temporary fix that was basically pulling all the data of a given type and filtering it before returning it to the ember app. You can find this temp fix here https://github.com/leojpod/ember-pouch/tree/implement-queries-7

I haven't had time to work on any ember project lately (it sucks but well... ) so this temp fix is behind the current version.

pjcarly commented 8 years ago

@stickbyatlas @leojpod same here, if you have a repo up somewhere make sure to communicate here, i'll be needing this functionality later this year and wouldn't want to duplicate work either.

leojpod commented 8 years ago

@pjcarly @stickbyatlas no not yet sorry, I am merely getting back to working on some ember project now. Hopefully I'll have some more time in a couple of months but nothing for sure. I managed to make do with the temp fix for now..

nicozimmermann94 commented 8 years ago

So to bump this a little, what is the actual status on this? As far as I understand is the only solution right now to use the fix from @leojpod (which would be slow for huge amounts of data right?) What about pouch-find?

pjcarly commented 8 years ago

Haven't made any progress, would probably start somewhere at the start of July with the project that requires this.

dweremeichik commented 8 years ago

For those who need this in a pinch (while waiting on the actual fix), you can hack a "work around" together with the following: (Disclaimer: this method is not overly efficient - you have been warned).

  model(){
    return new Ember.RSVP.Promise((resolve) => {
      //Grab all of the items.
      this.store.findAll('item').then((items) => {
        //Get a subset of items that have a type property set to 'food'
        resolve(items.filterBy('type', 'food'));
      });
    });
  }

This works best when you know you will only have a few records that get returned.

knownasilya commented 8 years ago

Any movement on this? cc @locks @bmac

dweremeichik commented 8 years ago

@knownasilya query and queryRecord functionality was merged in as of Sept 4th. Those replace the findQuery that this issue was originally opened on.

knownasilya commented 8 years ago

I tried return this.store.queryRecord('guide', { filter: { version, section, guide } }); but it doesn't seem to work, any suggestions? See https://github.com/ember-learn/guias/blob/em-pouch/app/routes/guide.js#L8 for code

dweremeichik commented 8 years ago

I haven't had the chance to use it yet, the tests that were in the commit I linked above should be helpful (perhaps).

What exactly are you trying to filter on? Based on the tests, your filter looks a little different.

Edit: see pouchdb-find for example usage.

knownasilya commented 8 years ago

I'm doing it that way because ember-pouch requires it, see https://github.com/nolanlawson/ember-pouch/blob/master/addon/adapters/pouch.js#L292

Basically I want to filter using an and on three fields.

dweremeichik commented 8 years ago

@knownasilya I became curious and went down quite the rabbits hole here, so bear with me.

Update your ember-pouch, it is too old for the functionality you are looking for. Run ember install ember-pouch it will grab all of the new libraries as well.
You need to call createIndex before you call query or queryRecord. See the following:

  model({ version, section, guide }){
    return this.get('store').adapterFor('application').db.createIndex({ index: {
        fields: ['version', 'section', 'guide'] }
      }).then(() => {
      return this.get('store').queryRecord('guide',{
        filter: {version: version, section: section, guide: guide},
        sort: ['version']});
    });
  }

Your database structure is invalid for ember-pouch. Everything excluding _id and _rev need to be wrapped in a data object like so:

{
  "_id": "07c7bed0d16968d166a6343deb6ac315",
  "data": {
    "type": "guide",
    "version": "v1.10.0",
    "section": "components",
    "guide": "customizing-a-components-element",
    "title": "components",
    "language": "en-US",
    "html": "<div id=\"content\" class=\"has-sidebar\">\n ....."
  },
  "_rev": "4-75a59af65905983289f34af7b4f7cb77"
}

Also note that you are not leveraging the ember-pouch/relational-pouch "type" of ids, such as guide_2_07c7bed0d16968d166a6343deb6ac315 which gives you the type attribute for "free".

@broerse @fsmanuel I ran into a few surprises with this:

I used @nolanlawson's awesome resource for learning pouchdb-find. In that and other examples, selector is used everywhere, where as filter is not. We should consider creating an alias so both work. I understand the reasoning behind making it filter, however query is the one major place where ember takes a hands off approach, leaving this more or less to pouch.
We have no documentation for the query queryRecord features; we don't even point people in the right direction, which is along the lines of why I created #149 .
Calling createIndex() the way I did above feels a little foreign to have to do, would it be a bad idea to have the adaptor call that on the users behalf somehow?

BernardTolosajr commented 8 years ago

@dweremeichik the way i called the createIndex() is in the adapters/application.js inside the createDb()

dweremeichik commented 8 years ago

@bernardtolosajr okay, that makes sense.

jlami commented 8 years ago

I see two problems with the current implementation of query.

The document returned is not parsed the same way that relational-pouch does. So any relations are ignored.
Querying an attribute actually finds multiple model types.

In my PR #159 I have changed the parsing of the find to go through relational-pouch to fix point 1. Point 2 would require some more work, but could be done in the same way as I do the findHasMany in the corresponding PR https://github.com/nolanlawson/relational-pouch/pull/61

gr2m commented 7 years ago

can this issue be closed? I feel like the original issue was addressed, maybe open follow ups for known issues?

broerse commented 7 years ago

@gr2m I think we can close this.

pouchdb-community / ember-pouch

Implement queries #7