search

getoutreach / epf

A framework for keeping your Ember.js apps in sync.
http://epf.io
MIT License
369 stars 33 forks source link

Embedded Update Error #141

Open dev-inigmas opened 10 years ago

dev-inigmas commented 10 years ago

After echoing back the "client_id" that was provided by EPF for the parent document, I discovered a non-deterministic error that occurs on updates. (Haven't been able to replicate it for create yet, but that doesn't mean it can't happen...)

I'm assuming that the problem would resolve itself if I included the client_id for every embedded document as well... I'm just reporting this here in hopes that we can work toward a version where client_id isn't required from the server.

Here's the error:

TypeError: Array.prototype.indexOf called on null or undefined

Adding several sub-documents to the parent document seems to increase how often this occurs. So far, I haven't been able to duplicate this when removing many sub-documents.

ghempton commented 10 years ago

Hmm that seems strange. We have several models that use lots of embedded documents. Do you have more of the stack trace handy?

dev-inigmas commented 10 years ago

That's all the reason(?) code said from the session.flush(). I'm guessing that your servers pass-through all of the client_ids, though. Right?

Right now, I'm just doing the ids for the parent documents.

Also, do your models with lots of embedded documents have more that one type of embedded document? I noticed that I'm getting this the most with my "Edge" documents. They have lists of "EdgeRequirements" and "EdgeEffects".

ghempton commented 10 years ago

Yes they have lots of nested embedded relationships. There are also some acceptance tests with multiple levels of embedded relationships.

On Sat, Jul 12, 2014 at 2:46 PM, David Patrick notifications@github.com wrote:

That's all the reason(?) code said from the session.flush(). I'm guessing that your servers pass-through all of the client_ids, though. Right?

Right now, I'm just doing the ids for the parent documents.

Also, do you models with lots of embedded documents have more that one type of embedded document? I noticed that I'm getting this the most with my "Edge" document. It has list of "EdgeRequirements" and "EdgeEffects".

— Reply to this email directly or view it on GitHub https://github.com/getoutreach/epf/issues/141#issuecomment-48824941.

Gordon L. Hempton http://codebrief.com 360.460.8098

dev-inigmas commented 10 years ago

Alright; I setup my server to "pass-through" all of the client ids, revs, etc.... But I still occasionally get that error. Just now, I actually got it while removing a lot of documents. I'm going to see if I can get a stack trace on it. Until then, I'll post my latest request/response/hash.

PUT /api/v1/edges/53c1d7bf97000054aa57aa41:

{
  "edge": {
    "id": "53c1d7bf97000054aa57aa41",
    "client_id": "edge540",
    "client_rev": 2,
    "name": "qwe",
    "rank": "nov",
    "for_ab": false,
    "description": "qwe",
    "module_id": "53c050319700005e2157aa20",
    "reqs": [{
      "id": "edge_req-53c1d7bf97000054aa57aa41-0",
      "client_id": "edge_req541",
      "client_rev": 1,
      "req": "qwe",
      "script": null,
      "edge_id": "53c1d7bf97000054aa57aa41"
    }],
    "effects": [{
      "id": "edge_effect-53c1d7bf97000054aa57aa41-0",
      "client_id": "edge_effect542",
      "client_rev": 1,
      "effect": "qwe",
      "script": null,
      "edge_id": "53c1d7bf97000054aa57aa41"
    }]
  }
}

Response 200:

{
  "edge": {
    "doc_meta": {
      "created": 1405212607575,
      "updated": 1405213787685,
      "created_by": "53bd768f970000004057a9f4"
    },
    "id": "53c1d7bf97000054aa57aa41",
    "name": "qwe",
    "rank": "nov",
    "for_ab": false,
    "effects": [{
      "id": "edge_effect-53c1d7bf97000054aa57aa41-0",
      "client_id": "edge_effect542",
      "client_rev": 1,
      "effect": "qwe",
      "script": null,
      "edge_id": "53c1d7bf97000054aa57aa41"
    }],
    "reqs": [{
      "id": "edge_req-53c1d7bf97000054aa57aa41-0",
      "client_id": "edge_req541",
      "client_rev": 1,
      "req": "qwe",
      "script": null,
      "edge_id": "53c1d7bf97000054aa57aa41"
    }],
    "description": "qwe",
    "module_id": "53c050319700005e2157aa20",
    "client_id": "edge540",
    "client_rev": 2
  }
}

Hash after extractProperty():

{
  "doc_meta": {
    "created": 1405212607575,
    "updated": 1405213787685,
    "created_by": "53bd768f970000004057a9f4"
  },
  "id": "53c1d7bf97000054aa57aa41",
  "name": "qwe",
  "rank": "nov",
  "for_ab": false,
  "effects": [{
    "id": "edge_effect-53c1d7bf97000054aa57aa41-0",
    "client_id": "edge_effect542",
    "client_rev": 1,
    "effect": "qwe",
    "script": null,
    "edge_id": "53c1d7bf97000054aa57aa41"
  }],
  "reqs": [{
    "id": "edge_req-53c1d7bf97000054aa57aa41-0",
    "client_id": "edge_req541",
    "client_rev": 1,
    "req": "qwe",
    "script": null,
    "edge_id": "53c1d7bf97000054aa57aa41"
  }],
  "description": "qwe",
  "module_id": "53c050319700005e2157aa20",
  "client_id": "edge540",
  "client_rev": 2
}
dev-inigmas commented 10 years ago

I removed my error handler from the session.flush().then() call. Then, after several edits, I got this:

TypeError: undefined is not a function
    at Ep.Session.reopen.merge (http://bebop.local:9494/dist/deps.js:71930:23)
    at Ep.ChildSession.Ep.Session.extend.merge (http://bebop.local:9494/dist/deps.js:71866:47)
    at apply (http://bebop.local:9494/dist/deps.js:31391:27)
    at superWrapper [as merge] (http://bebop.local:9494/dist/deps.js:30969:15)
    at http://bebop.local:9494/dist/deps.js:70533:40
    at Array.map (native)
    at http://bebop.local:9494/dist/deps.js:70532:31
    at invokeCallback (http://bebop.local:9494/dist/deps.js:33957:19)
    at publish (http://bebop.local:9494/dist/deps.js:33627:9)
    at publishRejection (http://bebop.local:9494/dist/deps.js:34055:7) 

Uncaught Error: Assertion Failed: TypeError: undefined is not a function deps.js:23517
Ember.assert deps.js:23517
RSVP.onerrorDefault deps.js:70077
__exports__.default.trigger deps.js:32921
Promise._onerror deps.js:33645
publishRejection

Any ideas?

dev-inigmas commented 10 years ago

Found the source and line number for you, in case that helps:

lib/session/merge.js:49

dev-inigmas commented 10 years ago

So.... I'm just about done converting my app over to EPF; but this remains a blocking issue. Please let me know if there's anything I can provide that would help us to find a fix/workaround.

Thanks.

ghempton commented 10 years ago

Hmm I would like to get to the bottom of this. You said that this happens after delete operations, what do the responses look like for those? Seems weird that model would be null inside of session.merge unless something weird was being sent down from the serializer.d

On Wed, Jul 16, 2014 at 12:15 PM, David Patrick notifications@github.com wrote:

So.... I'm just about done converting my app over to EPF; but this remains a blocking issue. Please let me know if there's anything I can provide that would help us to find a fix/workaround.

Thanks.

— Reply to this email directly or view it on GitHub https://github.com/getoutreach/epf/issues/141#issuecomment-49212897.

Gordon L. Hempton http://codebrief.com 360.460.8098

dev-inigmas commented 10 years ago

In the original ticket, I thought that this only happened after adding many documents. Lately, I've just been getting this any time there's a change to a list. My long comment from four days ago shows the request/response/afterDeserialization.

ghempton commented 10 years ago

What do you mean a change to a list?

dev-inigmas commented 10 years ago

I think that the following code examples should illustrate what I mean. In the controller, I sometimes create the embedded models right there and add them. For cases where items are being removed from a list, I'll use removeObject.

Models:

App.Foo = Ep.Model.extend
  bars: Ep.hasMany('bar')

App.Bar = Ep.Model.extend
  foo: Ep.belongsTo 'foo', inverse: 'bars'

App.FooSerializer = App.MyCrazySerializer.extend
  bars: embedded: 'always', inverse: 'foo'

Controller:

App.FooController = Em.ObjectController.extend
  actions:
    addBar: (barData) ->
      foo = @get('content')
      session = foo.get('session')
      bar = session.create 'bar', barData

      foo.get('bars').pushObject(bar)

Route:

App.FooRoute = Em.Route.extend
  actions:
    save: ->
      @get('currentModel.session').flush().then
        ()  => @send('success')
        (r) => @send('failure', r)
ghempton commented 10 years ago

Can you try:

foo = @get('content')
      session = foo.get('session')
      bar = session.create 'bar', barData

      foo.get('bars').pushObject(bar)
      bar.set('foo', foo) #added this line

I'm thinking there might be an issue updating the inverse for some reason.

dev-inigmas commented 10 years ago

Doesn't seem to help.

So, in another place in my app I'm able to replicate this error much more easily. Note that I only ever edit documents within a child session.

  1. Start with an empty child array in the primary document.
  2. Add an embedded document and flush()
  3. Remove the embedded document then add a new one.
  4. flush() again.

My current theory about this, is that the parent session is being told:

I also think that I may be getting this error more deterministically from this section in my app, because the embedded child has a relational Ep.belongsTo relationship defined within it.

ghempton commented 10 years ago

Ah so you are re-adding a child with the same id? I think semantically there are parts of EPF that assume that the removal of an embedded child means that the child is deleted.

On Wed, Jul 16, 2014 at 3:44 PM, David Patrick notifications@github.com wrote:

Doesn't seem to help.

So, in another place in my app I'm able to replicate this error much more easily. Note that I only ever edit documents within a child session.

  1. Start with an empty child array in the primary document.
  2. Add an embedded document and flush()
  3. Remove the embedded document then add a new one.
  4. flush() again.

My current theory about this, is that the parent session is being told:

  • Remove the child document whose id was XXX
  • Add a new child document whose id is XXX, where XXX is the same as the old child document's id.

I also think that I may be getting this error more deterministically from this section in my app, because the embedded child has a relational Ep.belongsTo relationship defined within it.

— Reply to this email directly or view it on GitHub https://github.com/getoutreach/epf/issues/141#issuecomment-49237904.

Gordon L. Hempton http://codebrief.com 360.460.8098

dev-inigmas commented 10 years ago

Yes. Removing a Child should delete it. But I've been generating IDs for my embedded objects. These generated id's all follow a similar pattern. Removing an embedded document, then adding a new embedded document caused the new document to have the same ID as the old one.

I'm working on changing that now to see if that resolves things.

ghempton commented 10 years ago

That makes sense. I hope this resolves it as well.

On Wed, Jul 16, 2014 at 3:59 PM, David Patrick notifications@github.com wrote:

Yes. Removing a Child should delete it. But I've been generating IDs for my embedded objects. These generated id's all follow a similar pattern. Removing an embedded document, then adding a new embedded document caused the new document to have the same ID as the old one.

I'm working on changing that now to see if that resolves things.

— Reply to this email directly or view it on GitHub https://github.com/getoutreach/epf/issues/141#issuecomment-49239091.

Gordon L. Hempton http://codebrief.com 360.460.8098

dev-inigmas commented 10 years ago

Hmm..... At this point, I'm not seeing any way around having to update every single sub-document in my database to have it's own id.

I was really hoping to avoid needing to do that.

ghempton commented 10 years ago

Yeah unfortunately I think that is going to be required for the short term. This could be avoided in the future with additional logic on EPF's end. In our app we have unique id's on our embedded sub-documents.

dev-inigmas commented 10 years ago

Well... I'm storing the embedded document ids for one of my Models now. But I'm still getting random errors:

TypeError: undefined is not a function
    at Ep.Session.reopen.merge (http://bebop.local:9494/dist/deps.js:71930:23)
    at Ep.ChildSession.Ep.Session.extend.merge (http://bebop.local:9494/dist/deps.js:71866:47)
    at apply (http://bebop.local:9494/dist/deps.js:31391:27)
    at superWrapper [as merge] (http://bebop.local:9494/dist/deps.js:30969:15)
    at http://bebop.local:9494/dist/deps.js:70533:40
    at Array.map (native)
    at http://bebop.local:9494/dist/deps.js:70532:31
    at invokeCallback (http://bebop.local:9494/dist/deps.js:33957:19)
    at publish (http://bebop.local:9494/dist/deps.js:33627:9)
    at publishRejection (http://bebop.local:9494/dist/deps.js:34055:7)

Not sure what to do now....

ghempton commented 10 years ago

Sorry you are having issues. We use embedded records quite extensively, but the embeddedness stops at the rest level (as our backend is relational). I'd be happy to skype/pair etc. with you to get this to work. You are pioneering using a document store with epf.