Js-code from lib/model.js gets included in SOAP-request causing SOAP-fault

[jmmeijer]

Steps to reproduce

• [ ] bug

I have auto discovered a SOAP 1.1 Webservice with lb soap and Loobback created models as expected. However when using the StrongLoop API explorer, the SOAP requests fail as they are not properly formatted.

Stack trace

"Error: faultcode: soap:Client faultstring: Unmarshalling Error: cvc-complex-type.2.4.a: Invalid content was found starting with element '__cachedRelations'. One of '{xxx, xxx}' is expected.
at XMLHandler.xmlToJson (testapp\node_modules\strong-soap\src\parser\xmlHandler.js:635:23)
at testapp\node_modules\strong-soap\src\client.js:279:28
at testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:172:22
at doNotify (testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:99:49)
at SOAPConnector.ObserverMixin._notifyBaseObservers (testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:122:5)
at SOAPConnector.ObserverMixin.notifyObserversOf (testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:97:8)
at cbForWork (testapp\node_modules\loopback\node_modules\loopback-datasource-juggler\lib\observer.js:162:14)
at Request._callback (testapp\node_modules\loopback-connector-soap\lib\http.js:98:9)
at Request.self.callback (testapp\node_modules\request\request.js:188:22)
at emitTwo (events.js:106:13)
at Request.emit (events.js:191:7)
at Request.<anonymous> (testapp\node_modules\request\request.js:1171:10)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
at IncomingMessage.<anonymous> (testapp\node_modules\request\request.js:1091:12)\
at IncomingMessage.g (events.js:292:16)"

When enabling the debug on loopback:persisted-model I can see the SOAP xml is cluttered by code originating from strongloop/loopback-datasource-juggler as is mentioned in issue 77 from loopback-connecter-soap by @liudonghua123. Since the code is in this repo I thought it would be best to open an issue here.

Expected result

I would expect the SOAP request to be properly formatted. And a SOAP response without an error.

Additional information

win32 x64 6.11.0

+-- loopback@3.8.0 +-- loopback-boot@2.24.1 +-- loopback-component-explorer@2.7.0 +-- loopback-connector-soap@4.0.0 +-- loopback-datasource-juggler@3.9.1

[jmmeijer]

I can't help but think this has something to do with the XMLParser and the conversion from JS object to JSON to XML. But until now I cannot quite put my finger on the issue.

[rashmihunt]

@jmmeijer Can you post your WSDL with XSDs here?

[rashmihunt]

@jmmeijer As per the stack trace, I see that you are getting a response from your Web Service and it's a Fault response and the Fault response message from your web service is Invalid content was found starting with element '__cachedRelations'. One of '{xxx, xxx}' is expected. Can you check why your Web Service is returning this fault for your request? Check your SOAP request and see where it has '__cachedRelations' in SOAP body.. I am suspecting the JSON input for the request may be incorrect.

[jmmeijer]

@rashmihunt Thank you for your fast reply!

It seems there is no XSD defined for the WSDL. I was trying to post to method getDeelnemer, which has two parameters: apiSleutel and deelnemernummer.

This is the request format generated by SoapUI:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:api="http://api.algemeen.webservices.eduarte.topicus.nl/">
   <soapenv:Header/>
   <soapenv:Body>
      <api:getDeelnemer>
         <!--Optional:-->
         <apiSleutel>xxx</apiSleutel>
         <deelnemernummer>xxx</deelnemernummer>
      </api:getDeelnemer>
   </soapenv:Body>
</soapenv:Envelope>

This returns valid response from the webservice.

This is the JSON that the Strongloop API Explorer proposes, which seems correspond with the SOAP XML.

{
  "apiSleutel": "string",
  "deelnemernummer": 0
}

Here are links to the WSDL:

• deelnemers.wsdl
• deelnemers.wsdl?wsdl=deelnemers.wsdl

[jmmeijer]

@rashmihunt The problem is similar to or the same as the issue i referred to. A complete SOAP request is posted over there. The problem is a part of the JS-code from /lib/model.js gets included in the SOAP request. Somehow the object of ModelBaseClass is converted to SOAP xml including some of the code to initiate its properties. This is how cachedRelations (a protected propertyof the model) gets converted to `<cachedRelations/>`.

[jmmeijer]

@rashmihunt Something goes wrong after the method jsonToXML gets called: xmlHandler.jsonToXml(soapBodyElement, nsContext, inputBodyDescriptor, args); I think the problem might be in the inputBodyDescriptor, when the form is unqualified, and because of that the object of ModelBaseClass is not properly converted to SOAP. Also the URI's referred in the WSDL's to are no longer available.

strong-soap:client client request. inputBodyDescriptor:

{
    "elements": [
        {
            "elements": [],
            "attributes": [],
            "qname": {
                "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
                "name": "apiSleutel"
            },
            "type": {
                "nsURI": "http://www.w3.org/2001/XMLSchema",
                "name": "string"
            },
            "form": "unqualified",
            "isMany": false,
            "isSimple": true,
            "refOriginal": {
                "elements": [],
                "attributes": [],
                "qname": {
                    "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
                    "name": "apiSleutel"
                },
                "type": {
                    "nsURI": "http://www.w3.org/2001/XMLSchema",
                    "name": "string"
                },
                "form": "unqualified",
                "isMany": false,
                "isSimple": true
            }
        },
        {
            "elements": [],
            "attributes": [],
            "qname": {
                "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
                "name": "deelnemernummer"
            },
            "type": {
                "nsURI": "http://www.w3.org/2001/XMLSchema",
                "name": "int"
            },
            "form": "unqualified",
            "isMany": false,
            "isSimple": true,
            "refOriginal": {
                "elements": [],
                "attributes": [],
                "qname": {
                    "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
                    "name": "deelnemernummer"
                },
                "type": {
                    "nsURI": "http://www.w3.org/2001/XMLSchema",
                    "name": "int"
                },
                "form": "unqualified",
                "isMany": false,
                "isSimple": true
            }
        }
    ],
    "attributes": [],
    "qname": {
        "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
        "name": "getDeelnemer"
    },
    "type": {
        "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
        "name": "getDeelnemer"
    },
    "form": "qualified",
    "isMany": false,
    "isSimple": false,
    "typeDescriptor": {
        "elements": [
            {
                "elements": [],
                "attributes": [],
                "qname": {
                    "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
                    "name": "apiSleutel"
                },
                "type": {
                    "nsURI": "http://www.w3.org/2001/XMLSchema",
                    "name": "string"
                },
                "form": "unqualified",
                "isMany": false,
                "isSimple": true,
                "refOriginal": {
                    "elements": [],
                    "attributes": [],
                    "qname": {
                        "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
                        "name": "apiSleutel"
                    },
                    "type": {
                        "nsURI": "http://www.w3.org/2001/XMLSchema",
                        "name": "string"
                    },
                    "form": "unqualified",
                    "isMany": false,
                    "isSimple": true
                }
            },
            {
                "elements": [],
                "attributes": [],
                "qname": {
                    "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
                    "name": "deelnemernummer"
                },
                "type": {
                    "nsURI": "http://www.w3.org/2001/XMLSchema",
                    "name": "int"
                },
                "form": "unqualified",
                "isMany": false,
                "isSimple": true,
                "refOriginal": {
                    "elements": [],
                    "attributes": [],
                    "qname": {
                        "nsURI": "http://api.algemeen.webservices.eduarte.topicus.nl/",
                        "name": "deelnemernummer"
                    },
                    "type": {
                        "nsURI": "http://www.w3.org/2001/XMLSchema",
                        "name": "int"
                    },
                    "form": "unqualified",
                    "isMany": false,
                    "isSimple": true
                }
            }
        ],
        "attributes": [],
        "name": "getDeelnemer",
        "xmlns": "http://api.algemeen.webservices.eduarte.topicus.nl/",
        "isSimple": false
    }
}

strong-soap:client client request, calling jsonToXml. args:

{
    "apiSleutel": "xxx",
    "deelnemernummer": 000
}

[jmmeijer]

@rashmihunt Upon further investigation I found out, that args is not properly passed to xmlHandler.jsonToXml(). Somehow the code of the lib/model.js gets into the method. I've added the line debug('jsonToXml, node: %s', node); after src/parser/xmlHandler.js:26 and the code of the object of ModelBaseClass gets logged...

[ghost]

Hi, I'm having the same problem in my application. Is there any way to fix this? Thanks

[ExTheSea]

@jmmeijer or @rashmihunt: Is there any workaround (e.g. by using an older version) or an expected duration until a fix will be available?

This only seems to affect models created using lb soap as creating the calls to the Soap Webservice yourself as described in e.g. in the strongloop blog seems to work.

I'm asking due to trying to evaluate whether it is worth it manually creating the soap calls until the fix will be available as lb soap would make this much easier.

[ExTheSea]

Fyi @william-santos-bwti @rashmihunt After a little bit of experimenting i have since found a way to work around the issue. While it's not that nice it seems to be enough to deep copy the input objects e.g. using JSON.parse(JSON.stringify(inputvar))

So the model methods inside server/models/.js would look like:

<ServiceBinding>.<someSoapMethod> = function(<inputObject>, callback) {
    <ServiceBinding>.<someSoapMethod>(JSON.parse(JSON.stringify(<inputObject>)), function(err, response) {
      var result = response;
      callback(err, result);
    });
  };

For one of the two soap services I tested though i had to additionally drill down to the first layer of the parsed object as it was otherwise added twice. So i did: JSON.parse(JSON.stringify(<inputObject>))["top_Element"]

It seems like the problem arises because the inputObject somehow either gets additional properties while the object is parsed to xml or the additional properties are ignored when doing JSON.stringify().

Hope this helps anyone and looking forward to updates.

[ghost]

Hi, @ExTheSea I decide to use node module "soap" and implement the requisitions for the webservices inside a middleware loopback and store in mongodb with the help of the "adapter" pattern to turn the soap model into my model. My models are loopback default, use the mongodb datasource and are published as REST services. This solved for me while the updates do not come.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmmeijer]

Bump to prevent this issue from being closed. Eventhough several workarounds have been proposed, this issue has not yet been resolved and persists in latest version.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmmeijer]

Hello @rashmihunt, are you still involved? If this is not the case, could this ticket be reassigned to someone else?

[liudonghua123]

I updated the latest dependencies, but still the same error.

my dependencies part of package.json is below.

  "dependencies": {
    "compression": "^1.7.1",
    "cors": "^2.8.4",
    "helmet": "^3.9.0",
    "loopback": "^3.17.1",
    "loopback-boot": "^2.27.0",
    "loopback-component-explorer": "^5.2.0",
    "loopback-connector-soap": "^4.0.1",
    "serve-favicon": "^2.4.5",
    "strong-error-handler": "^2.3.0"
  }

When I invoke an endpoint on loopback-explorer like /api/APISoapBinding/authenticate. The following error is shown.

The error message of backend console is.

未处理请求 POST /api/APISoapBinding/authenticate 的错误：Error: faultcode: soap:Client faultstring: Unmarshalling Error: 意外的元素 (uri:"", local:"__cachedRelations")
。所需元素为<{}password>,<{}user_at_domain>
    at XMLHandler.xmlToJson (D:\code\loopback\ynu-mail-ws\node_modules\strong-soap\src\parser\xmlHandler.js:637:23)
    at D:\code\loopback\ynu-mail-ws\node_modules\strong-soap\src\client.js:279:28
    at D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:250:22
    at doNotify (D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:155:49)
    at SOAPConnector.ObserverMixin._notifyBaseObservers (D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:178:5)
    at SOAPConnector.ObserverMixin.notifyObserversOf (D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:153:8)
    at cbForWork (D:\code\loopback\ynu-mail-ws\node_modules\loopback-datasource-juggler\lib\observer.js:240:14)
    at Request._callback (D:\code\loopback\ynu-mail-ws\node_modules\loopback-connector-soap\lib\http.js:98:9)
    at Request.self.callback (D:\code\loopback\ynu-mail-ws\node_modules\request\request.js:186:22)
    at Request.emit (events.js:159:13)
    at Request.<anonymous> (D:\code\loopback\ynu-mail-ws\node_modules\request\request.js:1163:10)
    at Request.emit (events.js:159:13)
    at IncomingMessage.<anonymous> (D:\code\loopback\ynu-mail-ws\node_modules\request\request.js:1085:12)
    at Object.onceWrapper (events.js:254:19)
    at IncomingMessage.emit (events.js:164:20)
    at endReadableNT (_stream_readable.js:1062:12)

[jmmeijer]

After reading up on ES6 I believe this issue might have to do with an object prototype which is iterated over with a for ... in loop for conversion to SOAP causing the methods of this object to be included. In this case using for...of loop would be preferable. Hope this helps! Read more: Difference between for...of and for...in

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmmeijer]

Another bump in order to prevent this issue from being closed by stale bot.

[liudonghua123]

@kjdelisle @jannyHou @loay @b-admike @ssh24 @virkt25 @dhmlau @zbarbuto @nitro404 Hi, could you help us with this issue?

[dhmlau]

@raymondfeng , could you please help?

[raymondfeng]

Object instances of LoopBack models need to be converted to plain object using toJSON() or toObject() before passing to json2xml.

[jersonjohn]

I am still facing the same issue. Getting created unwanted <__cachedRelations/>\n <__data> for every child node, I tried both toJSON and toObject but still the same. Spending like 3 days now to figure this out, Like tried almost everything suggested here, Could you please help me? If you need more details I can provide.

FYI I have generated the models using CLI, lb soap [WSDL local file]

AccountCreationAccountCreationSoap.AccountCreation = function(AccountCreation, callback) {

  //console.log('AccountCreation',AccountCreation);

  //iterate(soapDataSource.connector);

  //fs.writeFile('./GeneratedAccountCreationPayload.WSDL',soapDataSource.connector);

  var AccountCreationReq = AccountCreation.toObject();

  AccountCreationAccountCreationSoap.AccountCreation(AccountCreationReq, function (err, response) {

    console.log('GeneratedAccountCreationPayload',soapDataSource.connector);

    var result = response;
    callback(err, result);
  });

}

Thanks in advance

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmmeijer]

Bump in order to prevent issue from being closed without solution...

[martyjt]

This error has been open for a year? Is it likely to be fixed ever? trying to test on a simple SOAP service http://www.dneonline.com/calculator.asmx . it's posting junk to the service.

[martyjt]

@jmmeijer Reading through some of the ideas here , I don't know if this fixes the overall bug , but changing

https://github.com/strongloop/loopback-soap/blob/master/lib/codegen/model.ejs , line 30 to +

'(' + op.operationId + '.toObject(), function (err, response) {';

I just added the .toObject() seems to fix it for me so I can generate objects that will work.

[liudonghua123]

@martyjt You can send a pr to fix this issue.

[martyjt]

@liudonghua123 Sorry , I don't have time to learn about test coverage / create tests for this. It is really just to help out. I am not familar with github.

[jmmeijer]

@martyjt As far as I can tell strong-soap is being used instead of loopback-soap. However I did find similar code that is being used in the swagger API Explorer: https://github.com/strongloop/loopback-swagger/blob/master/lib/codegen/model.ejs

[bajtos]

When enabling the debug on loopback:persisted-model I can see the SOAP xml is cluttered by code originating from strongloop/loopback-datasource-juggler as is mentioned in issue 77 from loopback-connecter-soap by @liudonghua123. Since the code is in this repo I thought it would be best to open an issue here.

Please note that models backed by a service connectors (soap, REST, remote) must be configured with Model as the base model, not PersistedModel!

I believe lb model is offering Model by default when you select a datasource used by a service connector.

To change the base model in an existing codebase, open the model JSON file (e.g. common/models/APISoapBinding.json) and change the following line:

- "base": "PersistedModel"
+ "base": "Model"

With this change in place, your model instance should no longer have extra persistence-related properties like __cachedRelations.

[sanderboom]

Copied from Gitter:

@bajtos Thanks for the reply! Unfortunately all "base" properties are already set to "Model".

@sanderboom in that case I have run out of ideas of what may be wrong. AFAICT, the original bug report does not provide any app we could use to reproduce the problem. It would help a lot if you could create one - see https://loopback.io/doc/en/contrib/Reporting-issues.html#bug-report

[ThomasVuillaume]

Just to mention few things:

I still have this bug with this package.json:

{
  "name": "loopback-soap-test",
  "version": "1.0.0",
  "main": "server/server.js",
  "engines": {
    "node": ">=4"
  },
  "scripts": {
    "lint": "eslint .",
    "start": "node .",
    "posttest": "npm run lint"
  },
  "dependencies": {
    "compression": "^1.0.3",
    "cors": "^2.5.2",
    "helmet": "^3.10.0",
    "loopback": "^3.22.0",
    "loopback-boot": "^2.6.5",
    "loopback-component-explorer": "^6.2.0",
    "loopback-connector-soap": "^4.3.0",
    "serve-favicon": "^2.0.1",
    "strong-error-handler": "^3.0.0"
  },
  "devDependencies": {
    "eslint": "^3.17.1",
    "eslint-config-loopback": "^8.0.0"
  },
  "repository": {
    "type": "",
    "url": ""
  },
  "license": "UNLICENSED",
  "description": "loopback-soap-test"
}

I used lb soap command, as mentionned already, (just following this https://loopback.io/doc/en/lb3/Connecting-to-SOAP.html and trying to post stuff to this https://github.com/honestserpent/node-soap-example )

I double check what @bajtos said, all models are generated as "Models", for example :

{
  "name": "MessageSplitterRequest",
  "base": "Model",
  "idInjection": false,
  "options": {
    "validateUpsert": true
  },
  "forceId": "false",
  "excludeBaseProperties": [
    "id"
  ],
  "properties": {
    "message": {
      "type": "string"
    },
    "splitter": {
      "type": "string"
    }
  },
  "validations": [],
  "relations": {},
  "acls": [],
  "methods": {}
}

Finally, I tried what @ExTheSea proposed, and it finally worked. So my binding model is like this now:

  /**
   * MessageSplitter
   * @param {MessageSplitterSoapIn} MessageSplitterSoapIn MessageSplitterSoapIn
   * @callback {Function} callback Callback function
   * @returns {any} callback containing error or result. Result is the response/soap body in JSON form.
   */
  MessageSplitterServiceMessageSplitterServiceSoapBinding.MessageSplitter = function(MessageSplitterSoapIn, callback) {
    MessageSplitterServiceMessageSplitterServiceSoapBinding.MessageSplitter(JSON.parse(JSON.stringify(MessageSplitterSoapIn)), function(err, response) {
      var result = response;
      callback(err, result);
    });
  };

[liudonghua123]

@ThomasVuillaume Hi, thanks for your working code. Could you paste the full code of binding model js. I have an empty function like this

'use strict';

module.exports = function(Authenticate) {

};

[liudonghua123]

Finally, I find the code is in server/models/soap-api-soap-binding.js not in common/models/*.js.

[jmmeijer]

@liudonghua123 @ThomasVuillaume @ExTheSea I have also succesfully implemented the suggested code.

However the JSON-object passed to the SoapBinding methods has already been stringified when I log from inside the method: console.log(JSON.parse(<inputObject>));.

For me using just JSON.parse works since the JSON object being passed is a valid JSON-string (when using the StrongLoop API Explorer:

EDIT: I apologize for not testing thoroughly before commenting. I got an error when using only JSON.parse. A JS-object gets passed to the SoapBinding method which seems to be exactly the same after stringifing and parsing JSON... why then won't it work with the passed object?

<ServiceBinding>.<someSoapMethod> = function(<inputObject>, callback) {
    console.log(<inputObject>); // JS-object
    console.log(JSON.stringify(<inputObject>)); // string
    console.log(JSON.parse(JSON.stringify(<inputObject>))); // JS-object
    <ServiceBinding>.<someSoapMethod>(JSON.parse(JSON.stringify(<inputObject>)), function(err, response) {
      var result = response;
      callback(err, result);
    });
  };

Output:

{ apiSleutel: 'xxxxxxxxxxxxxxxx', deelnemernummer: 111111 }
{"apiSleutel":"xxxxxxxxxxxxxxxx","deelnemernummer":111111}
{ apiSleutel: 'xxxxxxxxxxxxxxxx', deelnemernummer: 111111 }

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmmeijer]

This bug has not yet been resolved. Bump to keep the issue active. Until Loopback 4 has a implementation for autodiscovery and model generation for SOAP, this is still a viable solution.

[bajtos]

I am afraid we don't have bandwidth to address this issue. @jmmeijer would you like to contribute the fix yourself? We are happy to help you along the way. See https://loopback.io/doc/en/contrib/code-contrib.html to get started.