Unable to remove tags

[isidrov]

Thank you for this much needed library.

The issue I am finding is that Zeep is sending all the tags available for a specific method. The server will throw an exception unless certain tags are removed. I dig into the documentation and unable to know how or if this is possible with Zeep

Zeep version 0.24

WSDL

https://gist.github.com/isidrov/24b73e5a8499f02d92c6f566148f3375

Example script

https://gist.github.com/isidrov/408ee6ba3914432287a0f235bdf50691

What I am actually sending with current script

https://gist.github.com/isidrov/047dace78d9e83ba337f602cf31b567f

What I need to send to make it work (confirmed by SOAPUI)

https://gist.github.com/isidrov/fd47320ac5312c707600db83d055b938

Let me know if anything else is needed.

Thanks!

[mvantellingen]

I've added a SkipValue. If you pass zeep.xsd.SkipValue as a value to one of the arguments then the value won't be rendered

[mvantellingen]

I'm closing this ticket for now as it should be fixed. If there are still any issues then please let me know so that I can re-open the issue.

[isidrov]

Hello,

With Zeep 0.27 my SOAP call does not work, script is the same just changed the http_auth by Session() v0.26 where soap attachments were inlcuded did not show this error . The error I am getting is below. Also would you be able to show a simple example on how to use SkipValue? do I have to replace SkipValue by the actual value or is the value passed between brackets? Thanks

header: Accept-Ranges header: ETag header: Server header: P3P header: X-Powered-By header: Date header: ntCoent-Length header: Content-Encoding header: Content-Length header: Content-Type header: Last-Modified header: Via header: Connection Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1596, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 974, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/isidrofernandez/PycharmProjects/cola/zeepVLC.py", line 256, in <module>
    'RelTime': '5'
  File "/usr/local/lib/python3.5/site-packages/zeep/client.py", line 35, in __call__
    self._op_name, args, kwargs)
  File "/usr/local/lib/python3.5/site-packages/zeep/wsdl/bindings/soap.py", line 109, in send
    options=options)
  File "/usr/local/lib/python3.5/site-packages/zeep/wsdl/bindings/soap.py", line 67, in _create
    serialized = operation_obj.create(*args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/zeep/wsdl/definitions.py", line 165, in create
    return self.input.serialize(*args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/zeep/wsdl/messages/soap.py", line 48, in serialize
    self.body.render(body, body_value)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 163, in render
    self._render_value_item(parent, value, render_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 181, in _render_value_item
    return self.type.render(node, value, None, render_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/types/complex.py", line 195, in render
    element.render(parent, element_value, child_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/indicators.py", line 187, in render
    element.render(parent, element_value, child_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 163, in render
    self._render_value_item(parent, value, render_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 181, in _render_value_item
    return self.type.render(node, value, None, render_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/types/complex.py", line 195, in render
    element.render(parent, element_value, child_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/indicators.py", line 187, in render
    element.render(parent, element_value, child_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 163, in render
    self._render_value_item(parent, value, render_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 181, in _render_value_item
    return self.type.render(node, value, None, render_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/types/complex.py", line 178, in render
    attribute.render(parent, attr_value, child_path)
  File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/any.py", line 219, in render
    for name, val in value.items():
AttributeError: '_StaticIdentity' object has no attribute 'items'

[mvantellingen]

It seems to work with the following code:

session = Session()
session.auth = HTTPBasicAuth(username, password)
session.verify = False
transport = Transport(session=session, cache=SqliteCache())

logFileClient = Client(LogCollectionWsdl, transport=transport)

args = tuple()
kwargs = {
    'FileSelectionCriteria': {
        'ServiceLogs': {'item': 'Cisco CallManager'},
        'Frequency': 'OnDemand',
        'JobType': 'DownloadtoClient',
        'RelText': 'Hours',
        'RelTime': '5',

        'SearchStr': xsd.SkipValue,
        'Port': xsd.SkipValue,
        'ZipInfo': xsd.SkipValue,
    }
}

[isidrov]

I get the same error with your code. Just to clarify the error happens even when not using SkipValue for a soap call it was working with 0.24 code. I will go deeper into the debug later today. Thanks.

[isidrov]

it seems that in the below code in def render() at indicators.py, we always hit the 'else' statement and hence the element_value = NotSet line

if name: if name in value: element_value = value[name] child_path = render_path + [name] else: element_value = NotSet child_path = render_path

While name is always set, name is never in value, which is a dict of the parameters I am passing: e.g this is taken from the debug

{'FileSelectionCriteria': {'JobType': 'DownloadtoClient', 'RelTime': '5', 'Frequency': 'OnDemand', 'ServiceLogs': {'item': 'Cisco CallManager'}, 'RelText': 'Hours', 'Port': SkipValue, 'SearchStr': 'SDL'}}

This causes at the def validate() in element.py to raise the "Missing element %s" exception

if not self.is_optional and not self.nillable and value in (None, NotSet): raise exceptions.ValidationError( "Missing element %s" % (self.name), path=render_path)

Let me know if it makes sense and you need any more info.

[isidrov]

By the way with 0.24 was not working either, I thought it was but through further testing I realize that the SOAP envelope had all attributes empty. This is the current error/ This only happens for those attributes where nillable is not set as True in the WSDL

header: Cache-Control header: Expires header: Set-Cookie header: Set-Cookie header: X-Frame-Options header: Content-Type header: Transfer-Encoding header: Date header: Server send: b'GET /soap/encoding/ HTTP/1.1\r\nHost: schemas.xmlsoap.org\r\nConnection: keep-alive\r\nAccept: /\r\nUser-Agent: Zeep/0.27.0 (www.python-zeep.org)\r\nAccept-Encoding: gzip, deflate\r\nAuthorization: Basic Y2NtYWRtaW46QzFzY28xMjM=\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' Traceback (most recent call last): File "/Users/username/PycharmProjects/cola/zeepVLC.py", line 267, in response = logFileClient.service.SelectLogFiles(FileSelectionCriteria) File "/usr/local/lib/python3.5/site-packages/zeep/client.py", line 35, in call self._op_name, args, kwargs) File "/usr/local/lib/python3.5/site-packages/zeep/wsdl/bindings/soap.py", line 109, in send options=options) File "/usr/local/lib/python3.5/site-packages/zeep/wsdl/bindings/soap.py", line 67, in _create serialized = operation_obj.create(*args, *kwargs) File "/usr/local/lib/python3.5/site-packages/zeep/wsdl/definitions.py", line 165, in create return self.input.serialize(args, **kwargs) File "/usr/local/lib/python3.5/site-packages/zeep/wsdl/messages/soap.py", line 48, in serialize self.body.render(body, body_value) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 157, in render self._render_value_item(parent, value, render_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 175, in _render_value_item return self.type.render(node, value, None, render_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/types/complex.py", line 195, in render element.render(parent, element_value, child_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/indicators.py", line 187, in render element.render(parent, element_value, child_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 157, in render self._render_value_item(parent, value, render_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 175, in _render_value_item return self.type.render(node, value, None, render_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/types/complex.py", line 195, in render element.render(parent, element_value, child_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/indicators.py", line 187, in render element.render(parent, element_value, child_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 151, in render self.validate(value, render_path) File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 197, in validate "Missing element %s" % (self.name), path=render_path) zeep.exceptions.ValidationError: Missing element SearchStr (SelectLogFiles.FileSelectionCriteria)

[mvantellingen]

The ValidationError is thrown correctly there this the SearchStr is not optional but required. If you want to skip it anyway then you can use the new zeep.xsd.SkipValue constant.

[isidrov]

I am actually passing a SearchStr attribute and value and Zeep builds correctly the below dict.

{'FileSelectionCriteria': {'JobType': 'DownloadtoClient', 'RelTime': '5', 'Frequency': 'OnDemand', 'ServiceLogs': {'item': 'Cisco CallManager'}, 'RelText': 'Hours', 'Port': SkipValue, 'SearchStr': 'SDL'}

FileSelectionCriteria = { 'FileSelectionCriteria': { 'ServiceLogs': {'item': 'Cisco CallManager'}, 'SearchStr': 'SDL', 'Frequency': 'OnDemand', 'JobType': 'DownloadtoClient', 'RelText': 'Hours', 'RelTime': '5', 'Port': xsd.SkipValue } }

[mvantellingen]

But that example is missing the ZipInfo element again. I'm really trying to understand the issue but failing so far. Can you provide a new full example which we can both run?

[isidrov]

Yes, adding ZipInfo SkipValue throw the same error, I actually tried multiple combinations, all with the same result. Below is what I am running right now:

` LogCollectionWsdl = 'https://' + pub + ':' + cmPort + '/logcollectionservice/services/LogCollectionPort?wsdl'

session = Session()
session.verify = False
session.auth = HTTPBasicAuth(username, password)
transport = Transport(session=session, cache=SqliteCache())

logFileClient = Client(wsdl = LogCollectionWsdl, transport=transport)

FileSelectionCriteria = {
    'FileSelectionCriteria': {
        'ServiceLogs': {'item': 'Cisco CallManager'},
        'SearchStr': 'SDL',
        'Frequency': 'OnDemand',
        'JobType': 'DownloadtoClient',
        'RelText': 'Hours',
        'RelTime': '5',
        'Port': xsd.SkipValue,
        'ZipInfo': xsd.SkipValue
    }
}

response = logFileClient.service.SelectLogFiles(FileSelectionCriteria)`

From the screenshot below, the attributes where nillable = True does not appear, will throw an error, even when specifically configure them with a value, with a SkipValue or omitting the attribute. I might be wrong but all attributes fail to meet the condition 'if name in value:', either when they are required or optional, which makes them go to the condition in validate() f not self.is_optional and not self.nillable and value in (None, NotSet)

The below request from SoapUI does work

[mvantellingen]

Try the following with master branch. The issue is that the wsdl doesn't set minOccurs=0 while you don't want to add the elements. So basically the working soap-ui response should be non valid.

FileSelectionCriteria = {
    'FileSelectionCriteria': {
        'ServiceLogs': {'item': 'Cisco CallManager'},
        'SystemLogs': xsd.SkipValue,
        'SearchStr': 'SDL',
        'Frequency': 'OnDemand',
        'JobType': 'DownloadtoClient',
        'RelText': 'Hours',
        'RelTime': '5',
        'ToDate': xsd.SkipValue,
        'FromDate': xsd.SkipValue,
        'Port': xsd.SkipValue,
        'ZipInfo': xsd.SkipValue,
        'UserName': xsd.SkipValue,
        'Password': xsd.SkipValue,
        'RemoteFolder': xsd.SkipValue,
    }
}

[isidrov]

Same error, if there is anything else I can provide, let me know. I can let you connect to a lab if you need to debug with a real call to this server.

File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 197, in validate "Missing element %s" % (self.name), path=render_path) zeep.exceptions.ValidationError: Missing element SearchStr (SelectLogFiles.FileSelectionCriteria)

[isidrov]

I didn't realized Master was upgraded. Tried again with the below error

File "/usr/local/lib/python3.5/site-packages/zeep/xsd/elements/element.py", line 201, in validate "Missing element %s" % (self.name), path=render_path) zeep.exceptions.ValidationError: Missing element SearchStr (SelectLogFiles.FileSelectionCriteria)

If it helps element_value never gets initialized with SkipValue. I tried with SearchStr below is a screenshot of the debug

[molszowy]

Hey!

I am working on the same SOAP API as Isidrio. The request is sent but rejected by the server('org.xml.sax.SAXException: SimpleDeserializer encountered a child element, which is NOT expected, in something it was trying to deserialize.') and it seems that the problem is missing xsi:type. Can you advise how to force zeep to add it(WSDL attached LogCollectionPortUCM105.txt?

<?xm version="1.0" encoding="utf-8"?><soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
  <soap-env:Body>
    <ns0:SelectLogFiles xmlns:ns0="http://schemas.cisco.com/ast/soap/">
      <FileSelectionCriteria>
        <ns1:ServiceLogs xmlns:ns1="http://cisco.com/ccm/serviceability/soap/LogCollection/">
          <ns1:item>Cisco CallManager</ns1:item>
        </ns1:ServiceLogs>
        <ns2:SearchStr xmlns:ns2="http://cisco.com/ccm/serviceability/soap/LogCollection/">SDL</ns2:SearchStr>
        <ns3:Frequency xmlns:ns3="http://cisco.com/ccm/serviceability/soap/LogCollection/">OnDemand</ns3:Frequency>
        <ns4:JobType xmlns:ns4="http://cisco.com/ccm/serviceability/soap/LogCollection/">DownloadtoClient</ns4:JobType>
        <ns5:TimeZone xmlns:ns5="http://cisco.com/ccm/serviceability/soap/LogCollection/">Client: (GMT-0:0) Greenwich Mean Time</ns5:TimeZone>
        <ns6:RelText xmlns:ns6="http://cisco.com/ccm/serviceability/soap/LogCollection/">Days</ns6:RelText>
        <ns7:RelTime xmlns:ns7="http://cisco.com/ccm/serviceability/soap/LogCollection/">10</ns7:RelTime>
      </FileSelectionCriteria>
    </ns0:SelectLogFiles>
  </soap-env:Body>
</soap-env:Envelope>

element = log_collection_client.get_type('{http://cisco.com/ccm/serviceability/soap/LogCollection/}ArrayOfString')
    ccm = element(item=['Cisco CallManager'])

  log_collection_client.service.SelectLogFiles({'ServiceLogs':ccm,
                                                              'SearchStr':'SDL',
                                                              'Frequency':'OnDemand',
                                                              'JobType':'DownloadtoClient',
                                                              'TimeZone':'Client: (GMT-0:0) Greenwich Mean Time',
                                                              'RelText':'Days',
                                                              'RelTime':'10',
                                                              'ToDate': xsd.SkipValue,
                                                              'FromDate': xsd.SkipValue,
                                                              'Port': xsd.SkipValue,
                                                              'ZipInfo': xsd.SkipValue,
                                                              'UserName': xsd.SkipValue,
                                                              'Password': xsd.SkipValue,
                                                              'RemoteFolder': xsd.SkipValue,
                                                              'IPAddress': xsd.SkipValue,
                                                              'SystemLogs': xsd.SkipValue})]

[mvantellingen]

Is it possible to get give me an example script so that I can easily reproduce it? It's all a bit vague for me now unfortunately. See the docs on how to create an example script, if you could add a assert statement in the script with the expected xml values then that would be even better