Invalid content was found starting with element \'ns0:quantity\'

[jedie]

I don't like to think that the mistake is mine. I get this error:

MESSAGE:
b'<?xml version=\'1.0\' encoding=\'UTF-8\'?><S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><S:Body><createResponse xmlns="http://www.docdatapayments.com/services/paymentservice/1_2/"><createError><error code="REQUEST_DATA_INCORRECT">XML request does not match XSD. The data is: cvc-complex-type.2.4.a: Invalid content was found starting with element \'ns0:quantity\'. One of \'{"http://www.docdatapayments.com/services/paymentservice/1_2/":quantity}\' is expected..</error></createError></createResponse></S:Body></S:Envelope>'

But in the soap "create" request contains: 1</ns0:quantity>

Any idea?

[jedie]

Think it's a XML namespaces problem!

TotalVatAmount.to_xml() and Quantity.to_xml() doesn't use suds.client.Factory().create()

EDIT: I found https://stackoverflow.com/questions/27640853/howto-set-text-value-for-suds-webservice-object ;)

[maerteijn]

With "I found it" you mean you have a solution for your problem?

[jedie]

Maybe, see: https://github.com/wearehoods/django-oscar-docdata/commit/1a2a079942f1af9f3a64cef2ab410c6c1504c30e

Don't know if this is related to the update from suds-jurko -> suds-community

[jedie]

Hm. Seems not to be fixed :( The bug was probably only hidden by #16 who raise a Traceback earlier...

Current "Type not found: 'unitOfMeasure'" traceback:

With this code:

class Quantity(object):
    """
    An quantity for Docdata

    <element name="quantity" minOccurs="1" maxOccurs="1">
        <annotation>
            <documentation>Quantity of this item that's being ordered.
            </documentation>
        </annotation>
        <complexType>
            <simpleContent>
                <extension base="int">
                    <attribute name="unitOfMeasure" use="required">
                        <annotation>
                            <documentation>
                                Unit of measurement.

                                The attribute can have the
                                following values: PCS - pieces
                                SEC- seconds BYT - bytes KB -
                                kilobytes
                            </documentation>
                        </annotation>
                        <simpleType>
                            <restriction base="string">
                                <enumeration value="PCS"/>
                                <enumeration value="SEC"/>
                                <enumeration value="BYT"/>
                                <enumeration value="KB"/>
                            </restriction>
                        </simpleType>
                    </attribute>
                </extension>
            </simpleContent>
        </complexType>
    </element>
    """
    def __init__(self, value, unit='PCS'):
        self.value = int(value)
        assert unit in (
             "PCS",  # pieces
             "SEC",  # seconds
             "BYT",  # bytes
             "KB",  # kilobytes
        )
        self.unit = unit

    def to_xml(self, factory):
        node = factory.create('ns0:quantity')
        node.unitOfMeasure = self.unit
        node.int = self.value
        return node

[jedie]

any news here?

[maerteijn]

Any change you sketch a detailed scenario when you encounter this so we can try to reproduce?

[jedie]

Nothing special is necessary for that: Simply use the payment test backend account. Try to buy and pay for an article.

The origin code is:

    def to_xml(self, factory):
        # Needs to be an xsd:int with an attribute
        # Can't do that with factory.create('ns0:quantity')
        #metadata = factory.resolver.find('ns0:quantity')
        #ns = metadata.namespace()

        element = Element('ns0:quantity')
        element.setText(str(self.value))
        element.set('unitOfMeasure', self.unit)
        return element

This was committed here: https://github.com/django-oscar/django-oscar-docdata/commit/ce62b43719fbc69e6ccc7eb18d4b5b4816109a69#diff-b997b1a5e15cf73c89f351aa34c328bbR806

With this origin code you can raise into different namespaces... e.g.: Everything is "ns0" and the inserted "quantity" has "ns1" prefix. Or the other case: everything has "ns1" and it's "ns0:quantity"

I think the code from https://github.com/django-oscar/django-oscar-docdata/issues/14#issuecomment-456024936 is the right solution, but raise into "Type not found: 'unitOfMeasure'"

[jedie]

any news?

[maerteijn]

I first focused on creating a working (and less hackish) sandbox. I have it now working and it is in the update-sandbox branch.

I will merge it when the cleanup is done. The idea is to write at least some functional tests on the sandbox the following week.

On topic: With the new sandbox I can reproduce your error now with python 3.7:

DocdataClient: failed to create payment for order 1901302210: code=REQUEST_DATA_INCORRECT, error=XML request does not match XSD. The data is: cvc-complex-type.2.4.a: Invalid content was found starting with element 'ns0:quantity'. One of '{"http://www.docdatapayments.com/services/paymentservice/1_2/":quantity}' is expected..
Order #1901302210: payment error (('XML request does not match XSD. The data is: cvc-complex-type.2.4.a: Invalid content was found starting with element \'ns0:quantity\'. One of \'{"http://www.docdatapayments.com/services/paymentservice/1_2/":quantity}\' is expected..', DocdataCreateError(XML request does not match XSD. The data is: cvc-complex-type.2.4.a: Invalid content was found starting with element 'ns0:quantity'. One of '{"http://www.docdatapayments.com/services/paymentservice/1_2/":quantity}' is expected..)))
Traceback (most recent call last):
  File "/Users/martijn/Dev/oss/django-oscar-docdata/oscar_docdata/facade.py", line 98, in create_payment
    order_key = super(Facade, self).create_payment(order_number, total, user, language=language, description=description, profile=profile, merchant_name=merchant_name, **kwargs)
  File "/Users/martijn/Dev/oss/django-oscar-docdata/oscar_docdata/interface.py", line 113, in create_payment
    createsuccess = client.create(**call_args)
  File "/Users/martijn/Dev/oss/django-oscar-docdata/oscar_docdata/gateway.py", line 351, in create
    raise DocdataCreateError(error._code, error.value)
oscar_docdata.exceptions.DocdataCreateError: XML request does not match XSD. The data is: cvc-complex-type.2.4.a: Invalid content was found starting with element 'ns0:quantity'. One of '{"http://www.docdatapayments.com/services/paymentservice/1_2/":quantity}' is expected..

As it does not happen with python 2.7 I'm pretty sure it's an encoding issue. Hopefully I have a fix soon.

[maerteijn]

Hmm, not as easy at is seams. I could only reproduce this once. Is it also occasionally happening on your side?

[jedie]

It's IMHO not about encoding. It's about XML namespaces, see: https://github.com/django-oscar/django-oscar-docdata/issues/14#issuecomment-458068355

It's a old problem, see also: https://stackoverflow.com/questions/27640853/howto-set-text-value-for-suds-webservice-object

I don't know if the real problem is with the provider and wrong WSDL files.

The idea is to write at least some functional tests on the sandbox

+1 I always do these in my projects, too.

[jedie]

@maerteijn don't miss to update suds, see: https://github.com/django-oscar/django-oscar-docdata/issues/15

[maerteijn]

Can you answer my question please: Does it happen all the time in your situation?

I want to know because suds can sometimes make a mess of the namespaces which I've seen in several other projects. If you can confirm it happens sometimes but not all the time I know where to look for.

And I will update suds once we found the problem and are able to reproduce it on demand. (narrow down the problem first with as less variables as possible)

[jedie]

Yes, it happens often but not all the time.

[maerteijn]

I had it once again but not in a reproducible way unfortunately. It looks like it is "fixed" with the development server restarted: Suds is doing some dynamic magic when parsing the WSDL so it could be that the auto-reloading of the Django development server (manage.py runserver) is causing this.

Can you confirm this? How did you deploy / run your application?

[jedie]

Hm, interesting...

Currently i use the Django development server via manage.py runserver ...

If that's true: Is then the very first request always correct and all others not?!? If there's a reload problem, it should have caught someone's eye, right?

[maerteijn]

If that's true: Is then the very first request always correct and all others not?!?

No, I mean when you use manage.py runserver and adjust the code somewhere it will automatically reload. I think this is causing suds to mess the namespaces up.

If there's a reload problem, it should have caught someone's eye, right?

I think this only happens with suds and python 3 in combination with the hardcodedns0:quantity, so not a use case which happens really often. Your proposed solution is I think in the right direction,I'll try some things now.

[maerteijn]

21efd5edd1aedf89962b7906b6351dd19b61427c seems like a solution, at least in my local test environment. Could you give it a try and see if it solves the namespace issues?

[jedie]

Seems not to work:

PaymentError: ('XML request does not match XSD. The data is: cvc-complex-type.2.4.a: Invalid content was found starting with element \'ns0:quantity\'. One of \'{"http://www.docdatapayments.com/services/paymentservice/1_2/":quantity}\' is expected..', DocdataCreateError(XML request does not match XSD. The data is: cvc-complex-type.2.4.a: Invalid content was found starting with element 'ns0:quantity'. One of '{"http://www.docdatapayments.com/services/paymentservice/1_2/":quantity}' is expected..,))

EDIT: One moment... Seems that i used the wrong code base...

[jedie]

OK, i checked it: With the old code base i get this:

With your changes i get this:

That doesn't look right, isn't it?

I add logging output, e.g.:

        metadata = factory.resolver.find('ns0:quantity')
        ns=metadata.namespace()
        logger.warning("Found namespace: %r", ns)
        element = Element("quantity", ns=ns)

        element.setText(str(self.value))
        element.set('unitOfMeasure', self.unit)
        logger.warning("Add element: %r", element)

        return element

output:

[maerteijn]

Can you post a complete traceback of the error you get with this code included please?

[jedie]

No, sorry. That contains to much information that i can't post ;) I don't think this will help very much, either.

Does the error not occur with you?

[maerteijn]

No. Can you rescramble the information then with fake data? It would really help to see the complete traceback.

[jedie]

No.

This Bug doesn't appear with you? Strange!

I'm trying to clean up a traceback.

[jedie]

Hm. I currently can't produce this eror...

Because if i use my fork, then i ran into the CSS error: https://github.com/django-oscar/django-oscar-docdata/issues/16

If i use your "master" then i only get the "numberOfDays" Bug: https://github.com/django-oscar/django-oscar-docdata/issues/19

[jedie]

I add more info in #16 and #19

[maerteijn]

The CSS errors and numberOfDays errors are occuring after the Quantity errors. (I can reproduce them with suds-community but not with suds-jurko).

So it looks like the changes in 21efd5edd1aedf89962b7906b6351dd19b61427c fixed at least this issue?

[jedie]

I can reproduce them with suds-community but not with suds-jurko

Yes! That's it. So there are some breaking changes in suds-community

[jedie]

Only happening with "suds-community" but not with "suds-jurko", see also: https://github.com/django-oscar/django-oscar-docdata/issues/15

So, i close this issues.

[maerteijn]

👍