Open holtkamp opened 9 years ago
This should be worked over, I'm stumbling into the same problem.
Can you provide an example XML too? Are you using my jms serializer fork, right?
I'm also running into this issue...any suggestions for how to resolve this, without having to manually handle this?
Quoting myself
Can you provide an example XML too? Are you using my jms serializer fork, right?
I can't provide the xsd as it's not public, but looks like @holtkamp provided a sample. I am using your fork of the jms serializer.
By commenting out the xml-element:namespace
, things begin to work, but this seems to have an adverse effect on nested children elements. I receive a PHP warning stating Node no longer exists
and cites lines 233 and 251 of XMLDeserializationVisitor.php
Ok, also for my own administration an example for integrating with the iDEAL Payment system. Apologies for the long comment, but I prefer to be complete ;).
XSD2PHP Converting XSD to PHP objects to be populate using based on this XML Schema File:
php bin/xsd2php convert:php https://raw.githubusercontent.com/juriansluiman/SlmIdealPayment/master/data/xsd/AcceptantAcquirer.xsd --ns-map='http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1;Ideal/Messages/' --ns-map='http://www.w3.org/2000/09/xmldsig#;XmlDigitalSignature' --ns-dest='Ideal/Messages/;src/Ideal/Messages/' --ns-dest='XmlDigitalSignature/;src/W3/'
XSD2YML Converting XSD to YML metadata to be used by JMS Serializer using based on this XML Schema File:
php bin/xsd2php convert:jms-yaml https://raw.githubusercontent.com/juriansluiman/SlmIdealPayment/master/data/xsd/AcceptantAcquirer.xsd --ns-map='http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1;Ideal/Messages/' --ns-map='http://www.w3.org/2000/09/xmldsig#;XmlDigitalSignature' --ns-dest='Ideal/Messages/;yml/Ideal/Messages/' --ns-dest='XmlDigitalSignature/;yml/W3/'
Results in the YAML metadata that will be used by Serializer (note the XML namespace of the last element, this should be different than the default namespace, coming to that later):
Ideal\Messages\DirectoryReq:
xml_root_name: DirectoryReq
xml_root_namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
properties:
version:
expose: true
access_type: public_method
serialized_name: version
accessor:
getter: getVersion
setter: setVersion
xml_attribute: true
type: string
createDateTimestamp:
expose: true
access_type: public_method
serialized_name: createDateTimestamp
xml_element:
namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
accessor:
getter: getCreateDateTimestamp
setter: setCreateDateTimestamp
type: Goetas\Xsd\XsdToPhp\XMLSchema\DateTime
merchant:
expose: true
access_type: public_method
serialized_name: Merchant
xml_element:
namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
accessor:
getter: getMerchant
setter: setMerchant
type: Ideal\Messages\DirectoryReq\MerchantAType
signature:
expose: true
access_type: public_method
serialized_name: Signature
xml_element:
namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
accessor:
getter: getSignature
setter: setSignature
type: XmlDigitalSignature\Signature
XML2PHP No lets try to deserialize the following XML instance using JML Serializer:
<?xml version="1.0" encoding="utf-8"?>
<!--URL: https://idealtest.rabobank.nl/ideal/iDEALv3-->
<DirectoryReq xmlns="http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1" version="3.3.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1 https://raw.githubusercontent.com/juriansluiman/SlmIdealPayment/master/data/xsd/AcceptantAcquirer.xsd http://www.w3.org/2000/09/xmldsig# https://raw.githubusercontent.com/cloudregistry/EPP-Launch-Phase-Extension-Specification/master/xsd/xmldsig-core-schema.xsd">
<createDateTimestamp>2013-06-20T18:54:36.000Z</createDateTimestamp>
<Merchant>
<merchantID>002093965</merchantID>
<subID>0</subID>
</Merchant>
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
<CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
<SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" />
<Reference URI="">
<Transforms>
<Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
</Transforms>
<DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
<DigestValue>0R00+jQyZlSLrTA+1gSmsH9vtbzbNbIIoWzh9SDJXGc=</DigestValue>
</Reference>
</SignedInfo>
<SignatureValue>
H5FBG+1ceWVginZdK1bT77g1HL0p8/OXJ1FTw2QWNBOzkKgg1q0L7OvbQTsW3i+RP1TVfvXr35X+C3KUQbZNrnm 59w6J765T+XMpNyPCU5MgN6aQTV1SqXQMFpDlsdxMxeDHamRwoKs+g9hga/mTUv5xEedkVhTZGuQ2aiUeAec=
</SignatureValue>
<KeyInfo>
<KeyName>7D665C81ABBE1A7D0E525BFC171F04D276F07BF2</KeyName>
</KeyInfo>
</Signature>
</DirectoryReq>
With Serializer 0.16
this results in PHP object where only the XSD simpleTypes seem to be populated:
Ideal\Messages\DirectoryReq Object
(
[version:Ideal\Messages\DirectoryReq:private] => 3.3.1
[createDateTimestamp:Ideal\Messages\DirectoryReq:private] =>
[merchant:Ideal\Messages\DirectoryReq:private] =>
[signature:Ideal\Messages\DirectoryReq:private] =>
)
With Serializer 0.16
and removing the extra XML namespace:
xml_element:
namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
this results in a PHP object that also has the XSD complexTypes populated:
Ideal\Messages\DirectoryReq Object
(
[version:Ideal\Messages\DirectoryReq:private] => 3.3.1
[createDateTimestamp:Ideal\Messages\DirectoryReq:private] => DateTime Object
(
[date] => 2013-06-20 18:54:36.000000
[timezone_type] => 3
[timezone] => UTC
)
[merchant:Ideal\Messages\DirectoryReq:private] => Ideal\Messages\DirectoryReq\MerchantAType Object
(
[merchantID:Ideal\Messages\DirectoryReq\MerchantAType:private] => 002093965
[subID:Ideal\Messages\DirectoryReq\MerchantAType:private] => 0
)
[signature:Ideal\Messages\DirectoryReq:private] => XmlDigitalSignature\Signature Object
(
[id:XmlDigitalSignature\SignatureType:private] =>
[signedInfo:XmlDigitalSignature\SignatureType:private] => XmlDigitalSignature\SignedInfo Object
(
[id:XmlDigitalSignature\SignedInfoType:private] =>
[canonicalizationMethod:XmlDigitalSignature\SignedInfoType:private] => XmlDigitalSignature\CanonicalizationMethod Object
(
[algorithm:XmlDigitalSignature\CanonicalizationMethodType:private] => http://www.w3.org/2001/10/xml-exc-c14n#
)
[signatureMethod:XmlDigitalSignature\SignedInfoType:private] => XmlDigitalSignature\SignatureMethod Object
(
[algorithm:XmlDigitalSignature\SignatureMethodType:private] => http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
[hMACOutputLength:XmlDigitalSignature\SignatureMethodType:private] =>
)
[reference:XmlDigitalSignature\SignedInfoType:private] => Array
(
[0] => XmlDigitalSignature\Reference Object
(
[id:XmlDigitalSignature\ReferenceType:private] =>
[uRI:XmlDigitalSignature\ReferenceType:private] =>
[type:XmlDigitalSignature\ReferenceType:private] =>
[transforms:XmlDigitalSignature\ReferenceType:private] => Array
(
[0] => XmlDigitalSignature\Transform Object
(
[algorithm:XmlDigitalSignature\TransformType:private] => http://www.w3.org/2000/09/xmldsig#enveloped-signature
[xPath:XmlDigitalSignature\TransformType:private] => Array
(
)
)
)
[digestMethod:XmlDigitalSignature\ReferenceType:private] => XmlDigitalSignature\DigestMethod Object
(
[algorithm:XmlDigitalSignature\DigestMethodType:private] => http://www.w3.org/2001/04/xmlenc#sha256
)
[digestValue:XmlDigitalSignature\ReferenceType:private] => XmlDigitalSignature\DigestValue Object
(
[__value:XmlDigitalSignature\DigestValue:private] => 0R00+jQyZlSLrTA+1gSmsH9vtbzbNbIIoWzh9SDJXGc=
)
)
)
)
[signatureValue:XmlDigitalSignature\SignatureType:private] => XmlDigitalSignature\SignatureValue Object
(
[__value:XmlDigitalSignature\SignatureValueType:private] =>
H5FBG+1ceWVginZdK1bT77g1HL0p8/OXJ1FTw2QWNBOzkKgg1q0L7OvbQTsW3i+RP1TVfvXr35X+C3KUQbZNrnm 59w6J765T+XMpNyPCU5MgN6aQTV1SqXQMFpDlsdxMxeDHamRwoKs+g9hga/mTUv5xEedkVhTZGuQ2aiUeAec=
[id:XmlDigitalSignature\SignatureValueType:private] =>
)
[keyInfo:XmlDigitalSignature\SignatureType:private] => XmlDigitalSignature\KeyInfo Object
(
[id:XmlDigitalSignature\KeyInfoType:private] =>
[keyName:XmlDigitalSignature\KeyInfoType:private] => Array
(
[0] => 7D665C81ABBE1A7D0E525BFC171F04D276F07BF2
)
[keyValue:XmlDigitalSignature\KeyInfoType:private] => Array
(
)
[retrievalMethod:XmlDigitalSignature\KeyInfoType:private] => Array
(
)
[x509Data:XmlDigitalSignature\KeyInfoType:private] => Array
(
)
[pGPData:XmlDigitalSignature\KeyInfoType:private] => Array
(
)
[sPKIData:XmlDigitalSignature\KeyInfoType:private] => Array
(
)
[mgmtData:XmlDigitalSignature\KeyInfoType:private] => Array
(
)
)
[object:XmlDigitalSignature\SignatureType:private] => Array
(
)
)
)
With the suggested Serializer "jms/serializer": "xsd2php-dev as 0.18.0"
we got a apparently properly populated PHP object:
Ideal\Messages\DirectoryReq Object
(
[version:Ideal\Messages\DirectoryReq:private] => 3.3.1
[createDateTimestamp:Ideal\Messages\DirectoryReq:private] => DateTime Object
(
[date] => 2013-06-20 18:54:36.000000
[timezone_type] => 2
[timezone] => Z
)
[merchant:Ideal\Messages\DirectoryReq:private] => Ideal\Messages\DirectoryReq\MerchantAType Object
(
[merchantID:Ideal\Messages\DirectoryReq\MerchantAType:private] => 002093965
[subID:Ideal\Messages\DirectoryReq\MerchantAType:private] => 0
)
[signature:Ideal\Messages\DirectoryReq:private] =>
)
Only, as we can see with "jms/serializer": "xsd2php-dev as 0.18.0"
that at least the <merchant/> element is processed properly. But another issue arises. It seems the XML namespace of the
element is incorrect in the generated YAML, it should be
http://www.w3.org/2000/09/xmldsig#```, therefore that property is not detected and populated properly.
Changing that part fixes everything:
Ideal\Messages\DirectoryReq:
xml_root_name: DirectoryReq
xml_root_namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
properties:
version:
expose: true
access_type: public_method
serialized_name: version
accessor:
getter: getVersion
setter: setVersion
xml_attribute: true
type: string
createDateTimestamp:
expose: true
access_type: public_method
serialized_name: createDateTimestamp
xml_element:
namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
accessor:
getter: getCreateDateTimestamp
setter: setCreateDateTimestamp
type: Goetas\Xsd\XsdToPhp\XMLSchema\DateTime
merchant:
expose: true
access_type: public_method
serialized_name: Merchant
xml_element:
namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
accessor:
getter: getMerchant
setter: setMerchant
type: Ideal\Messages\DirectoryReq\MerchantAType
signature:
expose: true
access_type: public_method
serialized_name: Signature
xml_element:
namespace: 'http://www.w3.org/2000/09/xmldsig#'
accessor:
getter: getSignature
setter: setSignature
type: XmlDigitalSignature\Signature
Resulting PHP:
Ideal\Messages\DirectoryReq Object
(
[version:Ideal\Messages\DirectoryReq:private] => 3.3.1
[createDateTimestamp:Ideal\Messages\DirectoryReq:private] => DateTime Object
(
[date] => 2013-06-20 18:54:36.000000
[timezone_type] => 2
[timezone] => Z
)
[merchant:Ideal\Messages\DirectoryReq:private] => Ideal\Messages\DirectoryReq\MerchantAType Object
(
[merchantID:Ideal\Messages\DirectoryReq\MerchantAType:private] => 002093965
[subID:Ideal\Messages\DirectoryReq\MerchantAType:private] => 0
)
[signature:Ideal\Messages\DirectoryReq:private] => XmlDigitalSignature\Signature Object
(
[id:XmlDigitalSignature\SignatureType:private] =>
[signedInfo:XmlDigitalSignature\SignatureType:private] => XmlDigitalSignature\SignedInfo Object
(
[id:XmlDigitalSignature\SignedInfoType:private] =>
[canonicalizationMethod:XmlDigitalSignature\SignedInfoType:private] => XmlDigitalSignature\CanonicalizationMethod Object
(
[algorithm:XmlDigitalSignature\CanonicalizationMethodType:private] => http://www.w3.org/2001/10/xml-exc-c14n#
)
[signatureMethod:XmlDigitalSignature\SignedInfoType:private] => XmlDigitalSignature\SignatureMethod Object
(
[algorithm:XmlDigitalSignature\SignatureMethodType:private] => http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
[hMACOutputLength:XmlDigitalSignature\SignatureMethodType:private] =>
)
[reference:XmlDigitalSignature\SignedInfoType:private] => Array
(
[0] => XmlDigitalSignature\Reference Object
(
[id:XmlDigitalSignature\ReferenceType:private] =>
[uRI:XmlDigitalSignature\ReferenceType:private] =>
[type:XmlDigitalSignature\ReferenceType:private] =>
[transforms:XmlDigitalSignature\ReferenceType:private] => Array
(
[0] => XmlDigitalSignature\Transform Object
(
[algorithm:XmlDigitalSignature\TransformType:private] => http://www.w3.org/2000/09/xmldsig#enveloped-signature
[xPath:XmlDigitalSignature\TransformType:private] =>
)
)
[digestMethod:XmlDigitalSignature\ReferenceType:private] => XmlDigitalSignature\DigestMethod Object
(
[algorithm:XmlDigitalSignature\DigestMethodType:private] => http://www.w3.org/2001/04/xmlenc#sha256
)
[digestValue:XmlDigitalSignature\ReferenceType:private] => XmlDigitalSignature\DigestValue Object
(
[__value:XmlDigitalSignature\DigestValue:private] => 0R00+jQyZlSLrTA+1gSmsH9vtbzbNbIIoWzh9SDJXGc=
)
)
)
)
[signatureValue:XmlDigitalSignature\SignatureType:private] => XmlDigitalSignature\SignatureValue Object
(
[__value:XmlDigitalSignature\SignatureValueType:private] =>
H5FBG+1ceWVginZdK1bT77g1HL0p8/OXJ1FTw2QWNBOzkKgg1q0L7OvbQTsW3i+RP1TVfvXr35X+C3KUQbZNrnm 59w6J765T+XMpNyPCU5MgN6aQTV1SqXQMFpDlsdxMxeDHamRwoKs+g9hga/mTUv5xEedkVhTZGuQ2aiUeAec=
[id:XmlDigitalSignature\SignatureValueType:private] =>
)
[keyInfo:XmlDigitalSignature\SignatureType:private] => XmlDigitalSignature\KeyInfo Object
(
[id:XmlDigitalSignature\KeyInfoType:private] =>
[keyName:XmlDigitalSignature\KeyInfoType:private] => Array
(
[0] => 7D665C81ABBE1A7D0E525BFC171F04D276F07BF2
)
[keyValue:XmlDigitalSignature\KeyInfoType:private] =>
[retrievalMethod:XmlDigitalSignature\KeyInfoType:private] =>
[x509Data:XmlDigitalSignature\KeyInfoType:private] =>
[pGPData:XmlDigitalSignature\KeyInfoType:private] =>
[sPKIData:XmlDigitalSignature\KeyInfoType:private] =>
[mgmtData:XmlDigitalSignature\KeyInfoType:private] =>
)
[object:XmlDigitalSignature\SignatureType:private] =>
)
)
Concluding, the workaround is to use the Goetas fork of JMS Serializer, we should try to get https://github.com/schmittjoh/serializer/pull/301 merged.
Regarding the generation of the YAML files, this might be a new issue? XML Schema Definitions that involve multiple namespaces are not recognized properly? Looking at this line in the XSD, we see that the element is referenced, maybe this is not supported yet by the XSD2YML operation?:
<xs:element ref="ds:Signature"/>
PS: note that a CustomTypeHandler is required to properly deal with the XSD:SimpleType and XSD:AnySimpleType types in the XML Digital Signing schema, also described in the documentation
So I'm having some issues with nested namespaces as well. The deserializer seems to be fine with an element like this:
<ns1:Root xmlns:n1="..." xmlns:ns2="..." xmlns:ns3="...">
<blah>...</blah>
<blah2>...</blah2>
<blah3>...</blah3>
</ns1:Root>
but chokes on something like this:
<ns1:Root xmlns:n1="..." xmlns:ns2="..." xmlns:ns3="...">
<ns2:blah>...</ns2:blah>
<ns3:blah2>...</ns3:blah2>
<ns2:blah3>...</ns2:blah3>
</ns1:Root>
When implementing the the fix mentioned above (remove xml_element:namespace) the first example works fine (it doesn't without this fix). However, the second example just leaves blank nodes like:
<Root>
<blah/>
<blah2/>
<blah3/>
</Root>
and shows numerous errors
PHP Warning: JMS\Serializer\XmlDeserializationVisitor::visitProperty(): Node no longer exists in .../vendor/jms/serializer/src/JMS/Serializer/XmlDeserializationVisitor.php on line 233
PHP Warning: JMS\Serializer\XmlDeserializationVisitor::visitProperty(): Node no longer exists in .../vendor/jms/serializer/src/JMS/Serializer/XmlDeserializationVisitor.php on line 251
Just to rule things out, are you sure you "feed" the Serializer with the metadata for all involved namespaces? Otherwise the Serializer might not know how to deal with them...
$serializerBuilder = SerializerBuilder::create();
$serializerBuilder->setMetadataDirs(
array(
'Vendor1\MessageNamespace1' => APPLICATION_ROOT.'/yml/Vendor1/Messages',
'Vendor2\MessageNamespace2' => APPLICATION_ROOT.'/yml/Vendor2/Messages',
)
);
Was hoping I was doing something wrong there, but I definitely am seeding the serializer correctly.
$serializerBuilder = SerializerBuilder::create();
$serializerBuilder->addMetadataDir('src/jms/path1','Path1');
$serializerBuilder->addMetadataDir('src/jms/path1','Path1\sub-path2');
$serializerBuilder->addMetadataDir('src/jms/path2/sub-path1','Path2\sub-path1');
$serializerBuilder->addMetadataDir('src/jms/path2/sub-path2','Path2\sub-path2');
$serializerBuilder->addMetadataDir('src/jms/path2/sub-path3','Path2\sub-path3');
$serializerBuilder->addMetadataDir('src/jms/path3/sub-path1','Path3\sub-path1');
I was really hoping this project was going to help with creating the classes based on the XSD (which it seems it does fine), but the serialization aspect was very important, but seems to be broken, or I'm just not doing it right...
Unfortunately, I can't share the XSDs as they are not public, but in general, there are 5 namespaces and some elements contain elements from a different namespace as previously shown above...
@holtkamp - maybe I am confused. where are you removing the:
xml_element:
namespace: 'http://www.idealdesk.com/ideal/messages/mer-acq/3.3.1'
from in the yaml files?
So...I looked again and figured out where you are doing that. It would seem that the fix includes not having xml_element:namespace
in the yaml file if the element's namespace is the same as the parent? Can this be added into the library?
yes indeed, in the YAML files, it seems when generating these YAML metadata files, not "all" namespaces are recognized correctly. You can try to generate the YAML files and then manually check whether these xml_element:namespace fields have the proper value. I suspect something goes wrong there when multiple namespaces are involved / elements are referenced in the XML Schema file.
Maybe you can try to isolate the part of the XML Schema file that goes wrong, otherwise the author of this library is still searching for the needle in the haystack ;).
On 26 September 2015 at 22:03, kevinmlong notifications@github.com wrote:
So...I looked again and figured out where you are doing that. It would seem that the fix includes not having xml_element:namespace in the yaml file if the element's namespace is the same as the parent? Can this be added into the library?
— Reply to this email directly or view it on GitHub https://github.com/goetas/xsd2php/issues/73#issuecomment-143491296.
i spent some time trying to reproduce the bug... but to me currently it looks working. any change to submit a failing test case?
When generating YAML metadata files to be used with JMS Serializer, it seems that for EACH element the XML namespace is defined in the resulting YAML files. Also for nested elements that already 'inherit' their namespace from the root namespace.
For nested elements this seems to confuse the JMS Serializer => the element is not properly detected and the property remains empty. By removing the XML namespace JMS Serializer detects the element again.
For example:
The
createDateTimestamp
property will not be populated properly.By removing the snippet
serialization works again and the
createDateTimestamp
property is properly populated with a DateTime object.The used XSD is here: https://github.com/juriansluiman/SlmIdealPayment/blob/master/data/xsd/AcceptantAcquirer.xsd, can provide example XML instances if required