marklogic / entity-services

Data modeling and code scaffolding for data integration in MarkLogic
https://docs.marklogic.com/guide/entity-services
Apache License 2.0
7 stars 10 forks source link

Update schema for 9.0-3 entity type changes #313

Closed grechaw closed 7 years ago

grechaw commented 7 years ago

The new vocabulary items in the entity services model need to be added to the XSD.

grechaw commented 7 years ago

Seems the schema was not updated even for the 9.0-1 release. There are bugs in there. This task is now a little bigger, to correct and make the schema valid for XML model descriptors.

grechaw commented 7 years ago

Also, add unit test for validating envelopes. The bug in the schema has to do with envelopes.

grechaw commented 7 years ago

I've got this change ready to go in the svn checkout for b9 and head. Here's the patch:

cgreer@fedor:/MarkLogic/git/b9_0$ git diff HEAD~
diff --git a/src/Config/entity-type.xsd b/src/Config/entity-type.xsd
index 37a827c..dcfc2dd 100644
--- a/src/Config/entity-type.xsd
+++ b/src/Config/entity-type.xsd
@@ -34,18 +34,23 @@
       </xs:sequence>
   </xs:complexType>

-  <xs:complexType name="EntitySourcesType">
+  <xs:complexType name="AttachmentsType">
       <xs:sequence>
           <xs:any processContents="skip" minOccurs="0" maxOccurs="unbounded"/>
       </xs:sequence>
   </xs:complexType>

-  <xs:complexType name="EntityEnvelopeType">
+  <xs:complexType name="EnvelopeType">
     <xs:sequence>
-        <xs:element ref="es:id"/>
-        <xs:element ref="es:document-uri"/>
+        <xs:element ref="es:instance" maxOccurs="unbounded"/>
+        <xs:element ref="es:attachments" minOccurs="0"/>
+    </xs:sequence>
+  </xs:complexType>
+
+  <xs:complexType name="InstanceType">
+    <xs:sequence>
+        <xs:element ref="es:info"/>
         <xs:any processContents="skip"/>
-        <xs:element ref="es:sources"/>
     </xs:sequence>
   </xs:complexType>

@@ -68,9 +73,8 @@
   <xs:element name="primary-key" type="xs:string"/>

   <!-- used in instance documents -->
-  <xs:element name="instance" type="es:EntityEnvelopeType"/>
-  <xs:element name="sources" type="es:EntitySourcesType"/>
-  <xs:element name="id"/>
-  <xs:element name="document-uri"/>
+  <xs:element name="envelope" type="es:EnvelopeType"/>
+  <xs:element name="instance" type="es:InstanceType"/>
+  <xs:element name="attachments" type="es:AttachmentsType"/>
grechaw commented 7 years ago

@bsrikan can you take a look at this issue and see that you think it's acceptable before I check it in? I've got the change for b9 and trunk ready to commit.

grechaw commented 7 years ago

I checked in the change to svn, and a test had to be adjusted because of it. That test is in the bugfixes PR now.

bsrikan commented 7 years ago

I will get to it later today. Finishing up on some 8.0-7 tasks.

bsrikan commented 7 years ago

When validating instance-to-envelope() got following error:

XDMP-VALIDATEUNEXPECTED: (err:XQDY0027) validate strict { nwind:instance-to-envelope($superstore-e) } -- Invalid node: Found text{"{"RowID":"1944", "Or..."} but expected (any(skip,!())*) at fn:doc("")/es:envelope/es:attachments/text() using schema "entity-type.xsd"

QUERY:

import module namespace nwind = "http://marklogic.com/test#Northwind-0.0.1" at "/ext/Northwind-0.0.1.xqy";

let $source := doc("/entity-services-e2e/e2e-nwind/data/third-party/csv/superstore.csv-1086.json")
let $superstore-e := nwind:extract-instance-Superstore($source)
return
(
  validate strict { nwind:instance-to-envelope($superstore-e, 'xml') }
)
grechaw commented 7 years ago

It looks like the definition of es:attachments does not allow for text as it should. Probably I simply need to add a 'mixed' attribute to this element definition.

grechaw commented 7 years ago

sending reviewed patch to mr triage.

bsrikan commented 7 years ago

Able to deploy e2e just fine now. mixed attribute for attachments seems to work. Shipping the issue.