search

marklogic / entity-services

Data modeling and code scaffolding for data integration in MarkLogic
https://docs.marklogic.com/guide/entity-services
Apache License 2.0
7 stars 10 forks source link

Issue #339 uncovered an issue with extracting empty values from JSON #347

Closed grechaw closed 7 years ago

grechaw commented 7 years ago

This bugfix takes care of a mistake I made when refactoring instance functions. Previously it had been simple to extract values from source JSON or XML, but the refactoring made JSON sources with arrays problematic.

With this change, tests are all good. The material change is that, previously, this empty pointer

<prop><OrderDetails/> </prop>

would be rendered in JSON as {"OrderDetails": {}}. Now it contains an empty string {"OrderDetails":""}.

If in deed there were a 'null' for a given property, then this pointer should not exist all: <prop/>

grechaw commented 7 years ago

Code for b57cb22 was reviewed by Erik.

@bsrikan This PR is ready for you. You may find you need to make minor changes, if you have empty objects in test cases.

bsrikan commented 7 years ago

Started working on it. Pushed couple of key updates for 339 and 345 in my branch. Working on example verification and e2e.

bsrikan commented 7 years ago

Get below error for example-person mldeploy. Verified in build: 9.0-20170911 with example-updates changes

Exception in thread "ThreadPoolTaskExecutor-1" com.marklogic.client.FailedRequestException: Local message: failed to apply resource at documents: Bad Request. Server Message: XDMP-DOCUTF8SEQ: xdmp:get-request-part-body("text") -- Invalid UTF-8 escape sequence at  line 1 -- document is not UTF-8 encoded
        at com.marklogic.client.impl.JerseyServices.checkStatus(JerseyServices.java:4626)
        at com.marklogic.client.impl.JerseyServices.postResource(JerseyServices.java:3594)
        at com.marklogic.client.impl.JerseyServices.postBulkDocuments(JerseyServices.java:3677)
        at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:564)
        at com.marklogic.client.impl.GenericDocumentImpl.write(GenericDocumentImpl.java:23)
        at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:557)
        at com.marklogic.client.impl.GenericDocumentImpl.write(GenericDocumentImpl.java:23)
        at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:541)
        at com.marklogic.client.impl.GenericDocumentImpl.write(GenericDocumentImpl.java:23)
        at com.marklogic.client.batch.RestBatchWriter$1.run(RestBatchWriter.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
:entity-services-examples:mlPostDeploy UP-TO-DATE
:entity-services-examples:mlDeploy
bsrikan commented 7 years ago

Races example deploy OK. QC workspace tab 11,12 and 13 needs update.

  1. Tab11: Must replace "/ext/Race-0.0.1.xqy" with "/ext/entity-services/Race-0.0.1.xqy" set Database to entity-services-content

  2. Tab12: Must replace "/ext/versions-translator.xqy" with "/ext/entity-services/translator.xqy" Must replace "/ext/Race-0.0.2.xqy" with "/ext/entity-services/Race-0.0.2.xqy" set Database to entity-services-content

  3. Tab13: Must set Database to entity-services-content Getting error XDMP-NONMIXEDCOMPLEXCONT: fn:data(fn:doc("/upconverts//races/data/race-data/races/10k1.xml")/es:envelope/es:instance/Race/raceCategory) -- Node has complex type with non-mixed complex content

bsrikan commented 7 years ago

Versions and suppliers are OK.

grechaw commented 7 years ago

great, I'll make those updates. confused by mldeploy error, I must have missed something or not done it on a fresh install.

grechaw commented 7 years ago

let's chat about your mlDeploy error -- no repro from my side, so need more information.

Making qconsole changes now.

grechaw commented 7 years ago

QConsole fix has been added to PR/branch.

bsrikan commented 7 years ago

QConsole changes for races are working fine. Adding my changes to functional tests for bug 339 and 345 to merged with this PR. Then entire thing can be merged to develop/b9_0

bsrikan commented 7 years ago

https://github.com/grechaw/entity-services/pull/9 is my PR to merge with this one.

grechaw commented 7 years ago

merged PR into this one. Will ask mr triage for goahead