craig-willis commented 2 years ago

~~D1 has renamed cn-stage-2.test.dataone.org to cn-sandbox.test.dataone.org. The hostname is redirecting and the cert is only valid on cn-sandbox.test.dataone.org.~~

See comment below. D1 has consolidated cn-stage-2 and cn-stage, of which dev.nceas.ucsb.edu is now a member node.

To Test:

Requires https://github.com/whole-tale/gwvolman/pull/168
Create a simple tale
Go to Settings, connect cn-stage.test.dataone.org
Publish your tale to https://dev.nceas.ucsb.edu/
Confirm that the resulting published dataset looks sensible
For grins, change something and publish again

codecov[bot] commented 2 years ago

Codecov Report

Merging #528 (785d088) into master (5252f49) will increase coverage by 0.04%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #528      +/-   ##
==========================================
+ Coverage   92.84%   92.88%   +0.04%     
==========================================
  Files          58       58              
  Lines        4513     4511       -2     
==========================================
  Hits         4190     4190              
+ Misses        323      321       -2

Impacted Files	Coverage Δ
server/constants.py	`89.13% <ø> (ø)`
server/lib/__init__.py	`100.00% <ø> (+1.92%)`	:arrow_up:
server/lib/dataone/fakeoauth.py	`100.00% <ø> (ø)`
server/lib/dataone/__init__.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 5252f49...785d088. Read the comment docs.

ThomasThelen commented 2 years ago

I checked with one of the DataONE devs and confirmed that cn-stage-2 was removed and consolidated down to just cn-stage. I confirmed this at the following URL, which lists dev.nceas.ucsb.edu as one of its member nodes, https://cn-stage.test.dataone.org/cn/v2/node/urn:node:mnTestKNB

So instead of https://cn-sandbox.test.dataone.org/cn/v2, the URL should be https://cn-stage.test.dataone.org/cn/v2

To double check, I logged into dev.nceas and checked the network tab for the /node endpoint and confirmed that it was pointing to https://cn-stage.test.dataone.org/cn/v2

Xarthisius commented 2 years ago

Thanks @ThomasThelen !

craig-willis commented 2 years ago

I've updated and both register from and publish to dev.nceas are failing for me.

Registering https://dashboard.local.wholetale.org/browse?api=https%3A%2F%2Fcn-stage.test.dataone.org%2Fcn%2Fv2&name=Dataone%20Dataset&uri=https://dev.nceas.ucsb.edu/view/doi:10.5072/FK2K075M25 results in a file named FK2K075M25 .

Do we even allow users to register from dev.nceas? I think not -- this should probably be dropped from testing.

Publishing to dev.nceas results in an error due to unknown manifest-sha1.txt:

[2022-03-02 19:16:16,542: ERROR/ForkPoolWorker-2] There was an error while uploading manifest-sha1.txt
(<d1_common.types.generated.dataoneTypes_v2_0.SystemMetadata object at 0x7fb519f8c3d0>, <pyxb.utils.fac.Configuration object at 0x7fb519f7c910>, [<pyxb.binding.basis.ElementContent object at 0x7fb519f13b20>, <pyxb.binding.basis.ElementContent object at 0x7fb519f236a0>], {<pyxb.binding.content.ElementDeclaration object at 0x7fb5197d7fa0>: [<d1_common.types.generated.dataoneTypes_v1.Checksum object at 0x7fb519ee4f40>], <pyxb.binding.content.ElementDeclaration object at 0x7fb5197dc070>: [<d1_common.types.generated.dataoneTypes_v1.Subject object at 0x7fb519f8faf0>], <pyxb.binding.content.ElementDeclaration object at 0x7fb5197dc100>: [<d1_common.types.generated.dataoneTypes_v1.Subject object at 0x7fb519f8f1f0>], <pyxb.binding.content.ElementDeclaration object at 0x7fb5197dc190>: [<d1_common.types.generated.dataoneTypes_v1.AccessPolicy object at 0x7fb519f807c0>], <pyxb.binding.content.ElementDeclaration object at 0x7fb519730940>: ['manifest-sha1.txt']})
[2022-03-02 19:16:16,984: ERROR/ForkPoolWorker-2] Task gwvolman.tasks.publish[227b0f1e-4ee2-4139-9599-6878808b1013] raised unexpected: ValueError('There was a fatal error while uploading manifest-sha1.txt. Please contact the support team.')
Traceback (most recent call last):
  File "/gwvolman/gwvolman/lib/dataone/publish.py", line 251, in publish
    self._upload_file(
  File "/gwvolman/gwvolman/lib/dataone/publish.py", line 392, in _upload_file
    self.client.create(pid, file_object, system_metadata)
  File "/usr/local/lib/python3.8/dist-packages/d1_client/mnclient.py", line 107, in create
    response = self.createResponse(
  File "/usr/local/lib/python3.8/dist-packages/d1_client/mnclient.py", line 102, in createResponse
    "sysmeta": ("sysmeta.xml", sysmeta_pyxb.toxml("utf-8")),
  File "/usr/local/lib/python3.8/dist-packages/pyxb/binding/basis.py", line 560, in toxml
    dom = self.toDOM(bds, element_name=element_name)
  File "/usr/local/lib/python3.8/dist-packages/pyxb/binding/basis.py", line 532, in toDOM
    self._toDOM_csc(bds, element)
  File "/usr/local/lib/python3.8/dist-packages/pyxb/binding/basis.py", line 2680, in _toDOM_csc
    order = self._validatedChildren()
  File "/usr/local/lib/python3.8/dist-packages/pyxb/binding/basis.py", line 2207, in _validatedChildren
    return self.__automatonConfiguration.sequencedChildren()
  File "/usr/local/lib/python3.8/dist-packages/pyxb/binding/content.py", line 638, in sequencedChildren
    cfg = self._diagnoseIncompleteContent(symbols, symbol_set)
  File "/usr/local/lib/python3.8/dist-packages/pyxb/binding/content.py", line 493, in _diagnoseIncompleteContent
    raise pyxb.IncompleteElementContentError(self.__instance, cfg, symbols, symbol_set)
pyxb.exceptions_.IncompleteElementContentError: (<d1_common.types.generated.dataoneTypes_v2_0.SystemMetadata object at 0x7fb519f8c3d0>, <pyxb.utils.fac.Configuration object at 0x7fb519f7c910>, [<pyxb.binding.basis.ElementContent object at 0x7fb519f13b20>, <pyxb.binding.basis.ElementContent object at 0x7fb519f236a0>], {<pyxb.binding.content.ElementDeclaration object at 0x7fb5197d7fa0>: [<d1_common.types.generated.dataoneTypes_v1.Checksum object at 0x7fb519ee4f40>], <pyxb.binding.content.ElementDeclaration object at 0x7fb5197dc070>: [<d1_common.types.generated.dataoneTypes_v1.Subject object at 0x7fb519f8faf0>], <pyxb.binding.content.ElementDeclaration object at 0x7fb5197dc100>: [<d1_common.types.generated.dataoneTypes_v1.Subject object at 0x7fb519f8f1f0>], <pyxb.binding.content.ElementDeclaration object at 0x7fb5197dc190>: [<d1_common.types.generated.dataoneTypes_v1.AccessPolicy object at 0x7fb519f807c0>], <pyxb.binding.content.ElementDeclaration object at 0x7fb519730940>: ['manifest-sha1.txt']})

After resolving that, publish creates a malformed dataset (but succeeds), which is now fixed by https://github.com/whole-tale/gwvolman/pull/168.

whole-tale / girder_wholetale

Change staging cn to cn-stage #528

Codecov Report