search

usnistgov / oar-pdr

The NIST Open Access to Research (OAR) Public Data Repository (PDR) system software
11 stars 10 forks source link

ODD-939: Properly display references on landing page #165

Closed RayPlante closed 3 years ago

RayPlante commented 3 years ago

This PR is one part of the solution addressing ODD-926, "Data Supplemental Paper Reference" (especially sub-task ODD-939). Its aim is to properly render references added via MIDAS on the landing page. It builds on #164 which ensures that references from MIDAS are properly labeled (via the refType field). This PR ensures references appear in two possible locations:

This PR can be tested via a special branch of oar-docker, ODD926-test-refs, using the following process:

  1. After checking out the test branch, change into its demo application folder.
  2. Build the demo application using the usual process (using localdeploy and oarctl). Note that the deployment.yml file is already updated to build the various branches required for this test. If you intend to test the "editEnabled" view, note that you must also follow the special build instructions for the cusomtizationapi component; see the file, building-customization-api.md, for details.
  3. In the demo apps testing instructions given in README.md, follow steps 1 and 2 to see the landing page before a reference is included; note that the page will not include a references section.
  4. Edit the POD record located as data/midas/review/1965/_pod.json; insert the line, "references": [ "http://dx.doi.org/10.6028/jres.118.019" ], into the file.
  5. Push the updated metadata to the PDR by executing the script, demopublish/updatepod. Execute it a second time.
  6. Reload the landing page that was loaded as part of (3) above. You should now see the references section as well as a link to the article near the top of the page.
  7. To see that the references are properly displayed on the public side, run the script, demopublish/publish. Then access the public landing page at https://localhost/od/id/746FA083A3434449E05324570681A5DD1965.
chuanlin2018 commented 3 years ago

Step 2 failed on SDP and distservice. Any idea?

../scripts/localdeploy -f sdp

######################### #

Building sdp-1.3.1...

# #########################

angular@0.0.0 build /home/build/angular ng build --prod --aot --vendor-chunk --common-chunk --delete-output-path --buildOptimizer

Browserslist: caniuse-lite is outdated. Please run: npx browserslist@latest --update-db

Date: 2021-01-08T16:53:47.953Z Hash: 80c70fde2eee08ed21ad Time: 94856ms chunk {0} runtime.78ca88719dbb95d1a056.js (runtime) 1.41 kB [entry] [rendered] chunk {1} main.6d2b426ba3c72a4593b6.js (main) 335 kB [initial] [rendered] chunk {2} polyfills.948bc83c1f2e4618cdae.js (polyfills) 109 kB [initial] [rendered] chunk {3} styles.78440b2dc44481bf1392.css (styles) 324 kB [initial] [rendered] chunk {4} vendor.2b7b7631388c33f32159.js (vendor) 1.6 MB [initial] [rendered] chunk {scripts} scripts.8fc0bc9362abad241d12.js (scripts) 639 kB [entry] [rendered]

===================

../scripts/localdeploy -f distservice

[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:07 min [INFO] Finished at: 2021-01-08T17:52:27+00:00 [INFO] Final Memory: 26M/113M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project oar-dist-service: Could not resolve dependencies for project gov.nist.oar:oar-dist-service:jar:1.0.0-SNAPSHOT: Failed to collect dependencies at io.springfox:springfox-swagger-ui:jar:2.9.0: Failed to read artifact descriptor for io.springfox:springfox-swagger-ui:jar:2.9.0: Could not transfer artifact io.springfox:springfox-swagger-ui:pom:2.9.0 from/to spring-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized. -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException localbuild: oar-dist-service: build failure: Failed to build components from oar-dist-service localbuild: The following components failed to build: localbuild: oar-dist-service: build failure: Failed to build components from oar-dist-service localbuild: Some components not available due to build failures localdeploy: One or more needed components failed to build

RayPlante commented 3 years ago

The Java builds broke recently. I'm going to have to hot-patch the repos. I'll get back to you.

RayPlante commented 3 years ago

I've fixed the oar-docker ODD926-test-refs branch, I think. To fix your local branch, do the following:

  1. Inside your oar-docker repo (with ODD926-test-refs checked out), run git pull
  2. In the demo application directory, rerun ../scripts/localdeploy.
  3. Continue with the original step 2 and run oarctrl to finish building.
chuanlin2018 commented 3 years ago

After git pull, localdeploy still failed on oar-dist-service. Oar-config also failed the first time but succeed in the 2nd run.

##################################

Building oar-dist-service

##################################

[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:33 min [INFO] Finished at: 2021-01-12T15:46:08+00:00 [INFO] Final Memory: 25M/70M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project oar-dist-service: Could not resolve dependencies for project gov.nist.oar:oar-dist-service:jar:1.0.0-SNAPSHOT: Failed to collect dependencies at io.springfox:springfox-swagger-ui:jar:2.9.0: Failed to read artifact descriptor for io.springfox:springfox-swagger-ui:jar:2.9.0: Could not transfer artifact io.springfox:springfox-swagger-ui:pom:2.9.0 from/to spring-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized. -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException localbuild: oar-dist-service: build failure: Failed to build components from oar-dist-service

RayPlante commented 3 years ago

Oops, I know what I did wrong.... will fix.

RayPlante commented 3 years ago

Okay, try again? oar-dist-service is now set to the integration branch which includes the needed build fix. Please run git pull in your oar-docker ODD926-test-refs branch, and then continue with localdeploy again.

chuanlin2018 commented 3 years ago

Didn't notice any error in deployment (might have missed it since there were so many log messages) but build failed:

============================================================================ Testsuite summary for onig 6.1.3

TOTAL: 9

PASS: 9

SKIP: 0

XFAIL: 0

FAIL: 0

XPASS: 0

ERROR: 0

============================================================================ make[5]: Leaving directory '/root/jq-dev/modules/oniguruma/sample' make[4]: Leaving directory '/root/jq-dev/modules/oniguruma/sample' make[3]: Leaving directory '/root/jq-dev/modules/oniguruma/sample' make[3]: Entering directory '/root/jq-dev/modules/oniguruma' make[3]: Nothing to be done for 'check-am'. make[3]: Leaving directory '/root/jq-dev/modules/oniguruma' make[2]: Leaving directory '/root/jq-dev/modules/oniguruma' make[2]: Entering directory '/root/jq-dev' Changes to the manual.yml require docs to be enabled to generate an updated manpage As a result, your manpage is out of date. make tests/man.test make[3]: Entering directory '/root/jq-dev' Changes to the manual.yml require docs to be enabled to run the tests Makefile:1839: recipe for target 'tests/man.test' failed make[3]: Leaving directory '/root/jq-dev' make[3]: [tests/man.test] Error 1 make[2]: [check-am] Error 2 Makefile:1644: recipe for target 'check-am' failed make[2]: Leaving directory '/root/jq-dev' make[1]: [check-recursive] Error 1 Makefile:1151: recipe for target 'check-recursive' failed make[1]: Leaving directory '/root/jq-dev' make: [check] Error 2 Makefile:1647: recipe for target 'check' failed ERROR: Service 'nerdmbase' failed to build: The command '/bin/sh -c git clone http://github.com/stedolan/jq.git jq-dev && cd jq-dev && git checkout 6306ac89667cf35f47ddc40aa0630546c57e387f && git submodule update --init && autoreconf -fi && ./configure --prefix=/usr/local --with-oniguruma=builtin && make -j8 && make check && make install' returned a non-zero code: 2

RayPlante commented 3 years ago

Ugh. Okay, how about now? Do a git pull and rerun ../scripts/oarctrl local build.

chuanlin2018 commented 3 years ago

Finally I finished all 7 steps and the reference showed up as expected. One thing I don't quite understand was that the line I added in to the json file didn't contain attribute "refType", why the reference showed up at top portion of the page. I tried to add a console log to see the nerdm record (removed archive files and re-deploy) but then I was unable to bring down/up the docker: P932405:demo lnc9$ ../scripts/oarctl local down

ERROR: for sdp UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=70)

ERROR: for demo_intpdr_1 UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=70)

ERROR: for pub-pdr UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=70)

ERROR: for demo_pubserver_1 UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=70)

ERROR: for customizationapi UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=70) ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information. If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

I may rebuild the docker and test edit enabled mode later tonight or tomorrow.

RayPlante commented 3 years ago

One thing I don't quite understand was that the line I added in to the json file didn't contain attribute "refType", why the reference showed up at top portion of the page.

The input record that you edited (which normally would come from MIDAS) is a POD record, which is not as rich as the NERDm equivalent. When the POD record is converted to NERDm, the code must assume what the refType should be and assign it. If fact, changing that assignment was what I had to fix in the python code.

Testing the landing page editing is optional but probably a good idea--just to be safe. Thanks for finding all the build bugs!

chuanlin2018 commented 3 years ago

Followed the instruction on edit mode, edited src/main/resources/bootstrap.yml. Using following url: https://localhost/pdr/od/id/746FA083A3434449E05324570681A5DD1965?editEnabled=true Got following message: This record is not currently available for editing. Please return to MIDAS and click "Edit Landing Page" to edit.

Need to register the record?

chuanlin2018 commented 3 years ago

Also tried https://localhost:4443/od/id/746FA083A3434449E05324570681A5DD1965?editEnabled=true. Same result.