search

DataONEorg / rdataone

R package for reading and writing data at DataONE data repositories
http://doi.org/10.5063/F1M61H5X
36 stars 19 forks source link

NPE error on createObject() #147

Open mbjones opened 8 years ago

mbjones commented 8 years ago

Testing the current HEAD for create support, but ran into an error on createObject(). The example XML file is attached (unzip it before using it). Code to reproduce and the error follow:

williams-fo.xml.zip

>  library(dataone)
> library(datapack)
> library(uuid)
> options(dataone_test_token = "eyJhbGciOiJSUzI ...")
> mndev <- MNode("https://dev.nceas.ucsb.edu/knb/d1/mn/v2")
> sampleEML <- "~/williams/williams-fo.xml"
> metadataId <- paste("urn:uuid:", UUIDgenerate(), sep="")
> metadataObj <- new("DataObject", id=metadataId, format="eml://ecoinformatics.org/eml-2.1.1", file=sampleEML)
> sysmeta <- metadataObj@sysmeta
> objectId <- createObject(mndev, metadataId, sampleEML, sysmeta)
Error in .local(x, ...) : 
  Error creating urn:uuid:3d5fcc13-0694-4cec-8410-2c069cdfe390:

and the Tomcat stack trace showing the NPE and the null value being returned somewhere in the stack:

In D1URLFilter.
HTTP Verb: POST
original pathInfo: /object
original requestURI: /knb/d1/mn/v2/object
stripping /knb/d1/mn/v2 from requestURI
new pathinfo: /object
metacat 20160503-17:21:47: [ERROR]: class java.lang.NullPointerException: null [edu.ucsb.nceas.metacat.restservice.D1ResourceHandler]
java.lang.NullPointerException
    at edu.ucsb.nceas.metacat.restservice.v2.MNResourceHandler.putObject(MNResourceHandler.java:1491)
    at edu.ucsb.nceas.metacat.restservice.v2.MNResourceHandler.handle(MNResourceHandler.java:289)
    at edu.ucsb.nceas.metacat.restservice.D1RestServlet.doPost(D1RestServlet.java:84)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at edu.ucsb.nceas.metacat.restservice.D1URLFilter.doFilter(D1URLFilter.java:48)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
metacat 20160503-17:21:47: [ERROR]: D1ResourceHandler: Serializing exception with code 500: null [edu.ucsb.nceas.metacat.restservice.D1ResourceHandler]
org.dataone.service.exceptions.ServiceFailure
    at edu.ucsb.nceas.metacat.restservice.v2.MNResourceHandler.handle(MNResourceHandler.java:533)
    at edu.ucsb.nceas.metacat.restservice.D1RestServlet.doPost(D1RestServlet.java:84)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at edu.ucsb.nceas.metacat.restservice.D1URLFilter.doFilter(D1URLFilter.java:48)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
mbjones commented 8 years ago

I tracked down the authentication issue here to an expired token. So, the underlying submission methods still seem to work correctly. However, we need much better error messages. First, there should not be an uncaught NPE on the server. Second, the message sent from the MN back to the client should be an InvalidToken exception. Third, the error message to the user in the R client should clearly indicate that the token is expired or otherwise invalid and should be replaced.

leinfelder commented 8 years ago

The stack trace location makes it look more like an issue with the multipart parameters not being set correctly. How did you determine this was an expired auth token?

And does there need to be a ticket for the server side implementation if there is indeed an issue handling expired auth tokens?

On May 3, 2016, at 7:01 PM, Matt Jones notifications@github.com wrote:

I tracked down the authentication issue here to an expired token. So, the underlying submission methods still seem to work correctly. However, we need much better error messages. First, there should not be an uncaught NPE on the server. Second, the message sent from the MN back to the client should be an InvalidToken exception. Third, the error message to the user in the R client should clearly indicate that the token is expired or otherwise invalid and should be replaced.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

gothub commented 8 years ago

The R client should have detected that the token was expired and not let the user call a service with a POST request, so I'll check into the token checking.

On Tue, May 3, 2016 at 7:01 PM, Matt Jones notifications@github.com wrote:

I tracked down the authentication issue here to an expired token. So, the underlying submission methods still seem to work correctly. However, we need much better error messages. First, there should not be an uncaught NPE on the server. Second, the message sent from the MN back to the client should be an InvalidToken exception. Third, the error message to the user in the R client should clearly indicate that the token is expired or otherwise invalid and should be replaced.

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub https://github.com/DataONEorg/rdataone/issues/147#issuecomment-216720462

Peter Slaughter, Software Engineer National Center for Ecological Analysis and Synthesis 735 State St, Santa Barbara, CA 93101 voice: 805 892 2533

mbjones commented 8 years ago

@leinfelder Yes, if there is an issue with the server side, we'd need a ticket there. It's still possible that this is solely due to the client side not properly displaying the returned errors, so let's confirm that and generate any server-side tickets as needed.

mbjones commented 8 years ago

And @leinfelder , the way I determined it was an expired token was by generating a new token, then rerunning the same code, and the error disappeared. So, my successful submit was the indicator.

gothub commented 8 years ago

After many tests with valid and expired tokens I'm unable to recreate this NPE. The R checks haven't haven't allowed an expired token through (either ORCID or CIlogon based), however I can make a token invalid by changing the signature portion of the JWT or send a prod env token to a test env node and in this case I get:

Error in .local(x, ...) : 
Error creating urn:uuid:ce24a678-d48c-4809-b442-78590d81aeb4: Session is required to WRITE to the Node.

I believe that the NPE is being generated after the server authenticated session is created, and the line number indicated in the traceback deals with extracting the pid from the http multipart request.

I'll keep plugging away at this, if you all have any suggestions, please pass them along.

PS - I'm using R 3.3.0 with a freshly cloned rdataone and datapack source repos, with a clean R environment, using tokens generated from search-stage.test.dataone.org and search.dataone.org (to submit a prod token as dataone_test_token to pass an invalid token). Tests have been submitted to dev.nceas.ucsb.edu, mn-stage-ucsb-2.test.dataone.org.

earnaud commented 3 years ago

Hi,

I am having the same error message (Error in .local(x, ...) : \n Error creating urn:uuid:5b21b7c5-80df-4fb1-a018-3f45bc4131aa: Session is required to WRITE to the Node.) after executing: 

dataone::uploadDataPackage(
      d1c, # CN : STAGING ; MN : urn:node:mnTestARCTIC 
      dp, # checked
      public = TRUE,
      accessRules = NA,
      quiet = FALSE,
      packageId = paste0("urn:uuid:", uuid::UUIDgenerate())
    )

I have a doubt here concerning the packageId argument.

Session info:

R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=en_US.utf8        LC_NUMERIC=C               LC_TIME=en_US.utf8        
 [4] LC_COLLATE=en_US.utf8      LC_MONETARY=en_US.utf8     LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=fr_FR.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C
taojing2002 commented 3 years ago

This message means you didn't provide a session or the session has expired when you call the createObject method. I just successfully created an object by the web metadata editor on testArctic. This means the system is working. So you may need a new session and try again.

Hope this is helpful.

Jing

On 8/9/21 5:19 AM, earnaud wrote:

Hi,

I am having the same error message (|Error in .local(x, ...) : \n Error creating urn:uuid:5b21b7c5-80df-4fb1-a018-3f45bc4131aa: Session is required to WRITE to the Node.|) after executing:

|dataone::uploadDataPackage( d1c, # CN : STAGING ; MN : urn:node:mnTestARCTIC dp, # checked public = TRUE, accessRules = NA, quiet = FALSE, packageId = paste0("urn:uuid:", uuid::UUIDgenerate()) ) |

I have a doubt here concerning the |packageId| argument.

Session info:

|R version 4.1.0 (2021-05-18) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 20.04.2 LTS Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0 locale: [1] LC_CTYPE=en_US.utf8 LC_NUMERIC=C LC_TIME=en_US.utf8 [4] LC_COLLATE=en_US.utf8 LC_MONETARY=en_US.utf8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=fr_FR.UTF-8 LC_NAME=C LC_ADDRESS=C [10] LC_TELEPHONE=C LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C |

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/DataONEorg/rdataone/issues/147#issuecomment-895176299, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5QQDDPPXBATSDFHEA6UDDT37BULANCNFSM4CC27QTA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

mbjones commented 3 years ago

@earnaud Can you confirm that you had a valid token set in your options for the dataone package? If not, please make sure that is set up correctly following the instructions in the package vignette: https://cran.rstudio.com/web/packages/dataone/vignettes/v02-dataone-federation.html#dataone-user-authentication-using-tokens If the problem persists after that, please let us know.

earnaud commented 3 years ago

Thanks, I indeed had bugs in my script.