search

IQSS / dataverse

Open source research data repository software
http://dataverse.org
Other
859 stars 481 forks source link

Dataset API: Cannot edit a dataset, throws null pointer exception. #914

Closed kcondon closed 9 years ago

kcondon commented 9 years ago

At some point the edit dataset command changed from a post to a put. I had tested using post and that seemed to work. When I try the put version, I get a null pointer exception.

To test, I used Phil's working json file, scripts/search/data/dataset01-create-new.json, created a dataset, then made a copy and added "Onion" to the front of the title and tried the put command.

The server log error is below.

[2014-09-09T12:46:16.787-0400] [glassfish 4.0] [WARNING] [] [javax.enterprise.web] [tid: _ThreadID=113 _ThreadName=http-listener-1(4)] [timeMillis: 1410281176787] [levelValue: 900] [[ StandardWrapperValve[edu.harvard.iq.dataverse.api.ApiConfiguration]: Servlet.service() for servlet edu.harvard.iq.dataverse.api.ApiConfiguration threw exception java.lang.NullPointerException at edu.harvard.iq.dataverse.util.json.JsonParser.parseMetadataBlocks(JsonParser.java:77) at edu.harvard.iq.dataverse.util.json.JsonParser.parseDatasetVersion(JsonParser.java:65) at edu.harvard.iq.dataverse.api.Datasets.updateDraftVersion(Datasets.java:284) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:125) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:152) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:91) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:346) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:341) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:101) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:224) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:198) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:946) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:323) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:372) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:335) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:218) at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:344) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:253) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:316) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160) at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673) at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174) at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673) at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:354) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:260) at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:188) at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:196) at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:173) at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:189) at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119) at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288) at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206) at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136) at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114) at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77) at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:838) at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113) at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115) at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55) at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135) at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564) at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544) at java.lang.Thread.run(Thread.java:744) ]

kcondon commented 9 years ago

I should add this is blocking development of the data deposit api

scolapasta commented 9 years ago

@kcondon @esotiri can we retest this?

esotiri commented 9 years ago

Data deposit API works ok editing the metadata. I tested changing the author and title.

pdurbin commented 9 years ago

As part of this ticket let's make sure we document well how to update a dataset with JSON either at https://github.com/IQSS/dataverse/blob/master/scripts/api/readme.md (for now) or in the Google Doc linked from https://github.com/IQSS/dataverse/issues/915#issuecomment-66475527 .

It's very important to be able to edit metadata via JSON because the XML used in SWORD is very limited in the number of metadata fields that are available.

esotiri commented 9 years ago

Native API throws java.lang.NullPointerException using the command below and a json I used to create the dataset initially. The only change I did to the json is that I modified the title.

curl -H "Content-type:application/json" -X PUT -d @dset2015.json https://$host/api/datasets/24/versions/:draft?key=$token Server log and API call report same:

HTTP Status 500 - Internal Server Error The server encountered an internal error that prevented it from fulfilling this request. javax.servlet.ServletException: java.lang.NullPointerException 

 StandardWrapperValve[edu.harvard.iq.dataverse.api.ApiConfiguration]: Servlet.service() for servlet edu.harvard.iq.dataverse.api.ApiConfiguration threw exception
java.lang.NullPointerException
    at edu.harvard.iq.dataverse.util.json.JsonParser.parseMetadataBlocks(JsonParser.java:157)
    at edu.harvard.iq.dataverse.util.json.JsonParser.parseDatasetVersion(JsonParser.java:145)
    at edu.harvard.iq.dataverse.util.json.JsonParser.parseDatasetVersion(JsonParser.java:104)
    at edu.harvard.iq.dataverse.api.Datasets.updateDraftVersion(Datasets.java:241)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:152)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
esotiri commented 9 years ago

here are the commands I used:

  1. get json from an existing dataset: curl -X GET https://$host/api/datasets/56?key=$token
  2. created dataset with it (changed doi) curl -H "Content-type:application/json" -X POST -d @dset2015.json https://$host/api/dvs/test2015/datasets/?key=$token
  3. update dataset metadata (title) curl -H "Content-type:application/json" -X PUT -d @dset2015.json https://$host/api/datasets/24/versions/:draft?key=$token json file: 
{
  "id": 24,
  "identifier": "5GUURR5",
  "persistentUrl": "http://dx.doi.org/10.5072/FK2/5GUURR5",
  "protocol": "doi",
  "authority": "10.5072/FK2",
  "initialVersion": {
    "id": 11,
    "versionState": "DRAFT",
    "distributionDate": "Distribution Date",
    "productionDate": "Production Date",
    "lastUpdateTime": "2014-12-16 02:23:46 -05",
    "createTime": "2014-12-16 02:20:45 -05",
    "metadataBlocks": {
      "citation": {
        "displayName": "Citation Metadata",
        "fields": [
          {
            "typeName": "title",
            "multiple": false,
            "typeClass": "primitive",
            "value": "time stamp title change ok changing again"
          },
          {
            "typeName": "author",
            "multiple": true,
            "typeClass": "compound",
            "value": [
              {
                "authorName": {
                  "typeName": "authorName",
                  "multiple": false,
                  "typeClass": "primitive",
                  "value": "Dataverse, Admin"
                },
                "authorAffiliation": {
                  "typeName": "authorAffiliation",
                  "multiple": false,
                  "typeClass": "primitive",
                  "value": "Dataverse"
                }
              }
            ]
          },
          {
            "typeName": "datasetContact",
            "multiple": true,
            "typeClass": "compound",
            "value": [
              {
                "datasetContactName": {
                  "typeName": "datasetContactName",
                  "multiple": false,
                  "typeClass": "primitive",
                  "value": "Dataverse, Admin"
                },
                "datasetContactAffiliation": {
                  "typeName": "datasetContactAffiliation",
                  "multiple": false,
                  "typeClass": "primitive",
                  "value": "Dataverse"
                },
                "datasetContactEmail": {
                  "typeName": "datasetContactEmail",
                  "multiple": false,
                  "typeClass": "primitive",
                  "value": "admin@malinator.com"
                }
              }
            ]
          },
          {
            "typeName": "dsDescription",
            "multiple": true,
            "typeClass": "compound",
            "value": [
              {
                "dsDescriptionValue": {
                  "typeName": "dsDescriptionValue",
                  "multiple": false,
                  "typeClass": "primitive",
                  "value": "desc"
                }
              }
            ]
          },
          {
            "typeName": "subject",
            "multiple": true,
            "typeClass": "controlledVocabulary",
            "value": [
              "Chemistry",
              "Mathematical Sciences"
            ]
          },
          {
            "typeName": "depositor",
            "multiple": false,
            "typeClass": "primitive",
            "value": "Admin Dataverse"
          },
          {
            "typeName": "dateOfDeposit",
            "multiple": false,
            "typeClass": "primitive",
            "value": "2014-12-16"
          }
        ]
      },
      "geospatial": {
        "displayName": "Geospatial Metadata",
        "fields": [
          {
            "typeName": "geographicCoverage",
            "multiple": true,
            "typeClass": "compound",
            "value": [
              {
                "country": {
                  "typeName": "country",
                  "multiple": false,
                  "typeClass": "controlledVocabulary",
                  "value": "United States"
                }
              }
            ]
          }
        ]
      },
      "socialscience": {
        "displayName": "Social Science and Humanities Metadata",
        "fields": [
          {
            "typeName": "universe",
            "multiple": true,
            "typeClass": "primitive",
            "value": [
              "Universe "
            ]
          }
        ]
      },
      "astrophysics": {
        "displayName": "Astronomy and Astrophysics Metadata",
        "fields": [
          {
            "typeName": "astroType",
            "multiple": true,
            "typeClass": "controlledVocabulary",
            "value": [
              "Image"
            ]
          }
        ]
      },
      "biomedical": {
        "displayName": "Life Sciences Metadata",
        "fields": [
          {
            "typeName": "studyDesignType",
            "multiple": true,
            "typeClass": "controlledVocabulary",
            "value": [
              "Case Control"
            ]
          }
        ]
      },
      "journal": {
        "displayName": "Journal Metadata",
        "fields": [
          {
            "typeName": "articleType",
            "multiple": false,
            "typeClass": "controlledVocabulary",
            "value": "abstract"
          }
        ]
      }
    }
  }
}
esotiri commented 9 years ago

issue resolved: command below allows editing a dataset. 

curl -H "Content-type:application/json" -X PUT -d @aaa54.json https://$host/api/datasets/54/versions/:draft?key=$token

json file: 

{
      "id": 28,
      "versionState": "DRAFT",
      "distributionDate": "Distribution Date",
      "productionDate": "Production Date",
      "lastUpdateTime": "2015-01-16 11:38:33 -05",
      "createTime": "2015-01-16 11:37:41 -05",
      "metadataBlocks": {
        "citation": {
          "displayName": "Citation Metadata",
          "fields": [
            {
              "typeName": "title",
              "multiple": false,
              "typeClass": "primitive",
              "value": "Test Licensing 6- TEST TEST TEST TEST"
            },
            {
              "typeName": "author",
              "multiple": true,
              "typeClass": "compound",
              "value": [
                {
                  "authorName": {
                    "typeName": "authorName",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "Dataverse, Admin"
                  },
                  "authorAffiliation": {
                    "typeName": "authorAffiliation",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "Dataverse"
                  }
                }
              ]
            },
            {
              "typeName": "datasetContact",
              "multiple": true,
              "typeClass": "compound",
              "value": [
                {
                  "datasetContactName": {
                    "typeName": "datasetContactName",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "Dataverse, Admin"
                  },
                  "datasetContactAffiliation": {
                    "typeName": "datasetContactAffiliation",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "Dataverse"
                  },
                  "datasetContactEmail": {
                    "typeName": "datasetContactEmail",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "admin@malinator.com"
                  }
                }
              ]
            },
            {
              "typeName": "dsDescription",
              "multiple": true,
              "typeClass": "compound",
              "value": [
                {
                  "dsDescriptionValue": {
                    "typeName": "dsDescriptionValue",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "6"
                  }
                }
              ]
            },
            {
              "typeName": "subject",
              "multiple": true,
              "typeClass": "controlledVocabulary",
              "value": [
                "Arts and Humanities"
              ]
            },
            {
              "typeName": "depositor",
              "multiple": false,
              "typeClass": "primitive",
              "value": "Admin Dataverse"
            },
            {
              "typeName": "dateOfDeposit",
              "multiple": false,
              "typeClass": "primitive",
              "value": "2015-01-16"
            }
          ]
        }
      }
    }