Error when retrieving collection, containing field with type (byte[]) Byte array (its actually screen shot images).

[Prax7-7]

I trying to retrieve the collection through a Java client connecting to RESTHEART which is connected to MONGODB ATLAS, when use the HTTP GET for these collections I get the following

**Error Message. "{"http status code":500,"_exceptions":[{"exception":"java.lang.NoClassDefFoundError","exception message":"javax/xml/bind/DatatypeConverter"}],"http status description":"Internal Server Error","message":"Error handling the request, see log for more information"}".**

How do I retrieve the collections containing a Byte array??

[ujibang]

can you please give this issue a short title and put everything else in the description.

Also can you please quote the code, se we can all read it? Have a look here for a reference on markdown https://guides.github.com/features/mastering-markdown/

[ujibang]

Hi @Prax7-7

I made a test, created a document with the mongo shell with a binary property as follows:

# Extract 1M random bytes, convert it to base64, and store it as /tmp/rrr
$ head -c 100 /dev/random | base64 > /tmp/r

$ mongo
> var r = cat ('/tmp/r')                # Reads into r BUT then terminates it with a NL
> var rr = r.substring (0, r.length-1)  # Remove the newline
> var p = BinData (0, rr)               # bring it into p
> db.coll.insert({"_id": "bin", "data": p })

 GET /db/coll/bin correctly returns the the BSON binary represented as follows:

{
  "_id": "bin",
  "data": {
        "$binary": "DqnEq7hiWZ1jHoYf/YJpNHevlGrRmT5V9NGN7daoPYetiTvgeP4C9n4j8Gu5mduhEYzWDFK2a3gO+CvzrDgM3BBFG07fF6qabHXDsGTo92m93QohjGtqn8nkNP6KVnWIcbgBbw==",
        "$type": "00"
  }
}

From the BSON specs (http://bsonspec.org/spec.html)

The BSON "binary" or "BinData" datatype is used to represent arrays of bytes. It is somewhat analogous to the Java notion of a ByteArray. BSON binary values have a subtype. This is used to indicate what kind of data is in the byte array. Subtypes from zero to 127 are predefined or reserved. Subtypes from 128-255 are user-defined. \x02 Binary (Old) - This used to be the default subtype, but was deprecated in favor of \x00

I'm not sure why you are getting this javax/xml/bind/DatatypeConverter class not found exception. Is it coming from your custom code?

Can you paste the restheart error log message.

[Prax7-7]

Hi Ujibang, Thanks you for your help.

Well when I use GET in MongoDB Shell, it does return the BSON Binary data in the above fashion, but when I try it with HTTPie (command line client) talking to RESTHeart or a Custom JAVA http client application talking to RESTHeart I get the below logs on the RESTHeart Terminal.

Prashanths-MacBook-Pro:restheart-3.0.3 prashanthsiddharamaiah$ http -a admin:changeit 127.0.0.1:8080/optimus/devices
HTTP/1.1 500 Internal Server Error
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, Auth-Token, Auth-Token-Valid-Until, Auth-Token-Location, X-Powered-By
Auth-Token: f7dc346b-1afe-4dad-943b-3221f93df2b8
Auth-Token-Location: /_authtokens/admin
Auth-Token-Valid-Until: 2017-10-16T11:12:55.263968Z
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 193
Content-Type: application/json
Date: Mon, 16 Oct 2017 10:58:09 GMT
ETag: 59e490936284c703d8bfe585
X-Powered-By: restheart.org

{
    "_exceptions": [
        {
            "exception": "java.lang.NoClassDefFoundError",
            "exception message": "javax/xml/bind/DatatypeConverter"
        }
    ],
    "http status code": 500,
    "http status description": "Internal Server Error",
    "message": "Error handling the request, see log for more information"
}

[ujibang]

Please provide the server log message.

Also can you try with the latest version of restheart.

[Prax7-7]

HERE . Server Logs

13:05:06.757 [XNIO-1 task-2] ERROR org.restheart.handlers.ErrorHandler - Error handling the request
java.lang.NoClassDefFoundError: javax/xml/bind/DatatypeConverter
    at org.bson.json.JsonWriter.doWriteBinaryData(JsonWriter.java:156)
    at org.bson.AbstractBsonWriter.writeBinaryData(AbstractBsonWriter.java:360)
    at org.bson.codecs.BsonBinaryCodec.encode(BsonBinaryCodec.java:31)
    at org.bson.codecs.BsonBinaryCodec.encode(BsonBinaryCodec.java:28)
    at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91)
    at org.bson.codecs.BsonDocumentCodec.writeValue(BsonDocumentCodec.java:136)
    at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:115)
    at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:41)
    at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91)
    at org.bson.codecs.BsonArrayCodec.encode(BsonArrayCodec.java:82)
    at org.bson.codecs.BsonArrayCodec.encode(BsonArrayCodec.java:37)
    at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91)
    at org.bson.codecs.BsonDocumentCodec.writeValue(BsonDocumentCodec.java:136)
    at org.bson.codecs.BsonDocumentCodec.encode(BsonDocumentCodec.java:115)
    at org.bson.BsonDocument.toJson(BsonDocument.java:809)
    at org.bson.BsonDocument.toJson(BsonDocument.java:799)
    at org.restheart.utils.JsonUtils.toJson(JsonUtils.java:672)
    at org.restheart.handlers.ResponseSenderHandler.handleRequest(ResponseSenderHandler.java:117)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.metadata.HookMetadataHandler.handleRequest(HookMetadataHandler.java:104)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.metadata.TransformerHandler.handleRequest(TransformerHandler.java:72)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.metadata.AbstractTransformerMetadataHandler.handleRequest(AbstractTransformerMetadataHandler.java:63)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.collection.GetCollectionHandler.handleRequest(GetCollectionHandler.java:151)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.metadata.AbstractTransformerMetadataHandler.handleRequest(AbstractTransformerMetadataHandler.java:63)
    at org.restheart.handlers.RequestDispacherHandler.handleRequest(RequestDispacherHandler.java:468)
    at org.restheart.handlers.injectors.CollectionPropsInjectorHandler.handleRequest(CollectionPropsInjectorHandler.java:98)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.injectors.DbPropsInjectorHandler.handleRequest(DbPropsInjectorHandler.java:92)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.injectors.AccountInjectorHandler.handleRequest(AccountInjectorHandler.java:56)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.security.handlers.AccessManagerHandler.handleRequest(AccessManagerHandler.java:55)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.security.handlers.AuthTokenInjecterHandler.handleRequest(AuthTokenInjecterHandler.java:70)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.security.handlers.AuthenticationCallHandler.handleRequest(AuthenticationCallHandler.java:54)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:55)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:61)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:85)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.security.handlers.SecurityHandler.handleRequest(SecurityHandler.java:50)
    at org.restheart.security.handlers.SecurityHandlerDispacher.handleRequest(SecurityHandlerDispacher.java:60)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.injectors.BodyInjectorHandler.handleRequest(BodyInjectorHandler.java:109)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.OptionsHandler.handleRequest(OptionsHandler.java:58)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.handlers.injectors.RequestContextInjectorHandler.handleRequest(RequestContextInjectorHandler.java:647)
    at org.restheart.handlers.PipedHttpHandler.next(PipedHttpHandler.java:136)
    at org.restheart.security.handlers.CORSHandler.handleRequest(CORSHandler.java:88)
    at org.restheart.handlers.RequestLoggerHandler.handleRequest(RequestLoggerHandler.java:87)
    at org.restheart.handlers.PipedHttpHandler.handleRequest(PipedHttpHandler.java:89)
    at io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:94)
    at io.undertow.server.handlers.HttpContinueAcceptingHandler.handleRequest(HttpContinueAcceptingHandler.java:78)
    at org.restheart.handlers.ErrorHandler.handleRequest(ErrorHandler.java:69)
    at io.undertow.server.handlers.encoding.EncodingHandler.handleRequest(EncodingHandler.java:72)
    at org.restheart.handlers.GzipEncodingHandler.handleRequest(GzipEncodingHandler.java:74)
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:210)
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:809)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.DatatypeConverter
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:185)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
    ... 69 common frames omitted
13:05:06.770 [XNIO-1 task-2] INFO  o.r.handlers.RequestLoggerHandler - GET http://localhost:8080/optimus/devices from /127.0.0.1:54221 => status=500 elapsed=20184ms contentLength=193 username=admin roles=[users, admins]

Till then I will try with the latest version of RESTHeart.

[ujibang]

I cannot replicate the problem even with RESTHerat 3.0.3.

Please provide versions of: rh, java and mongodb.

[Prax7-7]

RH : 3.0.3 Java : 1.8 Mongodb : MongoDb atlas

[ujibang]

can also you provide a mongo shell command to create a document that leads to this exception?

[Prax7-7]

The document is created by a Java application that talks directly to local MongoDB, we used this dump data and restored on MONGODB ATLAS. Anyways I wanted to say that the document is not created with mongo shell rather JAVA application itself. Here is How the Method looks

public void updateDeviceScreenshot(String udid, byte[] screenshot) {
        if (!screenshotExist(mongoClient.getDatabase(DATABASE_NAME).getCollection(COLLECTION_DEVICES), udid)) {
            mongoClient.getDatabase(DATABASE_NAME).getCollection(COLLECTION_DEVICES).updateOne(new BasicDBObject(KEY_DEVICES_UDID, udid).append(KEY_BUILD_ID, latestBuildID),
                    new BasicDBObject(QUERY_SET, new Document().append(KEY_SCREENSHOTS_SCREENSHOT, screenshot)));
        }
    }

[ujibang]

I added two test cases to try to reproduce the error. You can see them https://github.com/SoftInstigate/restheart/blob/master/src/test/java/org/restheart/test/integration/GetCollectionIT.java

see tests testBinaryProperty() and testBinaryPropertyLegacyDriver()

the second test resembles you code.

The problem is that tests pass. Everything works fine there.

Isn't that you are using java 9? I found this issue with it and java.lang.NoClassDefFoundError: javax/xml/bind/DatatypeConverter. Have a look at https://stackoverflow.com/questions/43574426/how-to-resolve-java-lang-noclassdeffounderror-javax-xml-bind-jaxbexception-in-j

[Prax7-7]

No I am using Java 1.8 only, and the problem persists. Just give you more info below my client method am using

static public void GetMongoDoc() throws Exception {
            CloseableHttpClient client = HttpClients.createDefault();

            URI uri = new URIBuilder()
                    .setScheme("http")
                    .setHost("localhost")
                    .setPort(8080)
                    .setUserInfo("admin","changeit")
                    .setPath("optimus/devices")
                    .build();
            HttpGet httpGet = new HttpGet(uri);
            httpGet.addHeader("Accept", "application/json, text/plain, */*");

            HttpResponse response = client.execute(httpGet);
            BufferedReader rd = new BufferedReader
                    (new InputStreamReader(
                            response.getEntity().getContent()));

            String line = "";

            line = rd.readLine();
            JSONObject params = new JSONObject(line);
            System.out.println(params.toString());

            client.close();
        }

Also have a look at the below image, because few GET operations are working fine

[ujibang]

the problem is java.lang.NoClassDefFoundError.

it doesn't find the javax/xml/bind/DatatypeConverter class which is definitely in java8 and not in java9. make sure RESTHeart runs with java8.

[Prax7-7]

Hi ujibang, You were it was clearly Java 9 issue, my JVM was Java 1.8 but my global JDK home was still Java 9. The issue is fixed. Thanks a lot for your quick support. 👍