NGSIToMongo 2.0.0 failed

[fisuda]

I encountered the following error in NGSIToMongo 2.0.0 (Draco 2.0.0) when using MONGO-TUTORIAL.

2022-01-22 23:32:16,079 ERROR [Timer-Driven Process Thread-9] o.a.nifi.processors.ngsi.NGSIToMongo NGSIToMongo[id=45007d62-fe1d-3e62-e8da-1a3ed5694475] Failed to process session due to java.lang.IllegalAccessError: tried to access class org.bson.BSON from class com.mongodb.util.JSONCallback; Processor Administratively Yielded for 1 sec: java.lang.IllegalAccessError: tried to access class org.bson.BSON from class com.mongodb.util.JSONCallback
java.lang.IllegalAccessError: tried to access class org.bson.BSON from class com.mongodb.util.JSONCallback
        at com.mongodb.util.JSONCallback.objectDone(JSONCallback.java:145)
        at org.bson.BasicBSONCallback.arrayDone(BasicBSONCallback.java:139)
        at com.mongodb.util.JSONParser.parseArray(JSON.java:629)
        at com.mongodb.util.JSONParser.parse(JSON.java:225)
        at com.mongodb.util.JSONParser.parse(JSON.java:157)
        at com.mongodb.util.JSON.parse(JSON.java:99)
        at com.mongodb.util.JSON.parse(JSON.java:80)
        at org.apache.nifi.processors.ngsi.ngsi.aggregators.MongoAggregator$RowAggregator.createDocWithMetadata(MongoAggregator.java:101)
        at org.apache.nifi.processors.ngsi.ngsi.aggregators.MongoAggregator$RowAggregator.aggregate(MongoAggregator.java:78)
        at org.apache.nifi.processors.ngsi.AbstractMongoProcessor.persistFlowFile(AbstractMongoProcessor.java:333)
        at org.apache.nifi.processors.ngsi.NGSIToMongo.onTrigger(NGSIToMongo.java:78)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1272)
        at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
        at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Reproduce

docker-compose.yml

version: "3.0"
services:
  orion:
    image: fiware/orion:3.4.0
    depends_on:
      - mongo-db
    ports:
      - 1026:1026
    command: -dbhost mongo-db -noCache

  draco:
    image: ging/fiware-draco:2.0.0
    environment:
      - NIFI_WEB_HTTP_PORT=9090
    ports:
      - 9090:9090

  mongo-db:
    image: mongo:4.4

Create a subscription

#!/bin/sh
set -eu
CB=192.168.0.1
curl -iX POST "http://${CB}:1026/v2/subscriptions" \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
-d '{
  "subject": {
    "entities": [
      {
        "idPattern": ".*"
      }
    ]
  },
  "notification": {
    "http": {
      "url": "http://draco:5050/v2/notify"
    }
  }
}'

Update an entity

#!/bin/bash
set -eu
CB=192.168.0.1
for i in {0..9}
do
  curl -iX POST "http://${CB}:1026/v2/entities?options=keyValues,upsert" \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d "{\"id\": \"I\", \"temperature\":${RANDOM}}"
  sleep 1
done

[anmunoz]

Hi, I will check this issue in this week.

BR,

[anmunoz]

Hi @fisuda,

In the new version of Draco 2.1.0 this bug is fixed. You can check the code in the official repository. The release link is https://github.com/ging/fiware-draco/releases/tag/2.1.0

Additionally, you can use the docker image available for this release by pulling it using docker pull ging/fiware-draco:2.1.0.

You can also use the Mongo-Tutorial template available inside of Draco where you have preconfigured the processors needed to persist in MongoDB.

One thing that you have to consider is that the new version of Draco is aligned with the 1.15.3 version of NiFi in where you need first to log in for access to the Web UI using the default credentials (user: admin, password: pass1234567890). You can check the official documentation for more information about it https://fiware-draco.readthedocs.io/en/latest/

[fisuda]

Hi @anmunoz,

Thank you for fixing this issue.

I tested the new version of Draco 2.1.0. It works fine!

version: "3.0"
services:
  orion:
    image: fiware/orion:3.4.0
    depends_on:
      - mongo-db
    ports:
      - 1026:1026
    command: -dbhost mongo-db -noCache

  draco:
    image: ging/fiware-draco:2.1.0
    environment:
      - NIFI_WEB_HTTP_PORT=9090
    ports:
      - 9090:9090

  mongo-db:
    image: mongo:4.4

> show dbs;
admin          0.000GB
config         0.000GB
local          0.000GB
orion-openiot  0.000GB
sth_openiot    0.000GB
> use sth_openiot;
switched to db sth_openiot
> show collections
sth_/_I_Thing
> db["sth_/_I_Thing"].find()
{ "_id" : ObjectId("62258b3f3ef1112412d78bd2"), "recvTime" : ISODate("2022-03-07T08:34:07.641Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "28752", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b403ef1112412d78bd3"), "recvTime" : ISODate("2022-03-07T08:34:08.637Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "13533", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b413ef1112412d78bd4"), "recvTime" : ISODate("2022-03-07T08:34:09.650Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "15070", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b423ef1112412d78bd5"), "recvTime" : ISODate("2022-03-07T08:34:10.662Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "21554", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b433ef1112412d78bd6"), "recvTime" : ISODate("2022-03-07T08:34:11.674Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "15192", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b443ef1112412d78bd7"), "recvTime" : ISODate("2022-03-07T08:34:12.687Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "8007", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b453ef1112412d78bd8"), "recvTime" : ISODate("2022-03-07T08:34:13.699Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "5246", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b463ef1112412d78bd9"), "recvTime" : ISODate("2022-03-07T08:34:14.711Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "28408", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b473ef1112412d78bda"), "recvTime" : ISODate("2022-03-07T08:34:15.723Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "32079", "attrMetadata" : {  } }
{ "_id" : ObjectId("62258b483ef1112412d78bdb"), "recvTime" : ISODate("2022-03-07T08:34:16.735Z"), "attrName" : "temperature", "attrType" : "Number", "attrValue" : "23890", "attrMetadata" : {  } }

Thanks!