Incorrect response for "/v2/types" API of Orion 2.6.1 and Orion 3.7.0

[ArqamFarooqui110719]

Bug description I have run 3 different versions of Orion via docker i.e. : Orion 3.7.0 + MongoDB 4.4 - on PORT 1026 Orion 2.6.1 + MongoDB 3.6 - on PORT 1027 Orion 2.1.0 + MongoDB 3.4 - on PORT 1028

When executing /v2/types/ query, I'm getting INCORRECT response with Orion-2.6.1 and Orion-3.7.0 as response is including all the attribute types belonging to different entity type. Screenshot for the same is attached below:

In the above entity, the "types of location attribute" is only "Integer", but in Orion2.6.1 and Orion3.7.0, the response of /v2/types/ query is also returning the extra "types of location attribute" i.e. registered with another Entity.

Same behavior is occurring with "uploadDate" attribute type in Orion2.6.1 and Orion3.7.0.

How to reproduce it

1. Run 3 different versions of Orion via docker as given above

2. Create below entity on each Orion (Attaching entity creation logs of Orion-2.6.1)

   [root@fiware-mgmsv2 ~]# curl -v http://localhost:1027/v2/entities -s -S -H 'Content-Type: application/json' -d @- <<EOF
   > {
   >    "type": "sigfoxDataIT",
   >    "id": "sigfoxDataIT.0",
   >    "name": {
   >      "type": "Text",
   >      "value": "Sigfoxƒf[ƒ^Integer"
   >    },
   >    "location": {
   >      "type": "Integer",
   >      "value": "35.681236, 139.767125"
   >    },
   >    "uploadDate": {
   >     "type": "DateTime",
   >     "value": "2020-01-01T00:00:00.000+09:00"
   >    },
   >    "creator": {
   >     "type": "Text",
   >     "value": "TestData"
   >    }
   > }
   > EOF
   * About to connect() to localhost port 1027 (#0)
   *   Trying ::1...
   * Connected to localhost (::1) port 1027 (#0)
   > POST /v2/entities HTTP/1.1
   > User-Agent: curl/7.29.0
   > Host: localhost:1027
   > Accept: */*
   > Content-Type: application/json
   > Content-Length: 363
   >
   * upload completely sent off: 363 out of 363 bytes
   < HTTP/1.1 201 Created
   < Connection: Keep-Alive
   < Content-Length: 0
   < Location: /v2/entities/sigfoxDataIT.0?type=sigfoxDataIT
   < Fiware-Correlator: 25554f0c-3727-11ed-8cfe-0242ac110005
   < Date: Sun, 18 Sep 2022 07:54:38 GMT
   <
   * Connection #0 to host localhost left intact
   [root@fiware-mgmsv2 ~]#

3. Get the entities on each Orion (Attaching response of Orion-2.6.1)

   [root@fiware-mgmsv2 ~]# curl -kv -X GET "http://localhost:1027/v2/entities/sigfoxDataIT.0" -s -S -H "accept: application/json" | python -mjson.tool
   * About to connect() to localhost port 1027 (#0)
   *   Trying ::1...
   * Connected to localhost (::1) port 1027 (#0)
   > GET /v2/entities/sigfoxDataIT.0 HTTP/1.1
   > User-Agent: curl/7.29.0
   > Host: localhost:1027
   > accept: application/json
   >
   < HTTP/1.1 200 OK
   < Connection: Keep-Alive
   < Content-Length: 330
   < Content-Type: application/json
   < Fiware-Correlator: 2565fc98-3728-11ed-b460-0242ac110005
   < Date: Sun, 18 Sep 2022 08:01:48 GMT
   <
   { [data not shown]
   * Connection #0 to host localhost left intact
   {
   "creator": {
       "metadata": {},
       "type": "Text",
       "value": "TestData"
   },
   "id": "sigfoxDataIT.0",
   "location": {
       "metadata": {},
       "type": "Integer",
       "value": "35.681236, 139.767125"
   },
   "name": {
       "metadata": {},
       "type": "Text",
       "value": "Sigfox\u0192f[\u0192^Integer"
   },
   "type": "sigfoxDataIT",
   "uploadDate": {
       "metadata": {},
       "type": "DateTime",
       "value": "2019-12-31T15:00:00.000Z"
   }
   }
   [root@fiware-mgmsv2 ~]#

4. Get the types of entity on each Orion Orion-2.1.0

   [root@fiware-mgmsv2 ~]# curl -v localhost:1028/v2/types/sigfoxDataIT -s -S -H 'Accept: application/json' | python -mjson.tool
   * About to connect() to localhost port 1028 (#0)
   *   Trying ::1...
   * Connected to localhost (::1) port 1028 (#0)
   > GET /v2/types/sigfoxDataIT HTTP/1.1
   > User-Agent: curl/7.29.0
   > Host: localhost:1028
   > Accept: application/json
   >
   < HTTP/1.1 200 OK
   < Connection: Keep-Alive
   < Content-Length: 145
   < Content-Type: application/json
   < Fiware-Correlator: 01df87e2-3725-11ed-9155-0242ac110007
   < Date: Sun, 18 Sep 2022 07:39:20 GMT
   <
   { [data not shown]
   * Connection #0 to host localhost left intact
   {
   "attrs": {
       "creator": {
           "types": [
               "Text"
           ]
       },
       "location": {
           "types": [
               "Integer"
           ]
       },
       "name": {
           "types": [
               "Text"
           ]
       },
       "uploadDate": {
           "types": [
               "DateTime"
           ]
       }
   },
   "count": 1
   }

   Orion-2.6.1

   [root@fiware-mgmsv2 ~]# curl -v localhost:1027/v2/types/sigfoxDataIT -s -S -H 'Accept: application/json' | python -mjson.tool
   * About to connect() to localhost port 1027 (#0)
   *   Trying ::1...
   * Connected to localhost (::1) port 1027 (#0)
   > GET /v2/types/sigfoxDataIT HTTP/1.1
   > User-Agent: curl/7.29.0
   > Host: localhost:1027
   > Accept: application/json
   >
   < HTTP/1.1 200 OK
   < Connection: Keep-Alive
   < Content-Length: 188
   < Content-Type: application/json
   < Fiware-Correlator: a58e7eea-3728-11ed-88e9-0242ac110005
   < Date: Sun, 18 Sep 2022 08:05:23 GMT
   <
   { [data not shown]
   * Connection #0 to host localhost left intact
   {
   "attrs": {
       "creator": {
           "types": [
               "Text"
           ]
       },
       "location": {
           "types": [
               "Float",
               "Integer",
               "Number",
               "Text",
               "geo:point"
           ]
       },
       "name": {
           "types": [
               "Text"
           ]
       },
       "uploadDate": {
           "types": [
               "DateTime",
               "Text"
           ]
       }
   },
   "count": 1
   }
   [root@fiware-mgmsv2 ~]#

   Orion-3.7.0

   [root@fiware-mgmsv2 ~]# curl -v localhost:1026/v2/types/sigfoxDataIT -s -S -H 'Accept: application/json' | python -mjson.tool
   * About to connect() to localhost port 1026 (#0)
   *   Trying ::1...
   * Connected to localhost (::1) port 1026 (#0)
   > GET /v2/types/sigfoxDataIT HTTP/1.1
   > User-Agent: curl/7.29.0
   > Host: localhost:1026
   > Accept: application/json
   >
   < HTTP/1.1 200 OK
   < Connection: Keep-Alive
   < Content-Length: 188
   < Content-Type: application/json
   < Fiware-Correlator: cb10d2ac-375b-11ed-b6c7-0242ac110003
   < Date: Sun, 18 Sep 2022 14:11:30 GMT
   <
   { [data not shown]
   * Connection #0 to host localhost left intact
   {
   "attrs": {
       "creator": {
           "types": [
               "Text"
           ]
       },
       "location": {
           "types": [
               "Float",
               "Integer",
               "Number",
               "Text",
               "geo:point"
           ]
       },
       "name": {
           "types": [
               "Text"
           ]
       },
       "uploadDate": {
           "types": [
               "DateTime",
               "Text"
           ]
       }
   },
   "count": 1
   }
   [root@fiware-mgmsv2 ~]#

Expected behavior Expected response of /v2/types/ request should be as per Orion-2.1.0 i.e. types of location and uploadDate attribute should belongs to same entity.

Additional information

Comparison of /v2/types/ query responses: 1) Orion-2.1.0 and Orion-2.6.1:

2) Orion-2.6.1 and Orion-3.7.0:

2) Orion-2.1.0 and Orion-3.7.0:

[ArqamFarooqui110719]

As per Orion's official document 'https://fiware-orion.readthedocs.io/en/latest/user/walkthrough_apiv2.html' /v2/types/ API returns UNION OF ATTRIBUTE TYPES BELONGS TO "SAME ENTITY-TYPE".

As per above investigation and analysis of the /v2/types/ API responses (shared above with issue description), this issue seems as a bug.

I have also confirmed that /v2/types/ API is working properly in Orion-2.2.0 version but from Orion-2.3.0 onwards, I'm getting incorrect response.

[fgalan]

@ArqamFarooqui110719 thank you for so detail issue report :)

Changes from 2.2.0 to 2.3.0 (https://github.com/telefonicaid/fiware-orion/compare/2.2.0...2.3.0) show modification on src/lib/mongoBackend/mongoQueryTypes.cpp that could be the cause of the issue. However, in the 2.3.0 changelog at https://github.com/telefonicaid/fiware-orion/releases/tag/2.3.0 The changes seems to be have done in PR https://github.com/telefonicaid/fiware-orion/pull/3510

[fgalan]

Function to fix mongoAttributesForEntityType() ?

[fgalan]

PR https://github.com/telefonicaid/fiware-orion/pull/4217

[fgalan]

The issue should has been fixed in master branch.

@ArqamFarooqui110719 could you please test it using the telefonicaiot/fiware-orion:latest image, please?

[ArqamFarooqui110719]

The issue should has been fixed in master branch.

@ArqamFarooqui110719 could you please test it using the telefonicaiot/fiware-orion:latest image, please?

I've tested and /v2/types/ APIs are working fine. Please find below stepwise details:

1. Used Mongo:4.4 and telefonicaiot/fiware-orion:latest images

   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# docker pull telefonicaiot/fiware-orion:latest
   latest: Pulling from telefonicaiot/fiware-orion
   b85a868b505f: Pull complete
   4e9601c6f42d: Pull complete
   c6e76a008e18: Pull complete
   Digest: sha256:c9cc4b081fdf01227030b977922bfa126be3069a8c8e0d54e5a2030ea23f4da6
   Status: Downloaded newer image for telefonicaiot/fiware-orion:latest
   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
   

   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# docker run --name mongodb -d mongo:4.4
   Unable to find image 'mongo:4.4' locally
   4.4: Pulling from library/mongo
   675920708c8b: Pull complete
   6f9c8c301e0f: Pull complete
   73738965c4ce: Pull complete
   7fd6635b9ddf: Pull complete
   73a471eaa4ad: Pull complete
   595a03cbac53: Pull complete
   cbc6ab7f7529: Pull complete
   2e737d6ddcde: Pull complete
   874be3f13403: Pull complete
   Digest: sha256:92ff83581bcd54a7e8a74df21acc958761dd3c155d143edad06a1d6a139a8c7e
   Status: Downloaded newer image for mongo:4.4
   b6a5f53f04be9c524d84442a813f5f0edfbf2116b7deccbd3b5531741eddd4bd
   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
   

   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# docker run -d --name orionlatest --link mongodb:mongodb -p 1036:1026 telefonicaiot/fiware-orion:latest -dbhost mongodb
   9840e6fc4e0abcae338e080ce1823226f9f8839ef01edbd850f90ad9bfeae2c4
   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
   

2. confirm the version and running status:

   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# curl localhost:1036/version
   {
   "orion" : {
   "version" : "3.7.0-next",
   "uptime" : "0 d, 0 h, 0 m, 59 s",
   "git_hash" : "7b45e6e4b80861dca5ac91f4c72806878756bcfd",
   "compile_time" : "Wed Sep 28 14:58:27 UTC 2022",
   "compiled_by" : "root",
   "compiled_in" : "2ef47310d08d",
   "release_date" : "Wed Sep 28 14:58:27 UTC 2022",
   "machine" : "x86_64",
   "doc" : "https://fiware-orion.rtfd.io/",
   "libversions": {
    "boost": "1_74",
    "libcurl": "libcurl/7.74.0 OpenSSL/1.1.1n zlib/1.2.11 brotli/1.0.9 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.43.0 librtmp/2.3",
    "libmosquitto": "2.0.12",
    "libmicrohttpd": "0.9.70",
    "openssl": "1.1",
    "rapidjson": "1.1.0",
    "mongoc": "1.17.4",
    "bson": "1.17.4"
   }
   }
   }
   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
   

3. Created Entity Room1 and Room2 of type Room and Building1 of entity-type Building entity-id=Room1, entity-type=Room:

   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# curl -v localhost:1036/v2/entities -s -S -H 'Content-Type: application/json' -d @- < {
   >    "id": "Room1",
   >    "type": "Room",
   >    "temperature": {
   >      "value": 23,
   >      "type": "Float"
   >    },
   >    "pressure": {
   >      "value": 720,
   >      "type": "Integer"
   >    }
   > }
   > EOF
   * About to connect() to localhost port 1036 (#0)
   *   Trying ::1...
   * Connected to localhost (::1) port 1036 (#0)
   > POST /v2/entities HTTP/1.1
   > User-Agent: curl/7.29.0
   > Host: localhost:1036
   > Accept: */*
   > Content-Type: application/json
   > Content-Length: 158
   >
   * upload completely sent off: 158 out of 158 bytes
   < HTTP/1.1 201 Created
   < Connection: Keep-Alive
   < Content-Length: 0
   < Location: /v2/entities/Room1?type=Room
   < Fiware-Correlator: 68f77ac0-3fae-11ed-a749-0242ac110008
   < Date: Thu, 29 Sep 2022 04:23:04 GMT
   <
   * Connection #0 to host localhost left intact
   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
   

   entity-id=Room2, entity-type=Room:

   [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# curl -v localhost:1036/v2/entities -s -S -H 'Content-Type: application/json' -d @- < {
   >    "id": "Room2",
   >    "type": "Room",
   >    "temperature": {
   >      "value": 23,
   >      "type": "Integer"
   >    },
   >   "pressure": {
   >      "value": 720,
   >      "type": "Float"
   >    }
   > }
   > EOF

• About to connect() to localhost port 1036 (#0)
• Trying ::1...
• Connected to localhost (::1) port 1036 (#0)

  POST /v2/entities HTTP/1.1 User-Agent: curl/7.29.0 Host: localhost:1036 Accept: / Content-Type: application/json Content-Length: 158

• upload completely sent off: 158 out of 158 bytes < HTTP/1.1 201 Created < Connection: Keep-Alive < Content-Length: 0 < Location: /v2/entities/Room2?type=Room < Fiware-Correlator: 82e188b8-3fae-11ed-a240-0242ac110008 < Date: Thu, 29 Sep 2022 04:23:47 GMT <
• Connection #0 to host localhost left intact [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# entity-id=Building1, entity-type=Building:

  [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# curl -v localhost:1036/v2/entities -s -S -H 'Content-Type: application/json' -d @- < {
  >   "id": "Building1",
  >   "type": "Building",
  >   "temperature": {
  >     "value": 11,
  >     "type": "String"
  >   },
  >   "pressure": {
  >     "value": 111,
  >     "type": "String"
  >   }
  > }
  > EOF

• About to connect() to localhost port 1036 (#0)
• Trying ::1...
• Connected to localhost (::1) port 1036 (#0)

  POST /v2/entities HTTP/1.1 User-Agent: curl/7.29.0 Host: localhost:1036 Accept: / Content-Type: application/json Content-Length: 156

• upload completely sent off: 156 out of 156 bytes < HTTP/1.1 201 Created < Connection: Keep-Alive < Content-Length: 0 < Location: /v2/entities/Building1?type=Building < Fiware-Correlator: b2066f32-3fae-11ed-958e-0242ac110008 < Date: Thu, 29 Sep 2022 04:25:06 GMT <
• Connection #0 to host localhost left intact [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
  4. Get /v2/types/, /v2/types/Room, and /v2/types/Building, Getting correct response.

     [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# curl localhost:1036/v2/types/Room -s -S  -H 'Accept: application/json' | python -mjson.tool
     {
     "attrs": {
     "pressure": {
         "types": [
             "Float",
             "Integer"
         ]
     },
     "temperature": {
         "types": [
             "Float",
             "Integer"
         ]
     }
     },
     "count": 2
     }
     [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
     

     [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# curl localhost:1036/v2/types/Building -s -S  -H 'Accept: application/json' | python -mjson.tool
     {
     "attrs": {
     "pressure": {
         "types": [
             "String"
         ]
     },
     "temperature": {
         "types": [
             "String"
         ]
     }
     },
     "count": 1
     }
     [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
     

     [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]# curl localhost:1036/v2/types -s -S  -H 'Accept: application/json' | python -mjson.tool                           [
     {
     "attrs": {
         "pressure": {
             "types": [
                 "String"
             ]
         },
         "temperature": {
             "types": [
                 "String"
             ]
         }
     },
     "count": 1,
     "type": "Building"
     },
     {
     "attrs": {
         "pressure": {
             "types": [
                 "Float",
                 "Integer"
             ]
         },
         "temperature": {
             "types": [
                 "Float",
                 "Integer"
             ]
         }
     },
     "count": 2,
     "type": "Room"
     }
     ]
     [root@fiware-stdsv5-20 telefonicaiot-fiware-orion]#
     

[fgalan]

I've tested and /v2/types/ APIs are working fine

Based on this positive report, we are going to close the issue.

Thanks for the feedback!