search

FIWARE / context.Orion-LD

Context Broker and CEF building block for context data management which supports both the NGSI-LD and the NGSI-v2 APIs
https://www.etsi.org/deliver/etsi_gs/CIM/001_099/009/01.06.01_60/gs_CIM009v010601p.pdf
GNU Affero General Public License v3.0
51 stars 43 forks source link

Problems in cache for Context Source Registrations #1692

Open ravaga opened 1 month ago

ravaga commented 1 month ago

I'm working in the development of an application which manages (creates, updates and deletes) Context Source Registrations (CSRs) in Orion-LD, so this application sends several request to the /ngsi-ld/v1/csourceRegistrations endpoint of the broker.

While performing some tests, I came across an issue related with the internal cache of the broker. Sometimes, when I create new CSRs and I send a GET request to retrieve all the CSRs present in the broker, the newest registrations aren't returned. However, if I make a query inside MongoDB (inside the registrations collection of the orion DB) the registrations are stored. Therefore, this must be an issue related with the internal Orion-LD cache. Furthermore, if I use the options=fromDb parameter to retrieve the CSR, the broker returns it, but it returns a 404 if the parameter isn't used.

In addition, if I try to recreate the same CSRs (using the same IDs), the broker returns a 409 because it looks for the registration in MongoDB first. Furthermore, the CSR is deleted without any problem.

ravaga commented 1 month ago

Let me add an specific example (in the context of the aerOS project) to try to shed light on the issue.

We have an environment composed of 3 Orion-LD instances (A,B and C) and we want to create the needed CSR in each broker that point to the other 2 brokers to be able to retrieve all the entities of a certain type by making a single GET /entities request to any of those brokers. We use 3 different types of CSRs, so each broker must have a total of 6 CSRs (3 pointing to each other broker) in each broker. The creation and deletion of these CSRs is managed by the custom Federator component.

First, we deploy the broker A and then we create one entity of type Domain, other of type Organization, another of type LowLevelOrchestrator, some of type InfrastructureElement and some of type Organization. This is the first broker, so we don't have to create CSRs yet.

Then, we deploy the broker B and create a new set of entities (with the same types as in broker A). This is the second deployed broker, so the Federator creates these 3 CSR in the broker A:

{
        "id": "urn:aeros:federation:b:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}

And these 3 CSRs in the broker B:

{
        "id": "urn:aeros:federation:a:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:a:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:a:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}

Now we have 2 CBs in our environment, but we want to add a third CB (broker C). The Federator creates 3 more CSRs in broker A and broker B pointing to the new broker C:

{
        "id": "urn:aeros:federation:c:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainC",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.103:1026",
        "aerosDomain": "DomainC",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:c:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainC",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.103:1026",
        "aerosDomain": "DomainC",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:c:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainC",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.103:1026",
        "aerosDomain": "DomainC",
        "aerosDomainFederation": true
}

Finally, the Federator creates 6 CSRs in broker C pointing to the existing brokers A and B:

{
        "id": "urn:aeros:federation:a:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:a:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:a:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
ravaga commented 1 month ago

After all these CSR creations, the environment is ready (unless the bug appears and the registrations aren't updated in the cache). Therefore, we can query all the entities described in the registrations and retrieve all the created registrations for management tasks.

After some time, a new broker can be added to the environment (each broker will have 3 more registrations pointing to it, and the new broker will have a total of 9 registrations).

In the same way, a broker can be removed from the system. When a broker is removed (let's say we want to remove broker B), the federator removes all the CSRs with the attribute "aerosDomainFederation": true (it sends a GET /csourceRegistrations?csf=aerosDomainFederation==true and then a DELETE /csourceRegistrations/{regID} for each CSR). Then, it removes all the CSRs in brokers A and C that point to broker B (urn:aeros:federation:b:infrastructure, urn:aeros:federation:b:organizations and urn:aeros:federation:b:services).

The bug in the cache use to appear after the deletion of a broker from the environment or after the addition of multiple borkers. However and unfortunatelly, I haven't been able to find a pattern for the bug or reproduce it intentionally.