openBackhaul / MicroWaveDeviceInventory

Physical and logical inventory of the MW SDN Domain
Apache License 2.0
5 stars 1 forks source link

[Ops] MWDI is stopped unexpectedly. #949

Open Ziabatcha opened 6 months ago

Ziabatcha commented 6 months ago

NP is enabled and MWDI is subscribed only for "notify-controller-attribute-value-changes". Device disconnected and connected status are updated MWDI successfully. Unexpectedly MWDI stopped with below error prints

Error prints:

[27.04.2024 00:08.33.375] [ERROR] Elasticsearch error occurred: {"_index":"mwdi_preprod_1.1.2.a_impl-000001","_type":"_doc","_id":"306558261B","_version":3,"result":"not_found","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":74634,"_primary_term":1}
/home/openbackhaul/microWaveDeviceInventory/node_modules/@elastic/elasticsearch/lib/Transport.js:367
        const error = new ResponseError(result)
                      ^

ResponseError: {"_index":"mwdi_preprod_1.1.2.a_impl-000001","_type":"_doc","_id":"306558261B","_version":3,"result":"not_found","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":74634,"_primary_term":1}
    at onBody (/home/openbackhaul/microWaveDeviceInventory/node_modules/@elastic/elasticsearch/lib/Transport.js:367:23)
    at IncomingMessage.onEnd (/home/openbackhaul/microWaveDeviceInventory/node_modules/@elastic/elasticsearch/lib/Transport.js:291:11)
    at IncomingMessage.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1358:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  meta: {
    body: {
      _index: 'mwdi_preprod_1.1.2.a_impl-000001',
      _type: '_doc',
      _id: '306558261B',
      _version: 3,
      result: 'not_found',
      _shards: { total: 2, successful: 1, failed: 0 },
      _seq_no: 74634,
      _primary_term: 1
    },
    statusCode: 404,
    headers: {
      'x-elastic-product': 'Elasticsearch',
      warning: '299 Elasticsearch-7.14.2-6bc13727ce758c0e943c3c21653b3da82f627f75 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.14/security-minimal-setup.html to enable security."',
      'content-type': 'application/json; charset=UTF-8',
      'content-length': '195'
    },
    meta: {
      context: null,
      request: {
        params: {
          method: 'DELETE',
          path: '/mwdi_preprod_1.1.2.a_impl/_doc/306558261B',
          body: '',
          querystring: '',
          headers: {
            'user-agent': 'elasticsearch-js/7.17.12 (linux 4.18.0-240.22.1.el8_3.x86_64-x64; Node.js v16.20.2)',
            'x-elastic-client-meta': 'es=7.17.12,js=16.20.2,t=7.17.12,hc=16.20.2'
          },
          timeout: 2000
        },
        options: {},
        id: 54805
      },
      name: 'elasticsearch-js',
      connection: Connection {
        url: <ref *1> URL {
          [Symbol(context)]: URLContext {
            flags: 400,
            scheme: 'http:',
            username: '',
            password: '',
            host: '10.20.64.4',
            port: 9200,
            path: [Array],
            query: null,
            fragment: null
          },
          [Symbol(query)]: URLSearchParams {
            [Symbol(query)]: [],
            [Symbol(context)]: [Circular *1]
          }
        },
        ssl: null,
        id: 'http://x.x.x.x:xxxx/',
        headers: { authorization: 'ApiKey API key not yet defined.' },
        deadCount: 0,
        resurrectTimeout: 0,
        caFingerprint: null,
        _openRequests: 0,
        _status: 'alive',
        roles: { master: true, data: true, ingest: true, ml: false },
        agent: Agent {
          _events: [Object: null prototype] {
            free: [Function (anonymous)],
            newListener: [Function: maybeEnableKeylog]
          },
          _eventsCount: 2,
          _maxListeners: undefined,
          defaultPort: 80,
          protocol: 'http:',
          options: [Object: null prototype] {
            keepAlive: true,
            keepAliveMsecs: 1000,
            maxSockets: 256,
            maxFreeSockets: 256,
            scheduling: 'lifo',
            path: null
          },
          requests: [Object: null prototype] {},
          sockets: [Object: null prototype] {},
          freeSockets: [Object: null prototype] { 'x.x.x.x:xxxx:': [Array] },
          keepAliveMsecs: 1000,
          keepAlive: true,
          maxSockets: 256,
          maxFreeSockets: 256,
          scheduling: 'lifo',
          maxTotalSockets: Infinity,
          totalSocketCount: 2,
          [Symbol(kCapture)]: false
        },
        makeRequest: [Function: request]
      },
      attempts: 1,
      aborted: false
    }
  }
}
nardecchia-siae commented 5 months ago

This was difficult to reproduce in lab, but we have recreated the same issue removing the part of code which control the presence of the CC in cache. Then we have protected that part so, in case of uncorrect delete of an inexistent CC in cache, the application don't crash anymore. The control of existence has been added again so now this behavior is double protected.