[Connectors][ServiceNow] Support of CI and Assignment group fields

cnasikas commented 5 months ago

Sumarry

ServiceNow incidents support Configuration items (CI) and Assignment groups. At the moment, the Elastic ServiceNow integration does not support those fields. We should update the ServiceNow connector's UI and APIs to support those fields.

Technical details

Changes to the Elastic ServiceNow applications

The Elastic ServiceNow applications use an import set table (temporary) where the incident is created when a request is made to SN from Kibana. Then with the user of a transformation map, the data are copied to the main incident table. Our transformation map lacks the mapping between our import set table and the main incident table for the Configuration item (cmdb_ci). To be able to support the CI we need to update the mapping in our Elastic ServiceNow applications to support the new field.

APIs

In the UI we need to make two API calls to fetch all the available CI and assignment groups. We should create two subactions for each API call where the UI would call to get the results. Users will choose from the results the item they are interested in.

Get all CI from SN:

GET /api/now/cmdb/instance/cmdb_ci

{
    "result": [
        {
            "sys_id": "00a96c0d3790200044e0bfc8bcbe5db4",
            "name": "MacBook Pro 15\""
        },
        {
            "sys_id": "00a9a80d3790200044e0bfc8bcbe5d1c",
            "name": "MacBook Air 13\""
        },
     ]
}

The response can contain thousands of items. The default limit is set to 10000. The API offers searching, filtering, and pagination capabilities.

Get all assignment groups from SN

GET /api/now/table/sys_user_group

{
    "result": [
        {
            "sys_id": "019ad92ec7230010393d265c95c260dd",
            "name": "Analytics Settings Managers"
        },
        {
            "sys_id": "0a52d3dcd7011200f2d224837e6103f2",
            "name": "Application Development"
        },
    ]
}

The response can contain hundreds of items. The default limit is set to 10000. The API offers searching, filtering, and pagination capabilities.

Challenges

The APIs, especially the CI, can return thousands of items. To handle this we have the following solutions a) Fetch only the first 10K items and b) search items while the user types. The former is easier to implement with a simpler UI but users are limited to the first 10K items. The latter needs more development effort but offers good UX and supports all items. We need to understand how often is to have more than 10K items and what it means in the UI to render 10K options. I would suggest going with the second option.

elasticmachine commented 5 months ago

Pinging @elastic/response-ops (Team:ResponseOps)

elasticmachine commented 5 months ago

Pinging @elastic/response-ops-cases (Feature:Cases)

shanisagiv1 commented 5 months ago

In addition, once these are added, we should support the following : (expiry field)

Customer Details - Name: Elastic ServiceNow Team Customer Details - Use Case: When the data is coming from Elastic for ITSM ServiceNow Connector for incident, expiry date should come along with for the certificate category Customer Details - Products/Versions: ServiceNow, Washington Customer Details - Workarounds (if any): NA Customer Details - Feature Request: Same as use case - expiry date/valid date details should also come along with payload in service now incident,alert tickets

cnasikas commented 3 months ago

Closed in favor of https://github.com/elastic/kibana/issues/183609

elastic / kibana