[Meta] [ML] DF Analytics map

[alvarezmelissa87]

Meta issue for DF Analytics map

General

• [x] Add Map tab to analytics management view
• [x] Map tab should persist until new job/model is selected (will need to persist selected jobId/modelId for map)
• [x] Endpoints should use new es client
• [x] switch 'inference model' node type to 'trained model'
• [x] Support for dark theme
• [x] Format create time to human readable
• [x] Handle source index pattern with wildcard
• [x] Should be space aware
  • [x] Should only show jobs and indices that exist in the same space as the root job/model
• [x] Can link to map just with url (so it’s share-able)
  • [x] _g=(ml:(jobId:classification-clientip-04,analysisType:classification,isMap:true))
  • [x] add new endpoints to apidoc.json
  • [x] add modelId in url params to start map from model

Map

• [ ] Add map action to
  • [x] legend should have job type icons
  • [ ] Map action icon should be all one color
    • awaiting https://github.com/elastic/eui/issues/4205 - [ ] Icons for transform, index, and inference model need to be added ~~ - awaiting https://github.com/elastic/eui/issues/4205~~
  • [x] DFA job list
  • [x] model list - [ ] only show model if there is something to show (created from job, etc)
• [x] map action disabled if job is not complete
• [x] Include root inference model in node map
• [x] select index pattern node to show all jobs created from it
• [x] Only show details for selected node (in a list similar to expanded row)
• [x] Show type of job - add icons
• [x] Highlight source node (coming from job/model)
• [x] Include indices between jobs in node map
• [x] Create map from:
  • [x] analyticsId
  • [x] inferenceModelId
• [x] Ability to delete jobs (https://github.com/elastic/kibana/pull/84299)
  • [x] should have same deletion flow as from list
  • [x] ensure map nodes updated after deletion
  • [x] clear url and remove map tab if root job is being deleted
• [x] Ability to create new DFA jobs off nodes in the map e.g. select transform node, and create classification job off the index created by the transform (https://github.com/elastic/kibana/pull/84299)
• [x] Ability to clone DFA jobs off nodes in the map (https://github.com/elastic/kibana/pull/84299)
• [x] Add 'Fit to page' (or 'reset zoom') button to the map - I often find myself zooming in / out by accident and then want to reset to the default zoom level (https://github.com/elastic/kibana/pull/84299)
• [x] Add refresh button for map so it gets refetched (https://github.com/elastic/kibana/pull/84299)

Tests

• [x] Add functional tests
• [x] Add API integration tests

Code level

• [x] use url generator for map url
• [x] move away from using dependency cache for toasts (in job map component)

Known issues to fix

• [x] When model's job has been deleted and map is started from the model's list nothing is returned. One node for that model should be returned. Or the map action should be disabled. (https://github.com/elastic/kibana/pull/84299)
• [x] When scrolling in the flyout, the cytoscape window also zooms in/out - that shouldn't happen
• [x] When in map tab, the Refresh button is in loading state
• [ ] cytoscape options - background image property needs a default icon to avoid console warnings
• [ ] Flyout cannot be re-opened until selected node gets deselected
• [x] When there is only one "row" of nodes (e.g. not multiple related jobs that are distributed vertically), the graph ends up at the top of the visible area and is cut off
• [ ] With narrow nodes the labels get hidden when there are many leaf nodes from one parent node.
• [x] Double scrollbar

Screenshots

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[peteharverson]

Feedback from testing of https://github.com/elastic/kibana/pull/81666:

• Format the create_time in the flyout as a date / time
• Add ability to create new DFA jobs off nodes in the map e.g. select transform node, and create classification job off the index created by the transform
• Add 'Fit to page' (or 'reset zoom') button to the map - I often find myself zooming in / out by accident and then want to reset to the default zoom level
• Is it possible to include the different types of DFA job in the legend, rather than just have the one circular job for analytics jobs?
• Add the transform / DFA job description and DFA job type to the flyout
• Is it possible to reduce the minimum spacing between nodes? For example, here why is the text for the last node not vertically aligned with the other nodes?

[walterra]

Looks like switching to a horizontal layout is definitely an improvement! With narrow nodes the labels tend to get hidden though.

Suggestions:

• Maybe cytoscape supports positioning labels to the right of nodes (would also left-align the text and make lots of nodes of different length easier to read)
• An alternative could be to hide labels if there are more than a certain threshold (similar to how nodes are hidden when zooming out) and showing labels only on hover then.

[walterra]

When there is only one "row" of nodes (e.g. not multiple related jobs that are distributed vertically), the graph ends up at the top of the visible area and is cut off:

[peteharverson]

I noticed when testing #84299 that if you decrease the size of the window after the initial render that the overflow isn't scrollable. Is this a known issue / 'feature' with Cytoscape?

[alvarezmelissa87]

All left over items have been moved to a follow up meta issue: https://github.com/elastic/kibana/issues/86487

Please add any additional feedback there