[<Ray Docs: "Monitoring and debugging" navigation>]

ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

https://ray.io

Apache License 2.0

31.98k stars 5.44k forks source link

[<Ray Docs: "Monitoring and debugging" navigation>] #40330

Open Wendi-anyscale opened 8 months ago

Wendi-anyscale commented 8 months ago

Description

We did a bunch of user sessions for docs in Q3, here is the abstract navigation structure that feels more intuitive to users:

Monitor & Debugging

Supported Metrics
How to collect Metrics
Monitor with Ray dashboard
Monitor with CLI or SDK
How to debug

@scottsun94 @rkooo567 @alanwguo @angelinalg What would be the layout of the "Monitor & Debug" section based on this structure? (see below for the proposed new navigation: https://github.com/ray-project/ray/issues/40331)

Link

No response

alanwguo commented 8 months ago

The high level outline makes sense to me. I re-ordered it a bit and put some of the existing docs under the new headings:

Monitor with Ray dashboard
- each page
Monitor with CLI or SDK
- state api reference
Metrics
- What are metrics. What is prometheus
- supported exported metrics
- how to add custom metrics
- how to collect/visualize metrics
- setup guides for prometheus and grafana
How to debug
- profiling
- logging
- tracing

There's also some observability docs in the serve documentation and in the future, the ray data documentation. I think it makes sense to keep those docs there.

scottsun94 commented 8 months ago

The high-level outline makes sense. I would suggest some edits on top of what @alanwguo have

Get started
- install ray with dashboard component. Start a local ray cluster. Run a simple script. Open the Ray Dashboard and view related info.
Key concepts
Monitor with Ray dashboard
- each page
Monitor with CLI or SDK
- state api reference
Metrics
- What are metrics. What is prometheus
- supported exported metrics
- how to add custom metrics
- how to collect/visualize metrics
- link to setup guides for prometheus and grafana in the cluster section since people who deploy clusters should set up those tools.
logging
profiling
tracing
debugging and optimization guides
- common gotchas
- memory issues
- hanging applications
- application failures
- performance optimization
- using Ray Debugger

btw, I left my comments about the high-level outline in https://github.com/ray-project/ray/issues/40331.

anyscalesam commented 8 months ago

@scottsun94 @Wendi-anyscale priorty and ownership of who will submit the PR to reformat into the proposed above

scottsun94 commented 8 months ago

It seems that contents are there. We just need to restructure it. I don't know if @alanwguo and @rkooo567 have time for this.

Maybe @angelinalg can help here?

anyscalesam commented 8 months ago

@angelinalg do you have an opinion on when we need to prioritize this? it is p1 so something we should pick up this or next sprint. if untrue we can downgrade this.