ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
32.93k stars 5.57k forks source link

[Train] Disable autofilled metrics #21988

Open bveeramani opened 2 years ago

bveeramani commented 2 years ago

Search before asking

Description

Currently, Train reports are autofilled with various metrics.

train.report(epoch=1)
# {'epoch': 1, '_timestamp': 1630716913, '_time_this_iter_s': 0.0039408206939697266, '_training_iteration': 1}

Users might not care about the autofilled metrics. To improve the readability, we could add a feature that disables autofilled metrics.

train.report(epoch=1)
# {'epoch': 1}

Use case

Adding this feature could make reported data easier to work with.

Additionally, disabling autofilled metrics could improve the quality of our tests. Right now, tests are complicated as to handle the autofilled metrics.

assert all(
        all(all(key in worker for key in BASIC_AUTOFILLED_KEYS) for worker in element)
        for element in log
    )

Related issues

No response

Are you willing to submit a PR?

anyscalesam commented 2 months ago

Discussed with @matthewdeng; downgrading > might be fixed already Matt can you confirm?