apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.39k stars 181 forks source link

Setting job name is not reflected in the Web UI #1019

Open jamesmcm opened 1 month ago

jamesmcm commented 1 month ago

Describe the bug

I'm setting the job name with:

    let start = SystemTime::now();
    let since_the_epoch = start
        .duration_since(UNIX_EPOCH)
        .expect("Time went backwards");

    let n: usize = 10;
    let job_name = format!("top_{}_{}", n, since_the_epoch.as_secs());

    let config = BallistaConfig::builder()
        .set("ballista.job.name", &job_name)
        .build()?;

But it isn't shown in the Web UI (None is shown as the job name).

This is the same both locally and in a GKE cluster.

To Reproduce

Run a job with the above config.

Expected behavior

The job name should appear in the Web UI

Additional context

It seems the relevant code is https://github.com/apache/datafusion-ballista/blob/04766d5fd7cbb15d583f736979c11de06c720928/ballista/core/src/config.rs#L30 but I can't see anything wrong.