search

oap-project / raydp

RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.
Apache License 2.0
293 stars 66 forks source link

[raydp-334] fix the workaround of working with ray 2.3.[0-1] #335

Closed jiafuzha closed 1 year ago

jiafuzha commented 1 year ago

we need to prepend jobid to hardcoded prefix log file, java-worker-*.log instead of getting prefix from the "ray.logging.file-prefix" system property since ray hasn't applied the fix (https://github.com/ray-project/ray/pull/33665) yet.

jiafuzha commented 1 year ago

@kira-lin @carsonwang please help review.

kira-lin commented 1 year ago

Why revert the change? It works both now and after that PR merged, right?

jiafuzha commented 1 year ago

Why revert the change? It works both now and after that PR merged, right?

It doesn't work as expected with ray 2.3.1 for now since ray still uses 'java-worker' as prefix even though we've set and get system property like below,

String prefix = System.getProperty("ray.logging.file-prefix", "java-worker");

Thus, we need to prepend "jobid" to java-worker*.log directly until the ray PR merged.

kira-lin commented 1 year ago

I see. thanks