Propagating prefect task names to ray tasks instead of it showing up as _run_prefect_task in the Ray dashboard

[achordia20]

Expectation / Proposal

Propagate the prefect task name (whether function or manual name override in the task decorator) to the ray task.

Traceback / Example

• [ ] I would like to help contribute a pull request to resolve this!

[azmyrajab]

Curious if anyone has been able to take a look at this?

One could achieve this by using .options as per below

        if remote_options:
            ray_decorator = ray.remote(**remote_options)
        else:
            ray_decorator = ray.remote

        self._ray_refs[key] = ray_decorator(self._run_prefect_task).options(name=user_specified_task_name).remote(
            sync_compatible(call.func), *upstream_ray_obj_refs, **call_kwargs
        )

[j-tr]

@azmyrajab Here's my attempt to solve this: #103. Happy to incorporate any feedback. From what I understand, options can only be used to change properties that can be set via the remote options (i.e. parameters to https://docs.ray.io/en/latest/ray-core/api/doc/ray.remote.html). Name is none of them.

[azmyrajab]

Hi Justin, thank you so much for taking a look at this

Fyi its a trick that does work, but I think its a feature that is not well documented as you noticed. It was added here https://github.com/ray-project/ray/issues/10371 to address these type of scenarios where a factory function produces multiple tasks.

Here is an example I just ran by 'patching' the Ray Prefect source:

        if remote_options:
            ray_decorator = ray.remote(**remote_options)
        else:
            ray_decorator = ray.remote

        self._ray_refs[key] = ray_decorator(self._run_prefect_task).options(name="hello_world").remote(
            sync_compatible(call.func), *upstream_ray_obj_refs, **call_kwargs
        )

In any case taking a look at your https://github.com/PrefectHQ/prefect-ray/pull/103 now - will be very grateful if it solves this problem no matter

[azmyrajab]

Yes, I think the ray feature is intended for that (see https://github.com/ray-project/ray/pull/10449/files for the implementation), and instead of modifying __qualname__ you can propagate the prefect task name via options and I think this will be a nice solution

[j-tr]

thank you for pointing me to #10371. just changed my MR to use options instead.

[j-tr]

desertaxle happy to receive any feedback on #103.