Prefect 2 - current working directory changes

chandruexf commented 2 weeks ago

First check

[X] I added a descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the Prefect documentation for this issue.
[X] I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

os.getcwd() I used this line in the task , what my expectation is /home/chandru/prefectver2/ed-2-app/app/ to give this path, but I got the result in logger /tmp/tmpa97l021qprefect some other path and not able to overcome this with the deployment.

Reproduction

deployment = Deployment.build_from_flow(
        flow=create_new_flow,  # replace with flow name
        name=user_tenant,  #replace with deployment name
        work_queue_name="test-queue",  # replace with agent name
        work_pool_name = "test-pool",
        path='/home/chandru/prefectver2/ed-2-app/app/',
        entrypoint='/home/chandru/prefectver2/ed-2-app/app/project/components/exfinsights/data_quality/prefect_layer2/tasks.py:create_new_flow', # replace with "flow file name : flow name"
        # schedule=IntervalSchedule(interval=timedelta,anchor_date=datetime.datetime.now(),timezone="UTC"),
        # parameters={"job_params":{
        #     "group_name":test_group_name,
        #     "user_tenant":user_tenant,
        #     "selected_version_data":selected_version_data
        # }}
    )

my deployment

Error

os.getcwd() I used this line in the task , what my expectation is /home/chandru/prefectver2/ed-2-app/app/ to give this path, but I got the result in logger /tmp/tmpa97l021qprefect some other path and not able to overcome this with the deployment.

Versions (`prefect version` output)

my op need to be /home/chandru/prefectver2/ed-2-app/app/  system path

Additional context

No response

zhen0 commented 2 weeks ago

Hi @chandruexf - thanks for the issue. Can I ask that you provide a full MRE and prefect version for us to be able to look into this further? It really helps us know if you are on a version where an issue has already been fixed and makes sure we're looking into the same thing as you are reporting.

You also look to be using the older build_from_flow way of creating a deployment - it might be worth checking out flow.deploy to create your MRE.

chandruexf commented 2 weeks ago

Hi Jenny,

name : prefect version : 2.19.6 description : Workflow orchestration and management.

I am working on this prefect version and /tmp/somename setting has its current working directory,

On Mon, 1 Jul 2024 at 20:30, Jenny G @.***> wrote:

Hi @chandruexf https://github.com/chandruexf - thanks for the issue. Can I ask that you provide a full MRE and prefect version for us to be able to look into this further? It really helps us know if you are on a version where an issue has already been fixed and makes sure we're looking into the same thing as you are reporting.

You also look to be using the older build_from_flow way of creating a deployment - it might be worth checking out flow.deploy https://docs.prefect.io/latest/guides/upgrade-guide-agents-to-workers/ to create your MRE.

— Reply to this email directly, view it on GitHub https://github.com/PrefectHQ/prefect/issues/14433#issuecomment-2200398537, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7KFZUY7XTBLCCIPVGGKJP3ZKFVH3AVCNFSM6AAAAABKFQXG6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBQGM4TQNJTG4 . You are receiving this because you were mentioned.Message ID: @.***>

zhen0 commented 2 weeks ago

Hi @chandruexf - Here's some guidance for what I'm looking for when I ask for an MRE.

For the prefect version, can you please provide all of the information given when running the prefect version command?

As we say in our issue template: Provide information about your Prefect version and environment. The easiest way to retrieve all of the information we require is the prefect version command. If using Prefect 1.x, it is useful to also include the output of prefect diagnostics. Please do not just include your Prefect version number. The command provides additional context such as your operating system, Prefect API type, Python version, etc. that we need to diagnose your problem.

chandruexf commented 2 days ago

Hi Jenny,

Some how I am able to all the tasks using the prefect 2. I need your help on.

When I try to use os.getcwd() I am getting default path which should not happen. /tmp/tmpz2orrx79prefect dyname=dynamic_flname(prefect_flow_name) deployment = Deployment.build_from_flow( flow = dyname, name=user_tenant, work_queue_name="test-queue", work_pool_name = "test-pool", path=f'{str(os.getcwd())}/', # entrypoint=f'{str(os.getcwd())}/project/components/exfinsights/data_quality/prefect_layer2/tasks.py:exfdq_app_wf',

replace with "flow file name : flow name"
```
    #
```
schedule=IntervalSchedule(interval=timedelta,anchor_date=datetime.datetime.now(),timezone="UTC"), parameters = { "group_name" : test_group_name, "user_tenant" : user_tenant, "selected_version_data" : selected_version_data, "user_name" : user_name } )
```
deployment_id = deployment.apply(upload=True,)
```
My deployment method that is used What should i include to overcome this current directory.

Thanks chandru

On Tue, 2 Jul 2024 at 19:11, Jenny G @.***> wrote:

Hi @chandruexf https://github.com/chandruexf - Here's some guidance https://stackoverflow.com/help/minimal-reproducible-example for what I'm looking for when I ask for an MRE.

For the prefect version, can you please provide all of the information given when running the prefect version command?

As we say in our issue template: Provide information about your Prefect version and environment. The easiest way to retrieve all of the information we require is the prefect version command. If using Prefect 1.x, it is useful to also include the output of prefect diagnostics. Please do not just include your Prefect version number. The command provides additional context such as your operating system, Prefect API type, Python version, etc. that we need to diagnose your problem.

— Reply to this email directly, view it on GitHub https://github.com/PrefectHQ/prefect/issues/14433#issuecomment-2203201196, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7KFZUZKJSL7LYIYN647BB3ZKKUYRAVCNFSM6AAAAABKFQXG6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBTGIYDCMJZGY . You are receiving this because you were mentioned.Message ID: @.***>

PrefectHQ / prefect