Add support for pip install -e

CoffeaTeam / lpcjobqueue

A dask-jobqueue plugin for the LPC Condor queue

BSD 3-Clause "New" or "Revised" License

12 stars 11 forks source link

Add support for pip install -e #5

Closed lgray closed 1 year ago

lgray commented 3 years ago

Allow shutil to follow symlinks through copytree.

nsmith- commented 2 years ago

Condor doesn't follow symlinks though? Or at least that was the case in https://github.com/CoffeaTeam/lpcjobqueue/blob/fa5d0a110663769f1898e295cb81aafb3ed37c1c/bootstrap.sh#L23

nsmith- commented 1 year ago

Can you confirm that this worked? Because at least at one point it didn't appear that HTCondor would copy the symlink in transfer_input_files

lgray commented 1 year ago

No - you're right it does not. Only real directories function here.

nsmith- commented 1 year ago

Closing for now. It would be a nice feature but I suppose it requires some refactor. Perhaps zipping the environment and then using the job_script_prologue to unzip on the worker would work well enough?