search

dask / dask-yarn

Deploy dask on YARN clusters
http://yarn.dask.org
BSD 3-Clause "New" or "Revised" License
69 stars 41 forks source link

.skein.sh: line 2: environment/bin/python: No such file or directory #159

Open jahd opened 2 years ago

jahd commented 2 years ago

I'm trying to run a dask yarn_cluster job on dataproc but i can't.

Dask worker log is giving me this issue:

.skein.sh: line 2: environment/bin/python: No such file or directory

while the packaged environnement i give to my yarn_cluster has python3.8 on it.

any idea ?

jacobtomlinson commented 2 years ago

Could you share a small reproducible example of how you came to this problem so we can investigate?

jahd commented 2 years ago

yes here it is :

LogType:dask.worker.log LogLastModifiedTime:Thu Aug 11 18:44:21 +0000 2022 LogLength:411 LogContents: Could not find platform independent libraries Consider setting $PYTHONHOME to [:] Fatal Python error: initfsencoding: Unable to get the locale encoding ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fd28c2a6740 (most recent call first): .skein.sh: line 2: 1442 Aborted (core dumped) environment/bin/python -m dask_yarn.cli services worker

jahd commented 2 years ago

i've tried common solutions already but i can't make it work, i'm using virtualenv

jacobtomlinson commented 2 years ago

Sorry I mean could you share your code and configuration so we can reproduce?