AmericanPresidentJimmyCarter
commented
2 years ago @mariepop13 Please give me the output of these two commands: python --version and cat flow.tmp.yml.
Open mariepop13 opened 2 years ago
AmericanPresidentJimmyCarter
commented
2 years ago @mariepop13 Please give me the output of these two commands: python --version and cat flow.tmp.yml.
mariepop13
commented
2 years ago @AmericanPresidentJimmyCarter
paperspace@pspkgwgm9:~$ python3 --version Python 3.8.10
paperspace@pspkgwgm9:~$ cat flow.tmp.yml cat: flow.tmp.yml: No such file or directory
AmericanPresidentJimmyCarter
commented
2 years ago OK, so python flow_parser.py did not run. Have you tried python3 flow_parser.py? If you are running outside of a virtualenv you will need to specify the python version.
mariepop13
commented
2 years ago I had to reinstall jina by pip. During the installation ERROR: docker 6.0.0 has requirement requests>=2.26.0, but you'll have requests 2.22.0 which is incompatible.
Still the same error as first pic
than I tried again cat flow.tmp.yml
paperspace@pspkgwgm9:~/dalle-flow$ cat flow.tmp.yml jtype: Flow with: env: JINA_LOG_LEVEL: debug monitoring: true port: 51005 protocol: grpc executors:
AmericanPresidentJimmyCarter
commented
2 years ago OK, you are indeed running it outside a virtual environment. Do this if you're trying to run stable-diffusion, from the dalle/dalle-flow folder:
sudo apt-get install python3 python3-pip
sudo pip3 install virtualenv
python3 -m virtualenv env
source env/bin/activate && cd -
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116
pip install numpy tqdm pytorch_lightning einops numpy omegaconf
pip install https://github.com/crowsonkb/k-diffusion/archive/master.zip
cd latent-diffusion && pip install -e . && cd -
cd stable-diffusion && pip install -e . && cd -
cd SwinIR && pip install -e . && cd -
cd glid-3-xl && pip install -e . && cd -Run jina with stable diffusion by:
cd dalle-flow
python flow_parser.py --enable-stable-diffusion --disable-dalle-mega --disable-glid3xl
jina flow --uses flow.tmp.ymlHi, again I was in
(env) paperspace@pspkgwgm9:~/dalle-flow/dalle/dalle-flow$ python flow_parser.py --enable-stable-diffusion --disable-dalle-mega --disable-glid3xl python: can't open file 'flow_parser.py': [Errno 2] No such file or directory
Or I have to go in ~/dalle-flow/?
I was trying to install the discord-bot too.
AmericanPresidentJimmyCarter
commented
2 years ago Can you confirm that your filesystem looks like this?
dalle/
|
|-- SwinIR/
|-- dalle-flow/
|-- glid-3-xl/
|-- latent-diffusion/
|-- stable-diffusion/Sure
I had to download from https://github.com/CompVis/latent-diffusion https://github.com/CompVis/stable-diffusion Because your link in https://github.com/AmericanPresidentJimmyCarter/dalle-flow/tree/stable-diffusion Where dead as well
(git clone https://github.com/CompVis/latent-diffusio324n.git git clone https://github.com/StableDiffusion/latent-diffusion.git)
So here what I have: (env) paperspace@pspkgwgm9:~/dalle-flow/dalle$ ls dalle-flow glid-3-xl latent-diffusion stable-diffusion SwinIR
mariepop13
commented
2 years ago (env) paperspace@pspkgwgm9:~/dalle-flow$ ls client.ipynb executors flow.tmp.yml LICENSE start.sh dalle flow-jcloud.yml flow.yml README.md Dockerfile flow_parser.py k8s_flow requirements.txt
versus
(env) paperspace@pspkgwgm9:~/dalle-flow/dalle/dalle-flow$ ls client.ipynb env flow-jcloud.yml k8s_flow README.md Dockerfile executors flow.yml LICENSE requirements.txt
AmericanPresidentJimmyCarter
commented
2 years ago Ok, it's best to start over. Running these commands will fix your installation and put everything into the ~/dalle folder. Copy the weights somewhere else if you have already downloaded them.
rm -rf ~/dalle-flow/
cd ~
mkdir dalle && cd dalle
git clone https://github.com/jina-ai/dalle-flow.git
git clone https://github.com/jina-ai/SwinIR.git
git clone https://github.com/CompVis/latent-diffusio324n.git
git clone https://github.com/StableDiffusion/latent-diffusion.git
git clone https://github.com/jina-ai/glid-3-xl.git
cd ~/dalle/dalle-flow
python3 -m virtualenv env
source env/bin/activate && cd -
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116
pip install numpy tqdm pytorch_lightning einops numpy omegaconf
pip install https://github.com/crowsonkb/k-diffusion/archive/master.zip
cd latent-diffusion && pip install -e . && cd -
cd stable-diffusion && pip install -e . && cd -
cd SwinIR && pip install -e . && cd -
cd glid-3-xl && pip install -e . && cd -Then put the weights into ~/dalle/stable-diffusion.
Finally,
cd ~/dalle/dalle-flow
python flow_parser.py --enable-stable-diffusion --disable-dalle-mega --disable-glid3xl
jina flow --uses flow.tmp.ymlI am so sorry... doesnt work. Why do I have
/home/paperspace/.local/bin/jina flow --uses
flow.yml
โญโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ Argument โ Value โ
โโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ cli โ flow โ
โ env โ None โ
โ inspect โ COLLECT โ
โ log-config โ default โ
โ name โ None โ
โ quiet โ False โ
โ quiet-error โ False โ
โ uses โ flow.yml โ
โ workspace โ None โ
โ workspace-id โ edf84112eb2f445d951de491c27a0ef5 โAnd not
/home/<user>/.local/lib/python3.9/site-packages/jina/__main__.py flow --uses flow.yml
can it be the problem???
otherwise
(env) paperspace@pspkgwgm9:~/dalle/dalle-flow$ jina flow --uses flow.tmp.yml
RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version! (raised from /usr/lib/python3/dist-packages/requests/__init__.py:89)
DeprecationWarning: 'urllib3.contrib.pyopenssl' module is deprecated and will be removed in a future release of urllib3 2.x. Read more in this issue: https://github.com/urllib3/urllib3/issues/2680 (raised from /usr/lib/python3/dist-packages/requests/__init__.py:95)
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWWWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNNNNNNNWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMNNNNNNNNNMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNNNNNNNNMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNNNWWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMWxxxxxxxxxOMMMMMNxxxxxxxxx0MMMMMKddddddxkKWMMMMMMMMMMMMXOxdddxONMMMM
MMMMMMMMMMMMXllllllllldMMMMM0lllllllllxMMMMMOllllllllllo0MMMMMMMM0olllllllllo0MM
MMMMMMMMMMMMXllllllllldMMMMM0lllllllllxMMMMMOlllllllllllloWMMMMMdllllllllllllldM
MMMMMMMMMMMMXllllllllldMMMMM0lllllllllxMMMMMOllllllllllllloMMMM0lllllllllllllllK
MMMMMMMMMMMMKllllllllldMMMMM0lllllllllxMMMMMOllllllllllllllKMMM0lllllllllllllllO
MMMMMMMMMMMMKllllllllldMMMMM0lllllllllxMMMMMOllllllllllllll0MMMMollllllllllllllO
MWOkkkkk0MMMKlllllllllkMMMMM0lllllllllxMMMMMOllllllllllllll0MMMMMxlllllllllllllO
NkkkkkkkkkMMKlllllllloMMMMMM0lllllllllxMMMMMOllllllllllllll0MMMMMMWOdolllllllllO
KkkkkkkkkkNMKllllllldMMMMMMMMWWWWWWWWWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MOkkkkkkk0MMKllllldXMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMWX00KXMMMMXxk0XMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
/home/paperspace/.local/bin/jina flow --uses
flow.tmp.yml
โญโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ Argument โ Value โ
โโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ cli โ flow โ
โ env โ None โ
โ inspect โ COLLECT โ
โ log-config โ default โ
โ name โ None โ
โ quiet โ False โ
โ quiet-error โ False โ
โ uses โ flow.tmp.yml โ
โ workspace โ None โ
โ workspace-id โ b93b245a44454b85acaf9780f31f2740 โ
โฐโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โ Waiting ... โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 0/0 -:--:--CRITIโฆ stable/rep-0@3066 can not load the executor from executors/stable/config.yml [08/26/22 19:12:35]
CRITIโฆ upscaler/rep-0@3067 can not load the executor from [08/26/22 19:12:35]
executors/swinir/config.yml
DEBUG gateway/rep-0@3069 adding connection for deployment clip_encoder/heads/0 to [08/26/22 19:12:35]
grpcs://demo-cas.jina.ai:2096
DEBUG store/rep-0@3068 <executor.DalleFlowStore object at 0x7fecb0a9f460> is [08/26/22 19:12:35]
successfully loaded!
DEBUG gateway/rep-0@3069 adding connection for deployment stable/heads/0 to
grpc://0.0.0.0:55035
DEBUG gateway/rep-0@3069 adding connection for deployment rerank/heads/0 to
grpcs://demo-cas.jina.ai:2096
DEBUG gateway/rep-0@3069 adding connection for deployment upscaler/heads/0 to
grpc://0.0.0.0:56296
DEBUG gateway/rep-0@3069 adding connection for deployment store/heads/0 to
grpc://0.0.0.0:64287
DEBUG store/rep-0@3068 start listening on 0.0.0.0:64287
DEBUG gateway/rep-0@3069 start server bound to 0.0.0.0:51005
DEBUG store/rep-0@3057 ready and listening [08/26/22 19:12:35]
DEBUG gateway/rep-0@3057 ready and listening [08/26/22 19:12:35]
โ ธ Waiting stable upscaler summary... โโโโโโโโโโโโโโโโโบโโโโโโโโโโโโโโโโโโโโโโโ 2/5 0:00:00ERROR stable/rep-0@3066 ImportError('can not import module from [08/26/22 19:12:35]
/home/paperspace/dalle/dalle-flow/executors/stable/executor.py') during
<class 'jina.serve.runtimes.worker.WorkerRuntime'> initialization
add "--quiet-error" to suppress the exception details
โญโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโฎ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/importer.py:128 โ
โ in _path_import โ
โ โ
โ 125 โ โ spec = importlib.util.spec_from_file_location(spec_name, a โ
โ 126 โ โ module = importlib.util.module_from_spec(spec) โ
โ 127 โ โ sys.modules[spec_name] = module โ
โ โฑ 128 โ โ spec.loader.exec_module(module) โ
โ 129 โ except Exception as ex: โ
โ 130 โ โ raise ImportError(f'can not import module from {absolute_p โ
โ 131 โ
โ <frozen importlib._bootstrap_external>:848 in exec_module โ
โ <frozen importlib._bootstrap>:219 in _call_with_frames_removed โ
โ โ
โ /home/paperspace/dalle/dalle-flow/executors/stable/executor.py:2 in โ
โ <module> โ
โ โ
โ 1 import PIL โ
โ โฑ 2 import k_diffusion as K โ
โ 3 import numpy as np โ
โ 4 import os โ
โ 5 import shutil โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
ModuleNotFoundError: No module named 'k_diffusion'
The above exception was the direct cause of the following exception:
โญโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโฎ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/orchestrate/poโฆ โ
โ in run โ
โ โ
โ 71 โ โ
โ 72 โ try: โ
โ 73 โ โ _set_envs() โ
โ โฑ 74 โ โ runtime = runtime_cls( โ
โ 75 โ โ โ args=args, โ
โ 76 โ โ ) โ
โ 77 โ except Exception as ex: โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in __init__ โ
โ โ
โ 28 โ โ :param kwargs: keyword args โ
โ 29 โ โ """ โ
โ 30 โ โ self._health_servicer = health.HealthServicer(experimental โ
โ โฑ 31 โ โ super().__init__(args, **kwargs) โ
โ 32 โ โ
โ 33 โ async def async_setup(self): โ
โ 34 โ โ """ โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in __init__ โ
โ โ
โ 66 โ โ โ
โ 67 โ โ self._setup_monitoring() โ
โ 68 โ โ send_telemetry_event(event='start', obj=self) โ
โ โฑ 69 โ โ self._loop.run_until_complete(self.async_setup()) โ
โ 70 โ โ
โ 71 โ def run_forever(self): โ
โ 72 โ โ """ โ
โ โ
โ /usr/lib/python3.8/asyncio/base_events.py:616 in run_until_complete โ
โ โ
โ 613 โ โ if not future.done(): โ
โ 614 โ โ โ raise RuntimeError('Event loop stopped before Future โ
โ 615 โ โ โ
โ โฑ 616 โ โ return future.result() โ
โ 617 โ โ
โ 618 โ def stop(self): โ
โ 619 โ โ """Stop running the event loop. โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in async_setup โ
โ โ
โ 57 โ โ # Keep this initialization order โ
โ 58 โ โ # otherwise readiness check is not valid โ
โ 59 โ โ # The DataRequestHandler needs to be started BEFORE the gr โ
โ โฑ 60 โ โ self._data_request_handler = DataRequestHandler( โ
โ 61 โ โ โ self.args, self.logger, self.metrics_registry โ
โ 62 โ โ ) โ
โ 63 โ โ await self._async_setup_grpc_server() โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in __init__ โ
โ โ
โ 41 โ โ self.args.parallel = self.args.shards โ
โ 42 โ โ self.logger = logger โ
โ 43 โ โ self._is_closed = False โ
โ โฑ 44 โ โ self._load_executor(metrics_registry) โ
โ 45 โ โ self._init_monitoring(metrics_registry) โ
โ 46 โ โ
โ 47 โ def _init_monitoring(self, metrics_registry: Optional['Collect โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in _load_executor โ
โ โ
โ 79 โ โ :param metrics_registry: Optional prometheus metrics regis โ
โ passed to the executor so that it can expose metrics โ
โ 80 โ โ """ โ
โ 81 โ โ try: โ
โ โฑ 82 โ โ โ self._executor: BaseExecutor = BaseExecutor.load_confi โ
โ 83 โ โ โ โ self.args.uses, โ
โ 84 โ โ โ โ uses_with=self.args.uses_with, โ
โ 85 โ โ โ โ uses_metas=self.args.uses_metas, โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/jaml/__init__.โฆ โ
โ in load_config โ
โ โ
โ 727 โ โ โ โ
โ 728 โ โ โ if allow_py_modules: โ
โ 729 โ โ โ โ _extra_search_paths = extra_search_paths or [] โ
โ โฑ 730 โ โ โ โ load_py_modules( โ
โ 731 โ โ โ โ โ no_tag_yml, โ
โ 732 โ โ โ โ โ extra_search_paths=(_extra_search_paths + [os. โ
โ 733 โ โ โ โ โ if s_path โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/jaml/helper.pyโฆ โ
โ in load_py_modules โ
โ โ
โ 271 โ _finditem(d) โ
โ 272 โ if mod: โ
โ 273 โ โ mod = [complete_path(m, extra_search_paths, raise_nonexist โ
โ โฑ 274 โ โ PathImporter.add_modules(*mod) โ
โ 275 โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/importer.py:159 โ
โ in add_modules โ
โ โ
โ 156 โ โ from jina.jaml.helper import complete_path โ
โ 157 โ โ โ
โ 158 โ โ for m in not_python_module_paths: โ
โ โฑ 159 โ โ โ _path_import(complete_path(m)) โ
โ 160 โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/importer.py:130 โ
โ in _path_import โ
โ โ
โ 127 โ โ sys.modules[spec_name] = module โ
โ 128 โ โ spec.loader.exec_module(module) โ
โ 129 โ except Exception as ex: โ
โ โฑ 130 โ โ raise ImportError(f'can not import module from {absolute_p โ
โ 131 โ
โ 132 โ
โ 133 class PathImporter: โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
ImportError: can not import module from
/home/paperspace/dalle/dalle-flow/executors/stable/executor.py
DEBUG stable/rep-0@3066 process terminated
ERROR upscaler/rep-0@3067 ImportError('can not import module from [08/26/22 19:12:35]
/home/paperspace/dalle/dalle-flow/executors/swinir/executor.py') during
<class 'jina.serve.runtimes.worker.WorkerRuntime'> initialization
add "--quiet-error" to suppress the exception details
โญโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโฎ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/importer.py:128 โ
โ in _path_import โ
โ โ
โ 125 โ โ spec = importlib.util.spec_from_file_location(spec_name, a โ
โ 126 โ โ module = importlib.util.module_from_spec(spec) โ
โ 127 โ โ sys.modules[spec_name] = module โ
โ โฑ 128 โ โ spec.loader.exec_module(module) โ
โ 129 โ except Exception as ex: โ
โ 130 โ โ raise ImportError(f'can not import module from {absolute_p โ
โ 131 โ
โ <frozen importlib._bootstrap_external>:848 in exec_module โ
โ <frozen importlib._bootstrap>:219 in _call_with_frames_removed โ
โ โ
โ /home/paperspace/dalle/dalle-flow/executors/swinir/executor.py:8 in โ
โ <module> โ
โ โ
โ 5 โ
โ 6 from jina import Executor, DocumentArray, Document, requests โ
โ 7 โ
โ โฑ 8 from dalle_flow_swin_ir.main_test_swinir import main as swin_ir_mai โ
โ 9 โ
โ 10 class SwinIRUpscaler(Executor): โ
โ 11 โ def __init__(self, swinir_path: str, **kwargs): โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
ModuleNotFoundError: No module named 'dalle_flow_swin_ir'
The above exception was the direct cause of the following exception:
โญโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโฎ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/orchestrate/poโฆ โ
โ in run โ
โ โ
โ 71 โ โ
โ 72 โ try: โ
โ 73 โ โ _set_envs() โ
โ โฑ 74 โ โ runtime = runtime_cls( โ
โ 75 โ โ โ args=args, โ
โ 76 โ โ ) โ
โ 77 โ except Exception as ex: โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in __init__ โ
โ โ
โ 28 โ โ :param kwargs: keyword args โ
โ 29 โ โ """ โ
โ 30 โ โ self._health_servicer = health.HealthServicer(experimental โ
โ โฑ 31 โ โ super().__init__(args, **kwargs) โ
โ 32 โ โ
โ 33 โ async def async_setup(self): โ
โ 34 โ โ """ โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in __init__ โ
โ โ
โ 66 โ โ โ
โ 67 โ โ self._setup_monitoring() โ
โ 68 โ โ send_telemetry_event(event='start', obj=self) โ
โ โฑ 69 โ โ self._loop.run_until_complete(self.async_setup()) โ
โ 70 โ โ
โ 71 โ def run_forever(self): โ
โ 72 โ โ """ โ
โ โ
โ /usr/lib/python3.8/asyncio/base_events.py:616 in run_until_complete โ
โ โ
โ 613 โ โ if not future.done(): โ
โ 614 โ โ โ raise RuntimeError('Event loop stopped before Future โ
โ 615 โ โ โ
โ โฑ 616 โ โ return future.result() โ
โ 617 โ โ
โ 618 โ def stop(self): โ
โ 619 โ โ """Stop running the event loop. โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in async_setup โ
โ โ
โ 57 โ โ # Keep this initialization order โ
โ 58 โ โ # otherwise readiness check is not valid โ
โ 59 โ โ # The DataRequestHandler needs to be started BEFORE the gr โ
โ โฑ 60 โ โ self._data_request_handler = DataRequestHandler( โ
โ 61 โ โ โ self.args, self.logger, self.metrics_registry โ
โ 62 โ โ ) โ
โ 63 โ โ await self._async_setup_grpc_server() โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in __init__ โ
โ โ
โ 41 โ โ self.args.parallel = self.args.shards โ
โ 42 โ โ self.logger = logger โ
โ 43 โ โ self._is_closed = False โ
โ โฑ 44 โ โ self._load_executor(metrics_registry) โ
โ 45 โ โ self._init_monitoring(metrics_registry) โ
โ 46 โ โ
โ 47 โ def _init_monitoring(self, metrics_registry: Optional['Collect โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/serve/runtimesโฆ โ
โ in _load_executor โ
โ โ
โ 79 โ โ :param metrics_registry: Optional prometheus metrics regis โ
โ passed to the executor so that it can expose metrics โ
โ 80 โ โ """ โ
โ 81 โ โ try: โ
โ โฑ 82 โ โ โ self._executor: BaseExecutor = BaseExecutor.load_confi โ
โ 83 โ โ โ โ self.args.uses, โ
โ 84 โ โ โ โ uses_with=self.args.uses_with, โ
โ 85 โ โ โ โ uses_metas=self.args.uses_metas, โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/jaml/__init__.โฆ โ
โ in load_config โ
โ โ
โ 727 โ โ โ โ
โ 728 โ โ โ if allow_py_modules: โ
โ 729 โ โ โ โ _extra_search_paths = extra_search_paths or [] โ
โ โฑ 730 โ โ โ โ load_py_modules( โ
โ 731 โ โ โ โ โ no_tag_yml, โ
โ 732 โ โ โ โ โ extra_search_paths=(_extra_search_paths + [os. โ
โ 733 โ โ โ โ โ if s_path โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/jaml/helper.pyโฆ โ
โ in load_py_modules โ
โ โ
โ 271 โ _finditem(d) โ
โ 272 โ if mod: โ
โ 273 โ โ mod = [complete_path(m, extra_search_paths, raise_nonexist โ
โ โฑ 274 โ โ PathImporter.add_modules(*mod) โ
โ 275 โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/importer.py:159 โ
โ in add_modules โ
โ โ
โ 156 โ โ from jina.jaml.helper import complete_path โ
โ 157 โ โ โ
โ 158 โ โ for m in not_python_module_paths: โ
โ โฑ 159 โ โ โ _path_import(complete_path(m)) โ
โ 160 โ
โ โ
โ /home/paperspace/.local/lib/python3.8/site-packages/jina/importer.py:130 โ
โ in _path_import โ
โ โ
โ 127 โ โ sys.modules[spec_name] = module โ
โ 128 โ โ spec.loader.exec_module(module) โ
โ 129 โ except Exception as ex: โ
โ โฑ 130 โ โ raise ImportError(f'can not import module from {absolute_p โ
โ 131 โ
โ 132 โ
โ 133 class PathImporter: โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
ImportError: can not import module from
/home/paperspace/dalle/dalle-flow/executors/swinir/executor.py
DEBUG upscaler/rep-0@3067 process terminated
DEBUG stable/rep-0@3057 waiting for ready or shutdown signal from runtime [08/26/22 19:12:35]
DEBUG upscaler/rep-0@3057 waiting for ready or shutdown signal from runtime [08/26/22 19:12:35]
DEBUG upscaler/rep-0@3057 shutdown is is already set. Runtime will end gracefully
on its own
DEBUG stable/rep-0@3057 shutdown is is already set. Runtime will end gracefully on
its own
DEBUG upscaler/rep-0@3057 terminating the runtime process
DEBUG stable/rep-0@3057 terminating the runtime process
DEBUG upscaler/rep-0@3057 runtime process properly terminated
DEBUG stable/rep-0@3057 runtime process properly terminated
DEBUG upscaler/rep-0@3057 terminated
DEBUG stable/rep-0@3057 terminated
DEBUG upscaler/rep-0@3057 joining the process
DEBUG stable/rep-0@3057 joining the process
DEBUG upscaler/rep-0@3057 successfully joined the process
DEBUG stable/rep-0@3057 successfully joined the process
ERROR Flow@3057 Flow is aborted due to ['stable', 'upscaler'] can not be started. [08/26/22 19:12:39]
DEBUG gateway/rep-0@3057 waiting for ready or shutdown signal from runtime [08/26/22 19:12:39]
DEBUG gateway/rep-0@3057 terminate
DEBUG gateway/rep-0@3057 terminating the runtime process
DEBUG gateway/rep-0@3057 runtime process properly terminated
โ น Waiting summary... โโโโโโโโโโโโโโโโโบโโโโโโโโโโโโโโโโโโโโโโโ 2/5 0:00:04DEBUG gateway/rep-0@3069 process terminated [08/26/22 19:12:39]
DEBUG gateway/rep-0@3057 terminated
DEBUG gateway/rep-0@3057 joining the process
DEBUG gateway/rep-0@3057 successfully joined the process
DEBUG store/rep-0@3057 waiting for ready or shutdown signal from runtime [08/26/22 19:12:39]
DEBUG store/rep-0@3057 terminate
DEBUG store/rep-0@3057 terminating the runtime process
โ น Waiting summary... โโโโโโโโโโโโโโโโโบโโโโโโโโโโโโโโโโโโโโโโโ 2/5 0:00:04DEBUG store/rep-0@3068 cancel WorkerRuntime [08/26/22 19:12:39]
DEBUG store/rep-0@3057 runtime process properly terminated
โ น Waiting summary... โโโโโโโโโโโโโโโโโบโโโโโโโโโโโโโโโโโโโโโโโ 2/5 0:00:04DEBUG store/rep-0@3068 stopped GRPC Server
DEBUG store/rep-0@3068 cancel WorkerRuntime
DEBUG store/rep-0@3068 stopped GRPC Server
DEBUG store/rep-0@3068 process terminated [08/26/22 19:12:39]
DEBUG store/rep-0@3057 terminated
DEBUG store/rep-0@3057 joining the process
DEBUG store/rep-0@3057 successfully joined the process
DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses (raised from /usr/lib/python3/dist-packages/apport/report.py:13)
Traceback (most recent call last):
File "/home/paperspace/.local/bin/jina", line 8, in <module>
sys.exit(main())
File "/home/paperspace/.local/lib/python3.8/site-packages/jina_cli/__init__.py", line 143, in main
getattr(api, args.cli.replace('-', '_'))(args)
File "/home/paperspace/.local/lib/python3.8/site-packages/jina_cli/api.py", line 173, in flow
with f:
File "/home/paperspace/.local/lib/python3.8/site-packages/jina/orchestrate/flow/base.py", line 1405, in __enter__
return self.start()
File "/home/paperspace/.local/lib/python3.8/site-packages/jina/orchestrate/flow/builder.py", line 33, in arg_wrapper
return func(self, *args, **kwargs)
File "/home/paperspace/.local/lib/python3.8/site-packages/jina/orchestrate/flow/base.py", line 1461, in start
self._wait_until_all_ready()
File "/home/paperspace/.local/lib/python3.8/site-packages/jina/orchestrate/flow/base.py", line 1560, in _wait_until_all_ready
raise RuntimeFailToStart
jina.excepts.RuntimeFailToStartAre you in the virtual environment where you installed everything? It must be loaded with source env/bin/activate to use all the programs you installed into it.
cd ~/dalle/dalle-flow
source env/bin/activate
python flow_parser.py --enable-stable-diffusion --disable-dalle-mega --disable-glid3xl
jina flow --uses flow.tmp.ymlI do.
Don't know if it can help.
(env) paperspace@pspkgwgm9:~/dalle/dalle-flow/env/bin$ ls
accelerate convert-onnx-to-caffe2 markdown_py pyrsa-sign torchrun
accelerate-config deactivate.nu normalizer pyrsa-verify tqdm
accelerate-launch f2py pip python wandb
activate f2py3 pip3 python3 wb
activate.csh f2py3.8 pip-3.8 python3.8 wheel
activate.fish ftfy pip3.8 shortuuid wheel3
activate.nu google-oauthlib-tool pygrun skivi wheel-3.8
activate.ps1 imageio_download_bin pyrsa-decrypt tensorboard wheel3.8
activate_this.py imageio_remove_bin pyrsa-encrypt tiff2fsspec
chardetect jsonschema pyrsa-keygen tiffcomment
convert-caffe2-to-onnx lsm2bin pyrsa-priv2pub tifffile(env) paperspace@pspkgwgm9:~/dalle/dalle-flow$ cat flow.tmp.yml
jtype: Flow
with:
env:
JINA_LOG_LEVEL: debug
monitoring: true
port: 51005
protocol: grpc
executors:
- external: true
host: demo-cas.jina.ai
name: clip_encoder
needs:
- gateway
port: 2096
tls: true
uses: jinahub+docker://CLIPTorchEncoder/latest-gpu
- env:
CUDA_VISIBLE_DEVICES: 0
XLA_PYTHON_CLIENT_ALLOCATOR: platform
name: stable
replicas: 1
timeout_ready: -1
uses: executors/stable/config.yml
- external: true
host: demo-cas.jina.ai
name: rerank
needs:
- stable
port: 2096
tls: true
uses: jinahub+docker://CLIPTorchEncoder/latest-gpu
uses_requests:
/: rank
- env:
CUDA_VISIBLE_DEVICES: 0
name: upscaler
uses: executors/swinir/config.yml
- floating: true
name: store
uses: executors/store/config.ymlflow.tmp.yml Looks good. Can you cat ~/dalle for me? Jina crashed because it couldn't find the SwinIR or k-diffusion installations, it's like several of the packages weren't installed into the virtual environment.
AmericanPresidentJimmyCarter
commented
2 years ago Sorry lol, it's been a long day. ls ~/dalle, which will list the directories.
mariepop13
commented
2 years ago lol, thanks to make me feel more guilty
(env) paperspace@pspkgwgm9:~/dalle/dalle-flow$ ls ~/dalle
dalle-flow glid-3-xl latent-diffusion stable-diffusion SwinIRNo, it's not you!
OK, you have all the right folders. Run these commands while inside the virtualenv (terminal prefixed with (env)):
cd ~/dalle/SwinIR
pip install -e .
cd ~/dalle/stable-diffusion
pip install -e .
pip install https://github.com/crowsonkb/k-diffusion/archive/master.zipThen run jina again with
cd ~/dalle/dalle-flow
python flow_parser.py --enable-stable-diffusion --disable-dalle-mega --disable-glid3xl
jina flow --uses flow.tmp.ymlDid that work?
mariepop13
commented
2 years ago Spoiler alert: no...
AmericanPresidentJimmyCarter
commented
2 years ago At this point I am unsure what is the issue. However, would you like to try installing dalle-flow through docker? I have updated the README in the bot repo:
If you follow that you can just run a docker image instead of having to install dalle-flow manually.
mariepop13
commented
2 years ago Ya sure thanks again for your time, I will update you with the results if you want.
AmericanPresidentJimmyCarter
commented
2 years ago Yes, it's always good to know if something works or does not. Unfortunately I am not sure why installing jina/dalle-flow normally is failing.
ThrownLemon
commented
2 years ago @mariepop13 I ran into similer issues on ubuntu 22.04
In my case it was about this line: https://github.com/jina-ai/dalle-flow/blob/d38e899b47694fb786b118b69408ce0db8ca0b8d/executors/swinir/executor.py#L8
As a workaround, swaping executor.py for the below version seemed to fix the errors for me, maybe worth a shot? https://github.com/AmericanPresidentJimmyCarter/dalle-flow/blob/main/executors/swinir/executor.py
You may also have to delete your SwinIR dir and clone this version instead: https://github.com/JingyunLiang/SwinIR
Also in your error log, its complaining about not finding module k_diffusion maybe try:
pip install git+https://github.com/crowsonkb/k-diffusion
JoanFM
commented
1 year ago Hey @mariepop13 ,
is this problem solved?
Here what I have.
I did what proposed In #1โฆ nothing changeโฆ
after pip install --upgrade jaxlib
Thanks for your time.