search

PacktPublishing / Neural-Search-From-Prototype-to-Production-with-Jina

Neural Search - From Prototype to Production with Jina, published by Packt
MIT License
10 stars 3 forks source link

wikipedia: Flow@12199[E]:Flow is aborted due to ['indexer'] can not be started. #25

Open alexcg1 opened 2 years ago

alexcg1 commented 2 years ago

On branch fix-protobuf-version, running python app.py -t flow causes this error:

            app@12199[I]:starting Flow
storage_encoder/head@12199[D]:Pulling from jinahub/k3bptin5
Pull complete ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.4/31.4 MB   100%
Pull complete ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB     100%
Pull complete ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.4/31.4 MB   100%
Pull complete ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB     100%
storage_encoder/head@12674[D]:Start listening on 0.0.0.0:59944
storage_encoder/rep-0@12708[D]:will use Docker image: jinahub/k3bptin5:622b1970e9404501a157b958
⠙ Unpacking HNSWPostgresIndexer ...Collecting psycopg2-binary>=2.8.6
⠴ Unpacking HNSWPostgresIndexer ...  Downloading psycopg2_binary-2.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 854.7 kB/s eta 0:00:00
⠋ Unpacking HNSWPostgresIndexer ...Collecting pytest-mock
⠙ Unpacking HNSWPostgresIndexer ...  Downloading pytest_mock-3.8.2-py3-none-any.whl (9.1 kB)
⠹ Unpacking HNSWPostgresIndexer ...Collecting hnswlib>=0.6.0
⠸ Unpacking HNSWPostgresIndexer ...  Downloading hnswlib-0.6.2.tar.gz (31 kB)
⠙ Unpacking HNSWPostgresIndexer ..done
⠦ Unpacking HNSWPostgresIndexer ..done
⠋ Unpacking HNSWPostgresIndexer ..done
⠙ Unpacking HNSWPostgresIndexer ...Collecting bidict
  Downloading bidict-0.22.0-py3-none-any.whl (36 kB)
⠸ Unpacking HNSWPostgresIndexer ...Collecting pre-commit
⠴ Unpacking HNSWPostgresIndexer ...  Downloading pre_commit-2.20.0-py2.py3-none-any.whl (199 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.5/199.5 kB 454.3 kB/s eta 0:00:00
⠙ Unpacking HNSWPostgresIndexer ...Collecting backports.zoneinfo
⠹ Unpacking HNSWPostgresIndexer ...  Downloading backports.zoneinfo-0.2.1.tar.gz (74 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.1/74.1 kB 530.4 kB/s eta 0:00:00
⠦ Unpacking HNSWPostgresIndexer ..done
⠋ Unpacking HNSWPostgresIndexer ..done
⠸ Unpacking HNSWPostgresIndexer ..done
Requirement already satisfied: tqdm in ./env/lib/python3.9/site-packages (4.64.1)
⠦ Unpacking HNSWPostgresIndexer ...Collecting pytest>=5.0
  Downloading pytest-7.1.3-py3-none-any.whl (298 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 298.2/298.2 kB 2.7 MB/s eta 0:00:00
Requirement already satisfied: numpy in ./env/lib/python3.9/site-packages (from hnswlib>=0.6.0) (1.23.3)
⠏ Unpacking HNSWPostgresIndexer ...Collecting toml
  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
⠋ Unpacking HNSWPostgresIndexer ...Collecting cfgv>=2.0.0
  Downloading cfgv-3.3.1-py2.py3-none-any.whl (7.3 kB)
⠙ Unpacking HNSWPostgresIndexer ...Collecting nodeenv>=0.11.1
⠹ Unpacking HNSWPostgresIndexer ...  Downloading nodeenv-1.7.0-py2.py3-none-any.whl (21 kB)
⠸ Unpacking HNSWPostgresIndexer ...Collecting identify>=1.0.0
  Downloading identify-2.5.5-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 kB 310.6 kB/s eta 0:00:00
⠏ Unpacking HNSWPostgresIndexer ...Collecting virtualenv>=20.0.8
  Downloading virtualenv-20.16.5-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 1.9 MB/s eta 0:00:00
Requirement already satisfied: pyyaml>=5.1 in ./env/lib/python3.9/site-packages (from pre-commit) (6.0)
Requirement already satisfied: setuptools in ./env/lib/python3.9/site-packages (from nodeenv>=0.11.1->pre-commit) (65.3.0)
⠋ Unpacking HNSWPostgresIndexer ...Requirement already satisfied: packaging in ./env/lib/python3.9/site-packages (from pytest>=5.0->pytest-mock) (21.3)
Collecting iniconfig
  Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
⠙ Unpacking HNSWPostgresIndexer ...Collecting py>=1.8.2
⠹ Unpacking HNSWPostgresIndexer ...  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 kB 2.8 MB/s eta 0:00:00
⠸ Unpacking HNSWPostgresIndexer ...Collecting pluggy<2.0,>=0.12
⠼ Unpacking HNSWPostgresIndexer ...  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: attrs>=19.2.0 in ./env/lib/python3.9/site-packages (from pytest>=5.0->pytest-mock) (22.1.0)
Collecting tomli>=1.0.0
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
⠴ Unpacking HNSWPostgresIndexer ...Collecting distlib<1,>=0.3.5
  Downloading distlib-0.3.6-py2.py3-none-any.whl (468 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.5/468.5 kB 1.5 MB/s eta 0:00:00
Requirement already satisfied: filelock<4,>=3.4.1 in ./env/lib/python3.9/site-packages (from virtualenv>=20.0.8->pre-commit) (3.8.0)
⠋ Unpacking HNSWPostgresIndexer ...Collecting platformdirs<3,>=2.4
  Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB)
⠙ Unpacking HNSWPostgresIndexer ...Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./env/lib/python3.9/site-packages (from packaging->pytest>=5.0->pytest-mock) (3.0.9)
⠹ Unpacking HNSWPostgresIndexer ...Building wheels for collected packages: hnswlib, backports.zoneinfo
⠧ Unpacking HNSWPostgresIndexer ..done
  Created wheel for hnswlib: filename=hnswlib-0.6.2-cp39-cp39-linux_x86_64.whl size=2062155 sha256=880462de6f6d0885a17d5af5a6aedcec47ff945f98c03873c9cafb0713252992
  Stored in directory: /home/alexcg/.cache/pip/wheels/42/ef/fe/bab7e480effd9f1bb90c3dfc354ac1bceb27eb0f7e0877efdc
⠸ Unpacking HNSWPostgresIndexer ..done
  Created wheel for backports.zoneinfo: filename=backports.zoneinfo-0.2.1-cp39-cp39-linux_x86_64.whl size=76284 sha256=41311d821d94c657b618ee78ddc674d7393e773c31f8949083feafef517f4016
  Stored in directory: /home/alexcg/.cache/pip/wheels/f6/da/c4/7ac0330c0d9763cbd549898f8815e8cbc01a5bd31a009be764
Successfully built hnswlib backports.zoneinfo
⠙ Unpacking HNSWPostgresIndexer ...Installing collected packages: iniconfig, distlib, tomli, toml, py, psycopg2-binary, pluggy, platformdirs, nodeenv, identify, hnswlib, cfgv, bidict, backports.zoneinfo, virtualenv, pytest, pytest-mock, pre-commit
⠦ Unpacking HNSWPostgresIndexer ...Successfully installed backports.zoneinfo-0.2.1 bidict-0.22.0 cfgv-3.3.1 distlib-0.3.6 hnswlib-0.6.2 identify-2.5.5 iniconfig-1.1.1 nodeenv-1.7.0 platformdirs-2.5.2 pluggy-1.0.0 pre-commit-2.20.0 psycopg2-binary-2.9.3 py-1.11.0 pytest-7.1.3 pytest-mock-3.8.2 toml-0.10.2 tomli-2.0.1 virtualenv-20.16.5
   indexer/head@13315[D]:Start listening on 0.0.0.0:62355
storage_encoder/head@12199[D]:ready and listening
   indexer/head@12199[D]:ready and listening
⠋ 0/3 waiting storage_encoder indexer gateway to be ready...  indexer/rep-0@13322[D]:Start listening on 0.0.0.0:52715
  gateway/rep-0@13323[D]:Adding connection for deployment storage_encoder/heads/0 to http://0.0.0.0:59944
  gateway/rep-0@13323[D]:Adding connection for deployment indexer/heads/0 to http://0.0.0.0:62355
  indexer/rep-0@13322[C]:can not load the executor from /home/alexcg/.jina/hub-package/dvp0845a/config.yml
  indexer/rep-0@13322[E]:ExecutorFailToLoad() during <class 'jina.serve.runtimes.worker.WorkerRuntime'> initialization
 add "--quiet-error" to suppress the exception details
Traceback (most recent call last):
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/serve/runtimes/request_handlers/data_request_handler.py", line 35, in _load_executor
    self._executor: BaseExecutor = BaseExecutor.load_config(
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/jaml/__init__.py", line 726, in load_config
    return JAML.load(tag_yml, substitute=False)
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/jaml/__init__.py", line 166, in load
    r = yaml.load(stream, Loader=JinaLoader)
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/yaml/constructor.py", line 55, in construct_document
    data = self.construct_object(node)
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/jaml/__init__.py", line 565, in _from_yaml
    return get_parser(cls, version=data.get('version', None)).parse(cls, data)
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/jaml/parsers/executor/legacy.py", line 69, in parse
    obj = cls(
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/serve/executors/decorators.py", line 60, in arg_wrapper
    f = func(self, *args, **kwargs)
  File "/home/alexcg/.jina/hub-package/dvp0845a/executor/hnswpsql.py", line 130, in __init__
    ) = self._init_executors(self._init_kwargs)
  File "/home/alexcg/.jina/hub-package/dvp0845a/executor/hnswpsql.py", line 229, in _init_executors
    kv_indexer = PostgreSQLStorage(dump_dtype=np.float32, **_init_kwargs)
  File "/home/alexcg/.jina/hub-package/dvp0845a/executor/postgres_indexer.py", line 75, in __init__
    self.handler = PostgreSQLHandler(
  File "/home/alexcg/.jina/hub-package/dvp0845a/executor/postgreshandler.py", line 74, in __init__
    self.postgreSQL_pool = psycopg2.pool.SimpleConnectionPool(
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/psycopg2/pool.py", line 59, in __init__
    self._connect()
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/psycopg2/pool.py", line 63, in _connect
    conn = psycopg2.connect(*self._args, **self._kwargs)
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/psycopg2/__init__.py", line 122, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: connection to server at "127.0.0.1", port 5432 failed: Connection refused
    Is the server running on that host and accepting TCP/IP connections?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/orchestrate/pods/__init__.py", line 81, in run
    runtime = runtime_cls(
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/serve/runtimes/worker/__init__.py", line 36, in __init__
    self._data_request_handler = DataRequestHandler(args, self.logger)
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/serve/runtimes/request_handlers/data_request_handler.py", line 30, in __init__
    self._load_executor()
  File "/mnt/data/work/repos/book/src/Chapter07/wikipedia-sentences-query-while-indexing/env/lib/python3.9/site-packages/jina/serve/runtimes/request_handlers/data_request_handler.py", line 61, in _load_executor
    raise ExecutorFailToLoad from ex
jina.excepts.ExecutorFailToLoad
  indexer/rep-0@13322[D]: Process terminated
⠙ 0/3 waiting storage_encoder indexer gateway to be ready...  indexer/rep-0@12199[D]:waiting for ready or shutdown signal from runtime
  indexer/rep-0@12199[D]:shutdown is is already set. Runtime will end gracefully on its own
  indexer/rep-0@12199[D]:terminated
   indexer/head@12199[D]:waiting for ready or shutdown signal from runtime
   indexer/head@12199[D]:terminate
   indexer/head@12199[D]:terminating the runtime process
   indexer/head@12199[D]: runtime process properly terminated
   indexer/head@13315[D]:Cancel HeadRuntime
   indexer/head@13315[D]:Cancel HeadRuntime
   indexer/head@13315[D]: Process terminated
   indexer/head@12199[D]:terminated
   indexer/head@12199[D]: Joining the process
   indexer/head@12199[D]: Successfully joined the process
  indexer/rep-0@12199[D]: Joining the process
  indexer/rep-0@12199[D]: Successfully joined the process
⠸ 1/3 waiting storage_encoder gateway to be ready...INFO:     Waiting for application startup.      
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:9000 (Press CTRL+C to quit)
⠼ 1/3 waiting storage_encoder gateway to be ready...  gateway/rep-0@12199[D]:ready and listening    
⠴ 2/3 waiting storage_encoder to be ready...storage_encoder/rep-0@12199[D]:ready and listening      
storage_encoder/head@12674[D]:recv ControlRequest f118e3cadc774b5583c002b246efcf3c ACTIVATE
storage_encoder/head@12674[D]:Adding connection for deployment worker/shards/0 to 0.0.0.0:63053
storage_encoder/rep-0@12708[I]:
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWWWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNNNNNNNWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMMMMMMMMMMMMMMMMMMNNNNNNNNNMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNNNNNNNNMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNNNWWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMWxxxxxxxxxOMMMMMNxxxxxxxxx0MMMMMKddddddxkKWMMMMMMMMMMMMXOxdddxONMMMM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMXllllllllldMMMMM0lllllllllxMMMMMOllllllllllo0MMMMMMMM0olllllllllo0MM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMXllllllllldMMMMM0lllllllllxMMMMMOlllllllllllloWMMMMMdllllllllllllldM
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMXllllllllldMMMMM0lllllllllxMMMMMOllllllllllllloMMMM0lllllllllllllllK
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMKllllllllldMMMMM0lllllllllxMMMMMOllllllllllllllKMMM0lllllllllllllllO
storage_encoder/rep-0@12708[I]:MMMMMMMMMMMMKllllllllldMMMMM0lllllllllxMMMMMOllllllllllllll0MMMMollllllllllllllO
storage_encoder/rep-0@12708[I]:MWOkkkkk0MMMKlllllllllkMMMMM0lllllllllxMMMMMOllllllllllllll0MMMMMxlllllllllllllO
storage_encoder/rep-0@12708[I]:NkkkkkkkkkMMKlllllllloMMMMMM0lllllllllxMMMMMOllllllllllllll0MMMMMMWOdolllllllllO
storage_encoder/rep-0@12708[I]:KkkkkkkkkkNMKllllllldMMMMMMMMWWWWWWWWWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:MOkkkkkkk0MMKllllldXMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:MMWX00KXMMMMXxk0XMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
storage_encoder/rep-0@12708[I]:
storage_encoder/rep-0@12708[I]:▶️  /usr/local/bin/jina executor --uses config.yml --name storage_encoder/rep-0 --workspace-id 0af776d4abe242958b4e0f042186673b --extra-search-paths  --polling ANY --port 63053 --native --upload-files --timeout-ready -1 --noblock-on-start
storage_encoder/rep-0@12708[I]:🔧️                            cli = executor                      
storage_encoder/rep-0@12708[I]:connection-list = None
storage_encoder/rep-0@12708[I]:daemon = False
storage_encoder/rep-0@12708[I]:disable-reduce = False
storage_encoder/rep-0@12708[I]:disable-remote = False
storage_encoder/rep-0@12708[I]:docker-kwargs = None
storage_encoder/rep-0@12708[I]:entrypoint = None
storage_encoder/rep-0@12708[I]:env = None
storage_encoder/rep-0@12708[I]:expose-public = False
storage_encoder/rep-0@12708[I]:🔧️             extra-search-paths = ['']                          
storage_encoder/rep-0@12708[I]:force-update = False
storage_encoder/rep-0@12708[I]:gpus = None
storage_encoder/rep-0@12708[I]:host = 0.0.0.0
storage_encoder/rep-0@12708[I]:host-in = 0.0.0.0
storage_encoder/rep-0@12708[I]:install-requirements = False
storage_encoder/rep-0@12708[I]:🔧️            k8s-connection-pool = True                          
storage_encoder/rep-0@12708[I]:k8s-namespace = None
storage_encoder/rep-0@12708[I]:log-config = /usr/local/lib/python3.7/site-
storage_encoder/rep-0@12708[I]:🔧️                           name = storage_encoder/rep-0         
storage_encoder/rep-0@12708[I]:🔧️                         native = True                          
storage_encoder/rep-0@12708[I]:🔧️               noblock-on-start = True                          
storage_encoder/rep-0@12708[I]:pod-role = WORKER
storage_encoder/rep-0@12708[I]:polling = ANY
storage_encoder/rep-0@12708[I]:🔧️                           port = 63053                         
storage_encoder/rep-0@12708[I]:port-jinad = 8000
storage_encoder/rep-0@12708[I]:pull-latest = False
storage_encoder/rep-0@12708[I]:py-modules = None
storage_encoder/rep-0@12708[I]:quiet = False
storage_encoder/rep-0@12708[I]:quiet-error = False
storage_encoder/rep-0@12708[I]:quiet-remote-logs = False
storage_encoder/rep-0@12708[I]:replica-id = 0
storage_encoder/rep-0@12708[I]:replicas = 1
storage_encoder/rep-0@12708[I]:runtime-backend = PROCESS
storage_encoder/rep-0@12708[I]:runtime-cls = WorkerRuntime
storage_encoder/rep-0@12708[I]:shard-id = 0
storage_encoder/rep-0@12708[I]:shards = 1
storage_encoder/rep-0@12708[I]:timeout-ctrl = 60
storage_encoder/rep-0@12708[I]:🔧️                  timeout-ready = -1                            
storage_encoder/rep-0@12708[I]:🔧️                   upload-files = []                            
storage_encoder/rep-0@12708[I]:🔧️                           uses = config.yml                    
storage_encoder/rep-0@12708[I]:uses-after-address = None
storage_encoder/rep-0@12708[I]:uses-before-address = None
storage_encoder/rep-0@12708[I]:uses-metas = None
storage_encoder/rep-0@12708[I]:uses-requests = None
storage_encoder/rep-0@12708[I]:uses-with = None
storage_encoder/rep-0@12708[I]:volumes = None
storage_encoder/rep-0@12708[I]:workspace = None
storage_encoder/rep-0@12708[I]:🔧️                   workspace-id = 0af776d4abe242958b4e0f04218667
storage_encoder/rep-0@12708[I]:
storage_encoder/rep-0@12708[I]:storage_encoder/rep-0@ 1[W]:You are using Jina version 3.1.1, however version 3.9.2 is available. You should consider upgrading via the "pip install --upgrade jina" command.
storage_encoder/rep-0@12708[I]:DeprecationWarning: `scipy.sparse.sparsetools` is deprecated!
storage_encoder/rep-0@12708[I]:scipy.sparse.sparsetools is a private module for scipy.sparse, and should not be used. (raised from /usr/local/lib/python3.7/site-packages/scipy/sparse/sparsetools.py:21)
storage_encoder/rep-0@12708[I]:2022-09-22 11:40:43,873 https://flair.informatik.hu-berlin.de/resources/embeddings/token/glove.gensim.vectors.npy not found in cache, downloading to /tmp/tmpprpnsne0
 36%|███▋      | 58112000/160000128 [00:27<00:42, 2401809.93B/s]<?, ?B/s]
 76%|███████▌  | 121603072/160000128 [00:53<00:27, 1385744.90B/s]
storage_encoder/rep-0@12708[I]:2022-09-22 11:41:51,201 copying /tmp/tmpprpnsne0 to cache at /root/.flair/embeddings/glove.gensim.vectors.npy
storage_encoder/rep-0@12708[I]:2022-09-22 11:41:51,367 removing temp file /tmp/tmpprpnsne0
storage_encoder/rep-0@12708[I]:2022-09-22 11:41:51,748 https://flair.informatik.hu-berlin.de/resources/embeddings/token/glove.gensim not found in cache, downloading to /tmp/tmpcx_yc_uz
100%|██████████| 160000128/160000128 [01:06<00:00, 2390839.88B/s]53<00:27, 1376401.85B/s]
storage_encoder/rep-0@12708[I]:2022-09-22 11:42:00,202 copying /tmp/tmpcx_yc_uz to cache at /root/.flair/embeddings/glove.gensim
storage_encoder/rep-0@12708[I]:2022-09-22 11:42:00,219 removing temp file /tmp/tmpcx_yc_uz
storage_encoder/rep-0@12708[I]:storage_encoder/rep-0@ 1[L]: Executor FlairTextEncoder started
           Flow@12199[E]:Flow is aborted due to ['indexer'] can not be started.                     
  gateway/rep-0@12199[D]:waiting for ready or shutdown signal from runtime
  gateway/rep-0@12199[D]:terminate
  gateway/rep-0@12199[D]:terminating the runtime process
  gateway/rep-0@12199[D]: runtime process properly terminated
INFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
  gateway/rep-0@13323[D]: Process terminated
  gateway/rep-0@12199[D]:terminated
  gateway/rep-0@12199[D]: Joining the process
  gateway/rep-0@12199[D]: Successfully joined the process
storage_encoder/rep-0@12199[D]:waiting for ready or shutdown signal from runtime
storage_encoder/rep-0@12199[D]:terminate
100%|██████████| 21494764/21494764 [00:08<00:00, 2603788.78B/s]<?, ?B/s]
storage_encoder/rep-0@12708[D]: Process terminated
storage_encoder/rep-0@12199[D]:terminating the runtime process
storage_encoder/rep-0@12199[D]: runtime process properly terminated
storage_encoder/rep-0@12199[D]:terminated
storage_encoder/head@12199[D]:waiting for ready or shutdown signal from runtime
storage_encoder/head@12199[D]:terminate
storage_encoder/head@12199[D]:terminating the runtime process
storage_encoder/head@12199[D]: runtime process properly terminated
storage_encoder/head@12674[D]:Cancel HeadRuntime
storage_encoder/head@12674[D]:Cancel HeadRuntime
storage_encoder/head@12674[D]: Process terminated
storage_encoder/head@12199[D]:terminated
storage_encoder/head@12199[D]: Joining the process
storage_encoder/head@12199[D]: Successfully joined the process
storage_encoder/rep-0@12199[D]: Joining the process
storage_encoder/rep-0@12199[D]: Successfully joined the process
           Flow@12199[D]:Flow is closed!
           Flow@12199[D]:Flow is closed!
            app@12199[I]:Caught . Shutting down...