search

jina-ai / examples

Jina examples and demos to help you get started
https://docs.jina.ai
Apache License 2.0
455 stars 142 forks source link

multires-lyrics-search exmaple errors #298

Closed yiouyou closed 3 years ago

yiouyou commented 3 years ago

1) In app.py

    if sys.argv[1] == 'index':
        workspace = os.environ['WORKDIR']
        if os.path.exists(workspace):
            print(f'\n +---------------------------------------------------------------------------------+ \
                    \n |                                   🤖🤖🤖                                        | \
                    \n | The directory {workspace} already exists. Please remove it before indexing again. | \
                    \n |                                   🤖🤖🤖                                        | \
                    \n +---------------------------------------------------------------------------------+')
        config()
        index()

shows error:

(py37-lyrics) sz@air:/mnt/d/coding/jina-ai/examples/multires-lyrics-search$ python3.7 app.py index
Traceback (most recent call last):
  File "app.py", line 70, in <module>
    if (os.environ['WORKDIR']):
  File "/usr/lib/python3.7/os.py", line 681, in __getitem__
    raise KeyError(key) from None
KeyError: 'WORKDIR'

I think the logic here is not quite right if there is no WORKDIR at the beginning.

2) By pass the 1) issue as

    if sys.argv[1] == 'index':
        config()
        index()

still get errors:

(py37-lyrics) sz@air:/mnt/d/coding/jina-ai/examples/multires-lyrics-search$ python3.7 app.py index
           Flow@5397[S]:successfully built Flow from a yaml config
        crafter@5417[I]:post initiating, this may take some time...
        crafter@5417[I]:post initiating, this may take some time takes 0 seconds (0.00s)
        crafter@5417[S]:successfully built Sentencizer from a yaml config
        crafter@5417[I]:setting up sockets...
        crafter@5417[I]:input tcp://0.0.0.0:54983 (SUB_CONNECT)          output tcp://0.0.0.0:47037 (PUSH_CONNECT)     control over tcp://0.0.0.0:47793 (PAIR_BIND)
        crafter@5417[S]:ready and listening
        encoder@5422[I]:post initiating, this may take some time...
        encoder@5422[I]:post initiating, this may take some time takes 0 seconds (0.00s)
        encoder@5422[S]:successfully built BaseExecutor from a yaml config
        encoder@5422[I]:setting up sockets...
        encoder@5422[I]:input tcp://0.0.0.0:47037 (PULL_BIND)    output tcp://0.0.0.0:58657 (ROUTER_BIND)      control over tcp://0.0.0.0:45535 (PAIR_BIND)
        encoder@5422[S]:ready and listening
        encoder@5427[I]:post initiating, this may take some time...
        encoder@5427[I]:post initiating, this may take some time takes 0 seconds (0.00s)
        encoder@5427[S]:successfully built BaseExecutor from a yaml config
        encoder@5427[I]:setting up sockets...
        encoder@5427[I]:input tcp://0.0.0.0:42049 (PULL_BIND)    output tcp://0.0.0.0:39111 (PUSH_CONNECT)     control over tcp://0.0.0.0:43847 (PAIR_BIND)
        encoder@5427[S]:ready and listening
        encoder@5432[I]:post initiating, this may take some time...
        encoder@5432[I]:post initiating, this may take some time takes 0 seconds (0.00s)
        encoder@5432[S]:successfully built TransformerTorchEncoder from a yaml config
        encoder@5432[I]:setting up sockets...
        encoder@5432[I]:input tcp://0.0.0.0:58657 (DEALER_CONNECT)       output tcp://0.0.0.0:42049 (PUSH_CONNECT)     control over tcp://0.0.0.0:59905 (PAIR_BIND)
        encoder@5432[S]:ready and listening
        encoder@5422[I]:received ControlRequest from encoder▸⚐
        encoder@5437[I]:post initiating, this may take some time...
        encoder@5437[I]:post initiating, this may take some time takes 0 seconds (0.00s)
        encoder@5437[S]:successfully built TransformerTorchEncoder from a yaml config
        encoder@5437[I]:setting up sockets...
        encoder@5437[I]:input tcp://0.0.0.0:58657 (DEALER_CONNECT)       output tcp://0.0.0.0:42049 (PUSH_CONNECT)     control over tcp://0.0.0.0:33847 (PAIR_BIND)
        encoder@5437[S]:ready and listening
        encoder@5422[I]:received ControlRequest from encoder▸⚐
      chunk_idx@5442[I]:post initiating, this may take some time...
      chunk_idx@5442[I]:post initiating, this may take some time takes 0 seconds (0.00s)
      chunk_idx@5442[S]:successfully built BaseExecutor from a yaml config
      chunk_idx@5442[I]:setting up sockets...
      chunk_idx@5442[I]:input tcp://0.0.0.0:39111 (PULL_BIND)    output tcp://0.0.0.0:33097 (ROUTER_BIND)      control over tcp://0.0.0.0:32853 (PAIR_BIND)
      chunk_idx@5442[S]:ready and listening
      chunk_idx@5447[I]:post initiating, this may take some time...
      chunk_idx@5447[I]:post initiating, this may take some time takes 0 seconds (0.00s)
      chunk_idx@5447[S]:successfully built BaseExecutor from a yaml config
      chunk_idx@5447[I]:setting up sockets...
      chunk_idx@5447[I]:input tcp://0.0.0.0:43389 (PULL_BIND)    output tcp://0.0.0.0:57583 (PUSH_CONNECT)     control over tcp://0.0.0.0:37655 (PAIR_BIND)
      chunk_idx@5447[S]:ready and listening
      chunk_idx@5452[I]:post initiating, this may take some time...
      chunk_idx@5452[I]:post initiating, this may take some time takes 0 seconds (0.00s)
      chunk_idx@5452[C]:unknown exception: ValueError('workspace=$JINA_WORKSPACE is not expandable or badly referred')
Traceback (most recent call last):
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/peapods/pea.py", line 348, in run
    self.loop_body()
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/peapods/pea.py", line 329, in loop_body
    self.load_executor()
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/peapods/pea.py", line 200, in load_executor
    self.args.separated_workspace, self.args.pea_id)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/executors/__init__.py", line 433, in load_config
    return yaml.load(tmp_s)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/main.py", line 343, in load
    return constructor.get_single_data()
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 113, in get_single_data
    return self.construct_document(node)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 118, in construct_document
    data = self.construct_object(node)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 146, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 181, in construct_non_recursive_object
    data = constructor(self, node)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/executors/compound.py", line 350, in from_yaml
    obj, data, from_dump = super()._get_instance_from_yaml(constructor, node)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/executors/__init__.py", line 479, in _get_instance_from_yaml
    constructor, node, deep=True)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 440, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 255, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 146, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 188, in construct_non_recursive_object
    for _dummy in generator:
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 717, in construct_yaml_seq
    data.extend(self.construct_sequence(node))
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 211, in construct_sequence
    return [self.construct_object(child, deep=deep) for child in node.value]
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 211, in <listcomp>
    return [self.construct_object(child, deep=deep) for child in node.value]
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 146, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 181, in construct_non_recursive_object
    data = constructor(self, node)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/executors/__init__.py", line 474, in from_yaml
    return cls._get_instance_from_yaml(constructor, node)[0]
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/executors/__init__.py", line 507, in _get_instance_from_yaml
    obj = cls(**data.get('with', {}), metas=data.get('metas', {}), requests=data.get('requests', {}))
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/executors/__init__.py", line 59, in __call__
    getattr(obj, '_post_init_wrapper', lambda *x: None)(m, r)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/executors/__init__.py", line 162, in _post_init_wrapper
    self._fill_metas(_metas)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/executors/__init__.py", line 224, in _fill_metas
    raise ValueError(f'{k}={v} is not expandable or badly referred')
ValueError: workspace=$JINA_WORKSPACE is not expandable or badly referred
      chunk_idx@5452[S]:terminated
      chunk_idx@5397[C]:fail to start <class 'jina.peapods.pea.BasePea'> with name chunk_idx-1, this often means the executor used in the pod is not valid
Traceback (most recent call last):
  File "app.py", line 79, in <module>
    index()
  File "app.py", line 44, in index
    with f:
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/flow/__init__.py", line 451, in __enter__
    return self.start()
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/flow/__init__.py", line 507, in start
    self.enter_context(v)
  File "/usr/lib/python3.7/contextlib.py", line 427, in enter_context
    result = _cm_type.__enter__(cm)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/peapods/pod.py", line 254, in __enter__
    return self.start()
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/peapods/pod.py", line 389, in start
    return super().start()
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/peapods/pod.py", line 224, in start
    self.enter_context(p)
  File "/usr/lib/python3.7/contextlib.py", line 427, in enter_context
    result = _cm_type.__enter__(cm)
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/peapods/pea.py", line 425, in __enter__
    return self.start()
  File "/home/sz/py37-lyrics/lib/python3.7/site-packages/jina/peapods/pea.py", line 418, in start
    raise PeaFailToStart
jina.excepts.PeaFailToStart

'workspace=$JINA_WORKSPACE is not expandable or badly referred' What does this mean?

maximilianwerk commented 3 years ago

Fixed here: https://github.com/jina-ai/examples/pull/299 Sorry, the error was introduced with a recent refactoring and harmonizing of our examples.