search

gnes-ai / gnes

GNES is Generic Neural Elastic Search, a cloud-native semantic search system based on deep neural network.
https://gnes.ai
Other
1.26k stars 210 forks source link

Stuck with on `create new stub`. On make `client query` #309

Closed ilham-bintang closed 4 years ago

ilham-bintang commented 4 years ago

Explanation

Hi. I has been implement the demo-poems-ir. And it is working properly. Right now I want to use the sample for my custom class. here the class:

class FaqClient(CLIClient):

    @property
    def bytes_generator(self):
        num_rows = 0

        for rr in csv.DictReader(self.args.txt_file, delimiter='\t'):
            yield json.dumps(rr['answer']).encode()
            num_rows += 1
            if num_rows > self.args.num_poems:
                return

    def query_callback(self, req, resp):
        print(colored(req.search.query, 'green'))
        for k in resp.search.topk_results:
            print(colored(k.doc.doc_id, 'magenta'))
            print(colored(k.doc.raw_text, 'yellow'))
            print(colored(k.score.value, 'blue'))
            pprint.pprint(json.loads(k.score.explained))
            input('press any key to continue...')

        if self.args.prompt:
            input('on to next query, press any key to continue...')

Problem

When I run make client_query or make client_index it will stuck in:

I:FaqClient:[bas:__i:136]:create new stub...

Here the screenshot Screen Shot 2019-10-07 at 23 36 31

Questions

What's exactly happened when the Client info create new stub...? I read the source it only init GnesRPCStub with the channel. It stuck there and can not passing the https://github.com/gnes-ai/gnes/blob/17f92874c21a22fa60d29da47e13bff58715ce95/gnes/client/base.py#L142

Kindly advise

hanxiao commented 4 years ago

this usually happens when https_proxy or http_proxy set. unset them and try again.

unset http_proxy && unset https_proxy

or in python

os.unsetenv('http_proxy')
os.unsetenv('https_proxy')
ilham-bintang commented 4 years ago

this usually happens when https_proxy or http_proxy set. unset them and try again.

unset http_proxy && unset https_proxy

or in python

os.unsetenv('http_proxy')
os.unsetenv('https_proxy')

Actually I has been unset the https_proxy and unset http_proxy before run the client image. Here my client-query.sh:

unset https_proxy && unset http_proxy && docker run -it --rm --network host -v ${PWD}/data:/data/ nullphantom/faq-gnes:client --mode query --txt_file /data/questions.csv --prompt
hanxiao commented 4 years ago

thanks again ❤️ for trying GNES and giving feedback at the early stage. I'd like to introduce you the new GNES Flow API (available since v0.0.46), enables a pythonic and intuitive way for building workflow in GNES. As an example, an indexing workflow can be simply defined as:

flow = (Flow(check_version=False, ctrl_with_ipc=True)
        .add_preprocessor(name='prep', yaml_path='yaml/prep.yml', replicas=3)
        .add_encoder(yaml_path='yaml/incep.yml', replicas=6)
        .add_indexer(name='vec_idx', yaml_path='yaml/vec.yml')
        .add_indexer(name='doc_idx', yaml_path='yaml/doc.yml', recv_from='prep')
        .add_router(name='sync', yaml_path='BaseReduceRouter', num_part=2, recv_from=['vec_idx', 'doc_idx']))

# then use it for indexing
with flow(backend='process') as fl:
    fl.index(bytes_gen=read_flowers(), batch_size=64)

🔰 You can find some resources here to help you getting started quickly:

ilham-bintang commented 4 years ago

Okay I will migrate my yml specs to pythonic gnes Flow new classes. Thank you gnes team!!

ilham-bintang commented 4 years ago

This issue raised because there is another process on grpc port: 8800 try to kill all process and rerun