search

gnes-ai / gnes

GNES is Generic Neural Elastic Search, a cloud-native semantic search system based on deep neural network.
https://gnes.ai
Other
1.27k stars 209 forks source link

🏗️GNES is under construction and rapid release cycles #86

Open hanxiao opened 5 years ago

hanxiao commented 5 years ago

🙇Hello there, I'd like to thank you for your interests in GNES, greatly appreciated!

After reading through the README, you may realize that GNES has an ambitious goal and the workload to enable any-to-any semantic search is quite something. Right now we are rapidly iterating GNES in terms of its usability, stability. This includes:

⏰A new release is scheduled every Friday's evening (GMT+8), the release procedure can be found here.

Please be patient if you encounter some issues, and please understand that we may not respond in time regarding your issues. Thanks for your understanding. 🙏

Meanwhile, if you'd like to understand the design principles behind GNES, feel free to read this blog post: Generic Neural Elastic Search: From bert-as-service and Go Way Beyond

For other questions, please contact our team via team@gnes.ai

hanxiao commented 5 years ago

Thanks all ❤️ for trying GNES and giving feedback at the early stage. I'd like to introduce you the new GNES Flow API (available since v0.0.46), enables a pythonic and intuitive way for building workflow in GNES. As an example, an indexing workflow can be simply defined as:

flow = (Flow(check_version=False, ctrl_with_ipc=True)
        .add_preprocessor(name='prep', yaml_path='yaml/prep.yml', replicas=3)
        .add_encoder(yaml_path='yaml/incep.yml', replicas=6)
        .add_indexer(name='vec_idx', yaml_path='yaml/vec.yml')
        .add_indexer(name='doc_idx', yaml_path='yaml/doc.yml', recv_from='prep')
        .add_router(name='sync', yaml_path='BaseReduceRouter', num_part=2, recv_from=['vec_idx', 'doc_idx']))

# then use it for indexing
with flow(backend='process') as fl:
    fl.index(bytes_gen=read_flowers(), batch_size=64)

🔰 You can find some resources here to help you getting started quickly: