search

ClankPan / ic-vectune

3 stars 0 forks source link

How to use upload command #1

Open anyon17 opened 3 weeks ago

anyon17 commented 3 weeks ago

Much thanks for this amazing implementation! I wanted to use the CLI and wondering how can I use upload command, for example given the dataset inside query_set, what can be the graph_meta_data ?

ClankPan commented 3 weeks ago

Hi, thank you for creating the issue!

The CLI tool in the main branch is currently not intended for general use, as it was originally written for my demo app. I am now updating it for practical use cases and it will likely be ready by next week. If you're interested, please check out the develop/kinic_db_instance branch.

anyon17 commented 3 weeks ago

Thanks @ClankPan for the quick response. I checked the mentioned branch. I believe only graph meta data is missing on the main branch to make upload command work if I am not wrong. Can you provide the json for that here itself and I can give it a run on my end?

ClankPan commented 3 weeks ago

In addition to graph_metadata, you also need source_data, both of which are built using ssd-vectune

anyon17 commented 3 weeks ago

ohh yes. So can you just add the command for upload as well with both these data in the README so that I can try to upload and test out the search command.

ClankPan commented 3 weeks ago

Yeah, I'm working for that. Could you share your use case for reference?

anyon17 commented 3 weeks ago

ohh okay great, looking forward for the completing. I am interested in modifying the ic-vectune to wire it in my own vector db server (that too powered by Vectune). I am able to port the server part by going through your vectune smart contract, just the last bits of the CLI is remaining, namely the source_data and graph_meta_data. For storage, I have my custom persistence layer.

anyon17 commented 3 weeks ago

Hey @ClankPan just a question, how did you uploaded the data for your case ? so that the command you have mentioned for search works ? Because my whole porting is almost done with just these two data i.e. graph_metadata and source_data_path remains. I will surely wait for your work to complete. Thanks again for the amazing work!

ClankPan commented 3 weeks ago

Will you deploy it to InternetComputer? or is it for another platform?

anyon17 commented 3 weeks ago

This is another platform

ClankPan commented 3 weeks ago

OK. If you need them ASAP, I recommend checking out ssd-vectune. Unfortunately, it is a bit unorganized at the moment, but you can take a look at the build code of them.

If you have any questions, feel free to ask me because there are no comments in the code.

https://github.com/ClankPan/ssd-vectune/blob/0c5ae4614a639f7a83765ca08916036a5f5a357c/src/main.rs#L554-L564

anyon17 commented 3 weeks ago

Hey much thanks @ClankPan for the quick revert and the help . Will check that out!

ClankPan commented 3 weeks ago

I'm going to bed now so I expect to replay after 8 hours.