feat: option to do indexing without a query

iynaix commented 9 months ago

Is there a way to run rclip without a query, just to update the index? Use case would be to have it run as a cron job so when a query needs to be run, there wouldn't be any waiting for indexing.

yurijmikhalevich commented 8 months ago

@iynaix, at the moment, there is no such option. I'll give it a closer look. You can always run rclip with a query and ignore the querying output. Since querying is quick, it shouldn't introduce any overhead.

Fethbita commented 1 month ago

Also related, I have around 300k images that needs to be indexed when I was using rclip, the underlying docker image had a segfault and rclip also crashed as a result of that. ~When I started indexing again, the already existing 58mb sqlite index was ignored and the indexing started over. It would be good if rclip could continue indexing from where it left off.~ rclip recognizes the sqlite database and continues off where it left off.

yurijmikhalevich commented 1 month ago

@Fethbita,

rclip recognizes the sqlite database and continues off where it left off.

Great to hear 🔥 yes, rclip should continue where it left off. It will still display the progress bar counting from 0, but should quickly catch up to where it left off after it verifies that none of the images it already indexed changed.

yurijmikhalevich commented 1 week ago

Marking this Issue for hacktoberfest.

One option can be to use nargs='?' on the query argument in argparse to let it be optional, and if it's not present, only index the dir without searching.

But I think it's not the best idea to let running rclip with no arguments at all start indexing since this is a heavier operation.

Another option is to:

add --index-only command-line argument that will put rclip into the index-only mode
error if the user uses --index-only and passes the query, e.g., rclip --index-only cat is not allowed, but rclip --index-only is allowed
error if the user uses --index-only with any arguments other than --indexing-batch-size, --exclude-dir, --device; only these three are allowed to be used in the --index-only mode
running rclip with no argument, rclip, should display a short help and an error, like it does right now
ideally, let's handle all of the above logic using argparse

Let me know if you have any thoughts on this.

YashBaviskar1 commented 2 days ago

Hey There, I can try to Solve this issue. Thanks

yurijmikhalevich commented 2 days ago

@YashBaviskar1, sounds great! Thank you! 😄

yurijmikhalevich / rclip

feat: option to do indexing without a query #92