search

StarlightSearch / EmbedAnything

A minimalist yet highly performant, lightweight, lightning fast, multisource, multimodal and local Ingestion, Inference and Indexing solution, built in Rust.
Apache License 2.0
313 stars 26 forks source link

Add Timing Information for File Reading, Preprocessing, and Encoding #38

Open cavitcakir opened 4 months ago

cavitcakir commented 4 months ago

Hi,

I would like to request the addition of timing information for various stages of the embedding pipeline, specifically for reading files, preprocessing them, and encoding them. Having detailed timing information will help in understanding the performance characteristics and identifying potential bottlenecks in the workflow.

Thank you for considering this feature request. I believe it will greatly enhance the usability and performance monitoring capabilities of the project.

akshayballal95 commented 4 months ago

We can possibly add a verbose argument on the embedding function to display these times and other stars if needed.