rahmansahinler1 / ragchat_local

ragchat is a chatbot that gives you most up to date iformation with resources in your documents
MIT License
1 stars 0 forks source link

Performance Tracking KPI's - 1 #1

Closed rahmansahinler1 closed 2 months ago

rahmansahinler1 commented 2 months ago

Description Performance of the individual processes like indexing, embedding, etc. needs to be tracked down for optimizing. Every run KPI's will be saved under a dictionary. Then, if yes creates .csv output.

Example For Embedding KPI's:

  1. Sentence Amount
  2. Total time spent on embedding - total_emb_time
  3. Batch size
  4. Average time spend on embedding (total time / total sentence) - avg_emb_time

Workflow

  1. Create a logic for every KPI
  2. Save KPI's to the dictionary
  3. Create a logic for generating .csv
  4. Add a control parameter to the globals.py (generate_csv)
  5. Create 10 sample .csv KPI files into the assets folder

Acceptance Criteria 10 .csv file for different runs for embedding and indexing

ozgurnsahin commented 2 months ago

Dictionary is created every runs recods are logged into a .csv file and tracked

ozgurnsahin commented 2 months ago

Iterated KPI tracking functions to only record unique batch and sentence amount pairs. Added linear regression model to optimize batch size of the embedding function and to speed up overall processes.