search

pokt-scan / pocket-ml-testbench

MIT License
0 stars 0 forks source link

61 evalator finish code for lmeh #65

Closed AguirreNicolas closed 4 months ago

AguirreNicolas commented 4 months ago

This PR concludes the Evaluator (hopefully). The main changes can be summarized as:

  1. Responses not properly formated or not answered at all are removed.
  2. Only those docs_ids filled by all its instances/responses are kept. This implies also filtering those docs_ids from self.dataset[self.eval_split].
  3. get_task_manager now initialize properly the logger into EvaluatePocketNetworkConfigurableTask.
  4. In MongoDB , the 'tokenizer' index now include inique=true.
  5. Fix metrics in packages/python/lmeh/utils/task_config.py.
  6. Code to save result in mongoDB in PocketNetworkMongoDBResultNumerical format.