isi-nlp / boteval

Chat bot evaluation
https://cutelab.name/boteval
Apache License 2.0
5 stars 1 forks source link

task deployment configurations should be configured from config file #71

Open wise-east opened 11 months ago

wise-east commented 11 months ago

each customizable item for a task deployment should be configured inside a config file and the frontend should be loaded accordingly

e.g.

model: 
  type: dropdown
  items: 
    - gpt4
    - gpt3 
number_turns:
  type: slider 
  start: 1
  end: 4 
...
wise-east commented 11 months ago

the goal is to remove any hardcoded dependencies to darma. please open any other relevant issues that you deem suitable for this goal

River-vocal commented 10 months ago

Refactoring steps to consider:

  1. Move the “limits” part of conf.yml into boteval repo.
  2. Create a new section in conf.yml to decide what options to display when creating a new task.
  3. The user of boteval can choose which entry of the “limits” will be displayed under the Actions column of topics.html (The entries not being displayed will always keep their default values)
  4. Create a new module under topics for the customized task deployment options
  5. We should have a new load_bot_agent() method under bots.py. Or we always initialize a new bot under the get_dialog_man() method of service.py.
  6. Following the previous bullet point, darma should have a new way to initialize a bot (with and only with a dict as argument)
  7. Each thread should store the customized dict setting