search

ConvLab / ConvLab-3

Apache License 2.0
104 stars 30 forks source link

[BUG] keyerror: 'semi' #101

Open lanlanlan3 opened 1 year ago

lanlanlan3 commented 1 year ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Go to Getting_Started.ipynb
  2. Click on sys_agent.response("I want to find a moderate hotel")
  3. Scroll down to '....'
  4. See error

Expected behavior no bug

Actual behavior KeyError Traceback (most recent call last) Cell In[3], line 1 ----> 1 sys_agent.response("I want to find a moderate hotel")

File ConvLab-3/convlab/dialog_agent/agent.py:175, in PipelineAgent.response(self, observation) 172 state = deepcopy(state) # get rid of reference problem 173 # get action 174 # get rid of reference problem --> 175 self.output_action = deepcopy(self.policy.predict(state)) 177 # get model response 178 if self.nlg is not None:

File ConvLab-3/convlab/policy/rule/multiwoz/rule.py:31, in RulePolicy.predict(self, state) 23 def predict(self, state): 24 """ 25 Predict an system action given state. 26 Args: (...) 29 action : System act, with the form of (act_type, {slot_name_1: value_1, slot_name_2, value_2, ...}) 30 """ ---> 31 return self.policy.predict(state)

File ConvLab-3/convlab/policy/rule/multiwoz/rule_based_multiwoz_bot.py:96, in RuleBasedMultiwozBot.predict(self, state) 94 # User's talking about other domain ... --> 184 for slot in state['belief_state'][domain.lower()]['semi']: 185 if state['belief_state'][domain.lower()]['semi'][slot] != "": 186 constraints.append([slot, state['belief_state'][domain.lower()]['semi'][slot]])

KeyError: 'semi'

Additional context Add any other context about the problem here.

zqwerty commented 1 year ago

Rule Policy uses old format as in convlab-2. I will update the tutorial with new unified format models ASAP. Thanks for reporting!

zqwerty commented 1 year ago

112 update tutorial with new models. More features are on the way.

JamesCao2048 commented 8 months ago

Hi, I want to evaluate the end2end pipeline performance with RuleDST. Is there any method to perform a hotfix? Or is there any guide for me to adapt the RuleDST to ConvLab3 format? I am happy to make a PR.

zqwerty commented 8 months ago

Sorry for the late reply. Could you show some error logs or give more details? Originally, this issue is for RulePolicy not RuleDST

JamesCao2048 commented 8 months ago

I am sorry for my mistake, it is not about RuleDST, but RulePolicy. For replication step, I run 'python examples/agent_examples/test_BERTNLU-RuleDST-RulePolicy-TemplateNLG.py' and met exact the same error like lanlanlan3 mentioned. So I wonder if it possible to adapt RulePolicy to ConvLab3? As I see in ConvLab2 evaluation results, the RulePolicy outperforms other learning-based policies, does RulePolicy outperforms newly added policies in ConvLab3?