search

LiuQL2 / MedicalChatbot

Medical-domain Dialogue System for Diseases Identification
117 stars 38 forks source link

The test results of each run are diffirent. Is this a bug? #4

Open guojianwei opened 5 years ago

guojianwei commented 5 years ago

I run test from saved model, but the SR/ABSR of each run is very different. In src/dialogue_system/agent/agent.py, the _build_action_space method feasible_actions are Initialized from slot_set and disease_symptom which are loaded from pickle. So the action order in action_sapce may be different. I modified the code in line 262: for slot in sorted(self.slot_set.keys()): line 269: for disease in sorted(self.disease_symptom.keys()): it worked for me.

LiuQL2 commented 5 years ago

Thank you very much for pointing this out and fixing it. I always thought it was caused by the randomness of the user simulator. Thanks again!

ares5221 commented 5 years ago

I run test from saved model, but the SR/ABSR of each run is very different. In src/dialogue_system/agent/agent.py, the _build_action_space method feasible_actions are Initialized from slot_set and disease_symptom which are loaded from pickle. So the action order in action_sapce may be different. I modified the code in line 262: for slot in sorted(self.slot_set.keys()): line 269: for disease in sorted(self.disease_symptom.keys()): it worked for me.

halo, i have some question when run test, ValueError: The passed save_path is not a valid checkpoint: ./../model/dqn/checkpoint/checkpoint_d4_agt1_dqn1/model_d4_agent1_dqn1_s0.619_r18.221_t4.266_wd0.0_e432.ckpt did you have same question

AndreaMarino94 commented 2 years ago

Eseguo il test dal modello salvato, ma l'SR/ABSR di ogni corsa è molto diverso. In src/dialogue_system/agent/agent.py, il metodo _build_action_space feasible_actions viene inizializzato da slot_set e disease_symptom che vengono caricati da pickle. Quindi l'ordine delle azioni in action_sapce potrebbe essere diverso. Ho modificato il codice nella riga 262: for slot in sorted(self.slot_set.keys()): riga 269: for disease in sorted(self.disease_symptom.keys()): ha funzionato per me.

alone, ho qualche domanda quando si esegue il test, ValueError: il save_path passato non è un checkpoint valido: ./../model/dqn/checkpoint/checkpoint_d4_agt1_dqn1/model_d4_agent1_dqn1_s0.619_r18.221_t4.266_wd0.0_e432.ckpt hai avuto la stessa domanda

you can take the saved_model path, from checkpoint.txt in "src\dialogue_system\model\dqn\checkpoint04"