search

cognitive-catalyst / WA-Testing-Tool

Scripts that run against Watson Assistant for K fold validation on training set, testing on blind test, and draw precision curves for comparison.
Apache License 2.0
78 stars 61 forks source link

#215 support for Actions-based logs #227

Closed andrewrfreed closed 1 year ago

andrewrfreed commented 1 year ago

Addresses #215 for logs

Keeps existing APIs, uses intelligent in-place toggles to deal with v1 or v2 in same branches. v2 log fetching requires a new argument, environment_id, which is tied to an Assistant and an Environment

Signed-off-by: Andrew R Freed afreed@us.ibm.com

lmazzoli commented 1 year ago

Hi @andrewrfreed, this looks great overall, but I Noticed a few minor things.

  1. For getAllLogs.py V1, I was still required to provide the assistant/environment ID even though the README example for the dialog-based workspace command. Also, I could not get the
  2. For getAllLogs.py V1, I could not get the logs using the example command for dialog-based Assistant.
  3. On extractConversations.py for V2 logs, the dialog_turn_counter does not seem to be resetting for each conversation.
andrewrfreed commented 1 year ago

Hi @andrewrfreed, this looks great overall, but I Noticed a few minor things.

1. For getAllLogs.py V1, I was still required to provide the assistant/environment ID even though the README example for the dialog-based workspace command.  Also, I could not get the

2. For getAllLogs.py V1, I could not get the logs using the example command for dialog-based Assistant.

3. On extractConversations.py for V2 logs, the dialog_turn_counter does not seem to be resetting for each conversation.

Addressed via additional example in readme after we determined the wrong conversation field marker was being used.