Toxicity detection test inconsistently passing

[Etesam913]

While the toxicity detection cypress test usually passes I would say that it fails too often.

This happens when the test types in "I hate bob!" and this is supposed to show the alert of the message being too toxic. There is a timeout of 40 seconds when trying to get the toxicity alert. However, often times the github action does not see the alert in time. As detoxify has to be ran to see the alert(the alert shows the numerical value of the toxicity) it is very time intensive for the github action.

I don't really know a solution to this. I guess I could increase the timeout to a number larger than 40 seconds. Any suggestions?

For example: In this run it fails to get the alert in 40,000ms: https://github.com/facebookresearch/Mephisto/runs/7416173868?check_suite_focus=true

In this run it gets it in 26155ms https://github.com/facebookresearch/Mephisto/runs/7416265029?check_suite_focus=true

[JackUrb]

One option you could roll with (though not great) is to monkeypatch or otherwise mock out detoxify for something much cheaper, like:

class Detoxify():
   ...
   def run(self, txt):
      if txt == 'I hate bob!":
          return True
      else:
          return False

Though then you'd have to find a way to inject this into your test script (perhaps with a hydra arg that uses this as imported from a test folder).

[Etesam913]

Doesn't really seem to be an issue anymore. I increased the timeout to 80000. This is also ameliorated by the path filtering as the test will only run when you change the toxicity detection folder.