Goal Directed Testing: Brainstorm

[podhrmic]

In GitLab by @zutshi on Oct 5, 2020, 13:45

• Identify and prioritize failure conditions:

  • Stall
  • Pitch oscillations
  • Ground collision
  • Air collision
  • ...

• Integrate test generation approaches, such as:

  • MCTS (Zikang/He/Suresh)
  • Robustness guided [S-taliro/Breach]
  • S3CAM

[podhrmic]

In GitLab by @zutshi on Oct 5, 2020, 13:46

cc @mattclark @cslockett @EthanJamesLew@bauer-matthews @podhrmic

[podhrmic]

In GitLab by @zutshi on Oct 5, 2020, 13:53

changed title from Goal Directed Testing: Integ{-ar-}te Falsification/test generation approaches to Goal Directed Testing: Integ{+ra+}te Falsification/test generation approaches

[podhrmic]

In GitLab by @podhrmic on Oct 6, 2020, 10:00

I think I will need a bit more context to provide a better answer, but just quickly:

• don't consider stall, because the plant doesn't reproduce stall accurately - checking flight limits (such as max alpha, min airspeed etc.) would be better
• ground collision - that is the easiest, if you assume a flat ground. Otherwise we would have to deal with some terrain models etc.
• air collision - doesn't neatly fit into the current framework (we have only 1 plant at a time), but could be added
• pitch oscillations - or flight characteristics (phugoid mode, dutch-roll mode) in general. Definitely interesting, but perhaps lower priority

So I would prioritize as:

1. ground collision assuming flat ground
2. flight limits violation

[podhrmic]

In GitLab by @bauer-matthews on Oct 8, 2020, 04:42

For test generation approaches, I would say we focus on MCTS for now. @zutshi Can you update this issue once you've had a chance to speak with Zikang? I'm particularly interested to know how long you think the integration will take.

[podhrmic]

In GitLab by @bauer-matthews on Oct 12, 2020, 06:39

changed title from Goal Directed Testing: {-Integrate Falsification/test generation approaches-} to Goal Directed Testing: {+Brainstorm+}

[podhrmic]

In GitLab by @bauer-matthews on Oct 12, 2020, 06:40

Bumping this to a brainstorm and creating a separate issue for MCTS.

[podhrmic]

In GitLab by @podhrmic on Jul 13, 2021, 14:21

@zutshi @EthanJamesLewclose this?