Test Summary - 19/07/2019 - 5004

[SME-Issues]

Total tests: 3906 Passed: 2963 Failed: 943

Canonical

• Query Balance Tests Canonical (418): 99% pass (396), 6 failed understood (#4801)
• Query Balance Tests Primary Expressions (6): 100% pass (6), 0 failed understood (-)
• Followup Tests Canonical (101): 99% pass (100), 1 failed understood (#4822)
• Followup Tests Stateful (10): 100% pass (10), 0 failed understood (-)
• Query Invoice Tests Canonical (247): 94% pass (225), 14 failed understood (#4825)
• Invoice Query Primary Expression Tests (6): 100% pass (6), 0 failed understood (-)
• Query Payment Tests Canonical (191): 91% pass (169), 16 failed understood (#4830)
• Payment Query Primary Expression Tests (38): 97% pass (37), 1 failed understood (#4834)

  Non Canonical

• Query Balance Tests Non Canonical (543): 97% pass (459), 13 failed understood (#4803)
• Followup Tests Non Canonical (355): 99% pass (235), 2 failed understood (#4823)
• Query Invoice Tests Non Canonical (783): 91% pass (592), 60 failed understood (#4828)
• Query Payment Tests Non Canonical (437): 81% pass (313), 73 failed understood (#4833)

  Compound

• Query Balance Tests Multiple Contexts (6): 100% pass (6), 0 failed understood (-)
• Compound Query Tests Balance Balance (21): 100% pass (21), 0 failed understood (-)
• Compound Query Tests Balance Followup (3): 100% pass (3), 0 failed understood (-)
• Compound Query Tests Balance Invoice (28): 96% pass (27), 1 failed understood (#4804)
• Compound Query Tests Balance None (12): 100% pass (11), 0 failed understood (#4805)
• Compound Query Tests Balance Partial (19): 63% pass (12), 7 failed understood (#4806)
• Compound Query Tests Balance Payment (29): 86% pass (24), 4 failed understood (#4807)
• Compound Query Tests Invoice Followup (13): 62% pass (8), 5 failed understood (#4808)
• Compound Query Tests Invoice Invoice (64): 92% pass (56), 5 failed understood (#4809)
• Compound Query Tests Invoice None (24): 81% pass (13), 3 failed understood (#4810)
• Compound Query Tests Invoice Partial (20): 63% pass (12), 7 failed understood (#4811)
• Compound Query Tests Invoice Payment (31): 50% pass (15), 15 failed understood (#4812)
• Compound Query Tests Payment Followup (12): 92% pass (11), 1 failed understood (#4813)
• Compound Query Tests Payment None (7): 100% pass (7), 0 failed understood (#4814)
• Compound Query Tests Payment Partial (18): 12% pass (2), 15 failed understood (#4815)
• Compound Query Tests Payment Payment (38): 57% pass (21), 16 failed understood (#4816)

  Comprehension

• Query Balance Tests Comprehension None (3): 100% pass (3), 0 failed understood (-)
• Query Balance Tests Comprehension Partial (19): 84% pass (16), 3 failed understood (#4802)
• General Comprehension None Tests (24): 74% pass (17), 6 failed understood (#4824)
• Query Invoice Tests Comprehension None (11): 70% pass (7), 3 failed understood (#4826)
• Query Invoice Tests Comprehension Partial (27): 62% pass (16), 10 failed understood (#4827)
• Query Payment Tests Comprehension None (26): 58% pass (15), 11 failed understood (#4831)
• Query Payment Tests Comprehension Partial (25): 68% pass (15), 7 failed understood (#4832)

  Utility

• Query Invoice Tests Not Supported (15): 58% pass (7), 5 failed understood (#4829)

  Demo Script

• Demo Script Stateful Tests (11): 91% pass (10), 1 failed understood (#4817)
• Demo Script Stateless Tests (37): 94% pass (34), 2 failed understood (#4818)
• Demo Script V 2 Stateful Tests (77): 91% pass (68), 7 failed understood (#4819)
• Demo Script V 2 Stateless Tests (77): 93% pass (70), 5 failed understood (#4820)
• Demo Script V 3 Stateful Tests (74): 100% pass (71), 0 failed understood (#4821)

  Overview

  Group	Total	Passed	Failed
  Canonical	1017	918	99
  Non Canonical	2118	1469	649
  Compound	345	238	107
  Comprehension	135	78	57
  Utility	15	7	8
  Demo Script	276	253	23