Test Summary - 7/4/2019 - 5004

[SME-Issues]

Total tests: 3906 Passed: 2987 Failed: 919

Canonical

• Query Balance Tests Canonical (418): 98% pass (395), 7 failed understood (#4767)
• Query Balance Tests Primary Expressions (6): 100% pass (6), 0 failed understood (-)
• Followup Tests Canonical (101): 99% pass (100), 1 failed understood (#4788)
• Followup Tests Stateful (10): 100% pass (10), 0 failed understood (-)
• Query Invoice Tests Canonical (247): 95% pass (226), 12 failed understood (#4791)
• Invoice Query Primary Expression Tests (6): 100% pass (6), 0 failed understood (-)
• Query Payment Tests Canonical (191): 94% pass (173), 12 failed understood (#4796)
• Payment Query Primary Expression Tests (38): 100% pass (38), 0 failed understood (-)

  Non Canonical

• Query Balance Tests Non Canonical (543): 97% pass (460), 12 failed understood (#4769)
• Followup Tests Non Canonical (355): 99% pass (235), 2 failed understood (#4789)
• Query Invoice Tests Non Canonical (783): 91% pass (594), 59 failed understood (#4794)
• Query Payment Tests Non Canonical (437): 82% pass (315), 71 failed understood (#4799)

  Compound

• Query Balance Tests Multiple Contexts (6): 100% pass (6), 0 failed understood (-)
• Compound Query Tests Balance Balance (21): 100% pass (21), 0 failed understood (-)
• Compound Query Tests Balance Followup (3): 100% pass (3), 0 failed understood (-)
• Compound Query Tests Balance Invoice (28): 96% pass (27), 1 failed understood (#4770)
• Compound Query Tests Balance None (12): 100% pass (11), 0 failed understood (#4771)
• Compound Query Tests Balance Partial (19): 63% pass (12), 7 failed understood (#4772)
• Compound Query Tests Balance Payment (29): 86% pass (24), 4 failed understood (#4773)
• Compound Query Tests Invoice Followup (13): 77% pass (10), 3 failed understood (#4774)
• Compound Query Tests Invoice Invoice (64): 92% pass (56), 5 failed understood (#4775)
• Compound Query Tests Invoice None (24): 81% pass (13), 3 failed understood (#4776)
• Compound Query Tests Invoice Partial (20): 63% pass (12), 7 failed understood (#4777)
• Compound Query Tests Invoice Payment (31): 57% pass (17), 13 failed understood (#4778)
• Compound Query Tests Payment Followup (12): 92% pass (11), 1 failed understood (#4779)
• Compound Query Tests Payment None (7): 100% pass (7), 0 failed understood (#4780)
• Compound Query Tests Payment Partial (18): 12% pass (2), 15 failed understood (#4781)
• Compound Query Tests Payment Payment (38): 70% pass (26), 11 failed understood (#4782)

  Comprehension

• Query Balance Tests Comprehension None (3): 100% pass (3), 0 failed understood (-)
• Query Balance Tests Comprehension Partial (19): 84% pass (16), 3 failed understood (#4768)
• General Comprehension None Tests (24): 74% pass (17), 6 failed understood (#4790)
• Query Invoice Tests Comprehension None (11): 70% pass (7), 3 failed understood (#4792)
• Query Invoice Tests Comprehension Partial (27): 62% pass (16), 10 failed understood (#4793)
• Query Payment Tests Comprehension None (26): 58% pass (15), 11 failed understood (#4797)
• Query Payment Tests Comprehension Partial (25): 68% pass (15), 7 failed understood (#4798)

  Utility

• Query Invoice Tests Not Supported (15): 58% pass (7), 5 failed understood (#4795)

  Demo Script

• Demo Script Stateful Tests (11): 91% pass (10), 1 failed understood (#4783)
• Demo Script Stateless Tests (37): 94% pass (34), 2 failed understood (#4784)
• Demo Script V 2 Stateful Tests (77): 91% pass (68), 7 failed understood (#4785)
• Demo Script V 2 Stateless Tests (77): 93% pass (70), 5 failed understood (#4786)
• Demo Script V 3 Stateful Tests (74): 100% pass (71), 0 failed understood (#4787)

  Overview

  Group	Total	Passed	Failed
  Canonical	1017	926	91
  Non Canonical	2118	1478	640
  Compound	345	245	100
  Comprehension	135	78	57
  Utility	15	7	8
  Demo Script	276	253	23