A few of the test suites have all duplicate test assets

NCATSTranslator / Tests

3 stars 2 forks source link

A few of the test suites have all duplicate test assets #46

Open maximusunc opened 4 months ago

maximusunc commented 4 months ago

This doesn't affect the currently running tests, but I just noticed that some of the test suites had duplicate test assets throughout.

pass_fail_test_suite_output
stress_test_CI

sierra-moxon commented 4 months ago

Wasn't the main distinguishing factor between the various suites, the environment, and the number of tests? Said another way, is this a problem?

maximusunc commented 4 months ago

No, I mean inside of each of these test suites, there are duplicates of every test assets, i.e.

{
  "TestCase": {
    "test_assets": [
      {
        "id": "TestAsset1"
      },
      {
        "id": "TestAsset1"
      },
      {
        "id": "TestAsset2"
      },
      {
        "id": "TestAsset2"
      },
      ...
    ]
  }
}

isbluis commented 2 months ago

I found the following duplicates in the test suite. Hope it is okay to add to this issue, and apologies if this is already being worked on.

Test Case 50

Assets 487 and 497 are identical
Assets 488 and 498 "
Assets 489 and 499 "
Assets 490 and 500 "
Assets 491 and 501 "

Test Case 47

Asset 407 is declared twice

Test Case 53

Assets 6 and 77 are identical (mentioned already in #28 )

maximusunc commented 2 months ago

Thanks for the report, Luis! @sierra-moxon it looks like all of these assets have different personas. The Test Assets currently don't reflect this property. We probably need to add this to the testing model?