search

inferno-framework / bulk-data-test-kit

Inferno Test Kit for servers implementing the Bulk Data Access Implementation Guide
Apache License 2.0
0 stars 0 forks source link

FI-2486: Cleanup Bulk Data Test Kit #10

Closed emichaud998 closed 6 months ago

emichaud998 commented 7 months ago

Summary

This PR updates the bulk data test kit's README and suite descriptions to have correct information related to the test kit, and a linter has been run on the test kit to clean up the code. This PR also fixes the issue with the Bulk Data capability statement export operation support test so that it checks for an export operation with the name 'export' for each of the different types of exports, rather than 'group-export' for group level export, 'patient-export' for patient level export, and 'export' for system level export.

Testing Guidance

Run the test kit and make sure everything runs as expected, and read through README and suite descriptions to make sure they look good as well.

arscan commented 6 months ago

Lots of 'purple errors' when just running tests against the presets. Purple errors should never show up, and indicates behavior of a system under that we didn't even consider to cause a failure/skip. So as a rule, never let presets result in purple errors anywhere (to me, it indicates we never even tried the presets against the tests). I created tickets.

Also, the description seems to presume that systems will have the backend services authorization in place (to start, register inferno...), which is definitely not necessary. So we need to revise the description a bit. I may just delete the language really quickly due to lack of time but this could be improved.

emichaud998 commented 6 months ago

Lots of 'purple errors' when just running tests against the presets. Purple errors should never show up, and indicates behavior of a system under that we didn't even consider to cause a failure/skip. So as a rule, never let presets result in purple errors anywhere (to me, it indicates we never even tried the presets against the tests). I created tickets.

Also, the description seems to presume that systems will have the backend services authorization in place (to start, register inferno...), which is definitely not necessary. So we need to revise the description a bit. I may just delete the language really quickly due to lack of time but this could be improved.

I just ran both presets, and the only purple errors being thrown are the ones from the TLS tests, which had to be fixed in the TLS test kit. If the TLS test kit has had a release since then, the bulk data test kit will just need to be updated to have the new release version with that change.

Also, I updated the suite descriptions so that it explains that the backend services authorization is not necessary

arscan commented 6 months ago

Thanks!

I just ran both presets, and the only purple errors being thrown are the ones from the TLS tests, which had to be fixed in the TLS test kit. If the TLS test kit has had a release since then, the bulk data test kit will just need to be updated to have the new release version with that change.

Hmmm, strange. I opened tickets with screenshots. Maybe because i was running all as a group instead of one by one? We don't have time to fix here.

I pushed with a few more updates; like wrapping text blocks at 80 characters, updating the version to 0.9.0 (we decided that we will have initial versions ready for deployment set to that level vs 0.1.0 or 0.0.1), fixed the links on the bottom that were pointing to the wrong spots. Just a few things to look out for the next time we are about to deploy (its a lot, I know).