search

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
283 stars 204 forks source link

Request QA Review for Rated Disabilities #3099

Closed jason-gcio closed 4 years ago

jason-gcio commented 5 years ago

Story

As an engineer, I want to request a review for Security and Load Testing with the QA Team to ensure that my feature meets the requirements for publication so that my pages will safely perform as expected under heavy loads typically found in the VA.gov environment.

Task

Acceptance Criteria

micahchiang commented 5 years ago

This ticket is blocked until https://github.com/department-of-veterans-affairs/va.gov-team/issues/1889 is resolved.

tlei123 commented 5 years ago

@jason-gcio , here's a question* from my VSA-DevOps team-lead @jcosta-gcio:

“If they do a UI for that call can we test the UI do we have a software for load testing ?”

Pls. let me know. Thx much!

*I'm currently only able to perform manual UI testing.

tlei123 commented 4 years ago

@jason-gcio , beyond the design comps provided, are there any interactive wireframes (like in InVision?) that clearly delineate the user stories/scenarios?

For starters, I was looking at the first two design comps, and couldn't figure out how the User navigates from the initial landing page (before sign-in) to the following page (after successful sign-in).

Please let me know. Thanks much! :)

jason-gcio commented 4 years ago

@tlei123, we don't have the unauthenticated page in place yet as Public Websites is creating that. Presumably, once Authenticated Experience determines the best home for where this page gets linked from we will know more about how this page gets accessed. As you can see in the mockup, the initial landing page, if already logged in, would take the user directly to the tool page. If not, they should be presented with a login and get redirected after authentication.

With respect to Joe's question, I am not sure what he is asking...should we have a quick call to discuss further?

tlei123 commented 4 years ago

@jason-gcio, re. @jcosta-gcio’s load-testing question, I'm just trying to see if I can actually take on the security & load testing you mentioned in the Description...

I am not aware of any UI available to QA for validating security or load performance, but Joe wants to make sure. These 2 types of testing are usually conducted without QA involvement.

That said, even if no UI exists for me to actually conduct testing, I would still like to see if I can validate/document the security-/load-test results.

Hope this clarifies the question for you.

Please let me know. Thanks much!

micahchiang commented 4 years ago

Hey @tlei123 - We have #2951 which I am currently starting work on. I was just planning on following the steps you've laid out in this document, and the document it links through to.

Is this ticket something different when it comes to load performance? I'm a little confused 😅

tlei123 commented 4 years ago

@jason-gcio, per @micahchiang’s comment above re. that separate load-testing ticket #2951, do you think we can remove load-testing from this ticket (3099) here?

Not sure if the two tickets are talking about the same load-testing requirement, so I thought I would double-check with you.

Please let us know. Thanks much!

micahchiang commented 4 years ago

@jason-gcio, @tlei123 - I think load testing can be removed from this ticket, as I have completed it in #2951. So, if that ticket takes care of load testing, I'm wondering what testing is left for this ticket. Is this purely a visual QA test of rated disabilities?

tlei123 commented 4 years ago

@jason-gcio & @micahchiang, would you happen to know how to UI-test the no-rating and API-/backend-error messages on Staging?

Here are the message comps:

I am figuring the no-rating message is probably a test-account-specific scenario, so I would probably just need the Staging no-rating test-account, but I do not know how to test the two error-messages on Staging.

Please let me know. Thanks much!

tlei123 commented 4 years ago

@jason-gcio , please confirm that for no-ratings scenario [vets.gov.user+10@gmail.com] the dynamic ratings page should be blank [see screenshot below].

I wasn't sure whether I should open a Bug for this:

rated-disabilities-no-ratings
tlei123 commented 4 years ago

Progress update:

Will continue test-runs for other browsers tomorrow (Tuesday).

*NOTE: Another bug may be opened pending @jason-gcio ’s answer to my question in previous Comment.

tlei123 commented 4 years ago

@jason-gcio , although I may ultimately attach a Testing Matrix spreadsheet w/ results once all test-runs are done, I’m sharing here a PDF preview of an interim Test-Case Report from the new TestRail test case management system (TCMS) that @peterhill-oddball & I are piloting right now...

Please check it out as time allows, and let me know what you think. Thanks much!

testrail-report-7.pdf

tlei123 commented 4 years ago

Update progress:

Although browser-/device-agnostic design/content changes are still upcoming, browser-/device-specific testing is well underway [for already-testable aspects like dynamic data-display and links to other resources].

jason-gcio commented 4 years ago

@jason-gcio , please confirm that for no-ratings scenario [vets.gov.user+10@gmail.com] the dynamic ratings page should be blank [see screenshot below].

I wasn't sure whether I should open a Bug for this:

rated-disabilities-no-ratings

@tlei123 For user #10, that seems to be an isolated case...other users that have no rating render correctly and we think the data for #10 is bad or weird. @micahchiang is looking deeper into it but it looks isolated at this time.

micahchiang commented 4 years ago

Hey @tlei123 - I did some more digging around, and with the help of another engineer, was able to determine the potential root of this issue. It seems like users who aren't in MVI don't get handled properly on our Rated Disabilities page, and that causes the page to break.

@jason-gcio - I've created https://github.com/department-of-veterans-affairs/va.gov-team/issues/4222 to address the work of adding MVI error handling to rated disabilities.

micahchiang commented 4 years ago

@tlei123 - For example, if you log in with test user 10 and checkout the network call for user, you'll see that it comes back with errors relating to MVI. Logging in with user 17, the same network call comes back without any errors.

jason-gcio commented 4 years ago

Hey @tlei123! I think we are ready to pick this back up. Please let me know if you would like to chat further!

tlei123 commented 4 years ago

Modified test cases after bugfix #4222 closed. Started a new test-run -- planning to complete new run by end of tomorrow (Tuesday).

New defect(s) logged so far:

jason-gcio commented 4 years ago

@tlei123 since those changes related to #4462 are already in place waiting for a PR, I am going to move this to Validate for now and we can close it once the PR comes in.

tlei123 commented 4 years ago

All test-runs except Win IE11* are now complete.

@jason-gcio , here a link to the Test Plan Summary report on our TestRail.com test case mgmt sytem (TCMS): https://dsvavsp.testrail.io/index.php?/reports/view/24 [VSA shared-account credentials emailed to you separately]

A PDF export is also attached: https://app.zenhub.com/files/133843125/08c42bcb-dce7-4745-9d4b-e043d9968565/download

I'll walk though this report in our upcoming online chat/mtg. There's a truckload of available reports available on TestRail, so I'd love to get your feedback.

* Re. Win IE11, it just plain froze at the ID.me sign-in popup. I'll be chatting w/ my VSA Engineering and DevOps Lead @jcosta-gcio about this, and I'll keep you posted on this front.

jason-gcio commented 4 years ago

@tlei123:

4300 is all set so once it gets through its next build it should be ready for those last two checks to be run. Keeping this in Icebox for now.

cc: @micahchiang

tlei123 commented 4 years ago

Blocked/untested/failed Test-Runs now all Passed, after PR merged for bugfix #4300 ...

@jason-gcio , Test Plan 2.1 Summary Report is now available on TestRail -- PDF export is attached.

Feel free to Close this ticket. :)

jason-gcio commented 4 years ago

Awesome, thanks @tlei123!