search

GSA-TTS / FAC

GSA's Federal Audit Clearinghouse
Other
19 stars 5 forks source link

Epic: Internal stress testing #1990

Closed jadudm closed 1 year ago

jadudm commented 1 year ago

How resilient is the FAC to intentional abuse?

For each form and form element, are there ways to break the application by intentionally? Are there ways to advance past a form with it incomplete?

This kind of testing is hard, because it requires the tester to look for ways to intentionally break or otherwise deceive the application.

### Bugs found
- [ ] #1994
- [ ] https://github.com/GSA-TTS/FAC/issues/1995
- [ ] #1996
- [ ] #1997
- [ ] https://github.com/GSA-TTS/FAC/issues/2000
- [ ] https://github.com/GSA-TTS/FAC/issues/2017

About this round of testing

Borrowing from a few sources...

(From https://usersnap.com/blog/web-application-testing/)

Functional testing

We've done a lot of this. Our regression tests do this. Our user tests did this. So, we're not doing functional testing here.

User testing

Nope. That's definitely something else.

Interface testing

More like this.

What happens if you start uploading a file, and close the browser window part-way through? Does it... show as uploaded? Does the audit get destroyed? Does the FAC melt down?

Compatibility testing

Can we do a complete submission under Firefox? Edge? Opera? Brave?

(A better list might be something like this: https://www.oberlo.com/statistics/browser-market-share)

Can we submit from both Windows and Mac, using those browsers?

Performance testing

This is for another day. We'll automate that. We've already done some of this, but not systematically.

Security testing

This was conducted by GSA IT.

Reporting bugs

The bug reporting ticket is your friend. To help out, after completing a bug ticket, label it as "BUG", set the project to FAC Task Tracking, and set the status to Triage. You can also assign it to jadudm.

Now, there's a lot to be said about reporting bugs effectively. You should definitely try and be thorough in your bug reporting, but also, we have a lot of fields on our bug report ticket. Be as thorough as you think you need to be for the dev team to reproduce what you did. However, you're also members of the team, so... if your ticket is a bit thin, and we have questions, we're in the same Slack, and can definitely follow up.

Tracking your work

You can use a GDoc to sync, or you can edit this ticket and use it. Feel free to add comments as you discover things, have questions, etc.

The checklist below is a starting point to help you orient around the application.

### Parts of the FAC
- [ ] Pre-screener (form)
- [ ] Access info/email addresses (form)
- [ ] General information (form)
- [ ] Audit information (form)
- [ ] PDF upload (form/file)
- [ ] XLSX upload (file uploads)
- [ ] Pre-submission validation (button press)
- [ ] Auditor certification
- [ ] Auditee certification
- [ ] Submission

Testing uploads

I would start by asking @Leighdiddy for a copy of the recent testing materials; this will provide instructions as well as files. We can generate more workbooks for you as well, if you want.

Bonus

Super-mega bonus if you can find a way to create workbooks that are intentionally wrong, but you manage to upload regardless. This may be too difficult to do without knowledge of what a correct submission should be. You would almost have to look at the 2022 uniform guidance (PDF) and think about how to intentionally enter wrong information into our form/workbooks, and then upload things that we should not accept.

LeoFotovich commented 1 year ago

Found plenty of things thus far and plan to ticket accordingly soon-ish, after consolidating everything on a manageable/trackable document.

LeoFotovich commented 1 year ago

Submitted some tickets about a couple of the bugs I found, more on the way!

jadudm commented 1 year ago

Here's a fun idea:

  1. I've created a new tasklist for "bugs found."
  2. I added one of your bug tickets.
  3. You can click to add more tasks to that list.
  4. Each entry can just be the ticket number; e.g. #1999.

Github will automatically link from this issue to that bug that way, and we can find all of them easily. (We can find them anyway, but it might be nice to link them here.)

LeoFotovich commented 1 year ago

I don't think I have access to that task list?

jadudm commented 1 year ago

I mean, I created it after I wrote that note. :) And, I put all your bugs-to-date in.

LeoFotovich commented 1 year ago

lol sounds good!

jadudm commented 1 year ago

All bugs resolved or moved to triage or available in the task board. Closing this epic.