Update ArgoCD Terminal Access Request Template (and supporting docs)

[ryan-mcneil]

Description

The current template, while generally effective, is a bit confusing for requesters (specifically the various checkboxes that are meant to be a reference to support, not a TODO list), and generally leaves people in limbo for a LONG time (This user has been waiting for a month, despite providing everything we asked) because of the shifting requirements from OCTODE. We should update accordingly.

UPDATE Per Bill's comments, we're going to remove the requirement for eQIP and have the user confirm they have SOCKS

Definition of Done

• All tasks below are done

Tasks

• [ ] Remove mention of eQIP
• [ ] Add Checkbox/notes/link to confirm that user has SOCKS and direct to SOCKS form otherwise (see comments)
• [x] Remove the confusing checkboxes (any support directions should go int support docs)
• [x] Follow any other patterns from new Sidekiq template that might help here
• [ ] Update Internal Support Docs as necessary.

[rmtolmach]

Remove the confusing checkboxes (any support directions should go in support docs)

I get this, but I'm worried people won't read the docs 😂 One idea is to do something similar to the Offboarding doc to hopefully make it less confusing:

Or... at the very least a link to some docs.

[ryan-mcneil]

Yeah, I understand. I think that if we get approval to remove some requirements, and I shape it similar to the Sidekiq request, I think I can satisfy your concerns. Will run it by ya.

[ryan-mcneil]

@BillChapmanUSDS Today I chatted with @little-oddball and several members on the support team to hash out some confusion surrounding the ArgoCD terminal access template, eQIP requirements, and your involvement in the process.

Currently, from a high level:

• The user fills out the request with necessary info, and adds their proof of eQIP transmittal
• We provide access to lower environments
• If Prod access is requested, we tag it accordingly to get your approval

The requirement of eQIP has caused a lot of confusion, as the current template suggests they only need to check a box that they've submitted the survey, while your requirement seems to be more strict than that (i.e. they must include a screenshot). This led to me creating this ticket.

However, during our call today we made a realization that should hopefully simplify this. ArgoCD is behind SOCKS, so this SHOULD imply that they have submitted their eQIP Transmittal and have had it approved, no? Asking again here seems redundant.

Our recommendation for the updated process would be the following:

• The user fills out the request with necessary info (No eQIP needed)
• Assuming all information was provided, we grant access to lower environments
• If Prod access is requested, we tag it accordingly to get your approval. You would no longer need to confirm eQIP (per this redundancy), only give your "blessing". This will hopefully expedite things both for you, and the requester trying to do their job.

I'll also note that we discussed new ways of handling the re-request:

Short Term

• We come up with some sort of explicit verbiage that tells the user to set a reminder that they need to come back to this ticket within 1 week of expiration and tell us that they need another 3 months.
  • Per the backend docs, a reapproval can be given by a couple of our lead engineers, so we'll facilitate that.
• If they do not respond, a couple days before expiration we'll bump them, and let them know it's going to expire.
• If they fail to do so, we'll remove access and force them to go through the process again.

Long Term Clearly this "short term" solution is still tedious. We intend on coming up with a more elegant, automated solution, and will put our heads together at some point and think through what that might look like.

Let me know your thoughts! Thanks!

[BillChapmanUSDS]

Short Answer: I think this sounds fine in the short term. But I'd like it to be noted explicitly in the ticket that SOCKS access has already been granted, preferably by a team lead, COR, etc... The documentation is what we need, it's not about vetting as much right now. These tickets are currently serving as much of the documentation necessary for our ATO and therefore need to be complete. When they are complete, I almost always stamp them and move on with the occasional gripe about best practices when too much access is being required.

Long Answer The solution I have been discussing for ongoing access to production systems is two fold. For VFS Teams I think platform can happily rubber stamp any request that has the blessing of a federally employed PO and a statement along the lines of "I acknowledge that this individual has the approproriate expertise has gone through the appropriate steps to gain access to these systems, including the necessary preliminary security assessment (E-Qip etc.."

For a support team that lives under platform I think we want a more organized solution.

In the long term we need better roles organization and assignment, if you fill a certain role you get the access that role needs and it's ongoing while you serve the role. But we also need fewer folks with direct access to production resources. Because we have these two issues happening at once and they both are at odds with each other, this solution is not moving quickly.

@little-oddball and I have talked a lot about this and I don't now that there are actually any truly technical hurdles, it really is an administrative problem right now.

Support is one of those areas where a lot of access is often necessary and I'd love to discuss a solution piloted with support engineers that works as follows.

Support team and leadership together decides the roles that are on the team and which access is appropriate for that, we discuss and refine them. In the future it's just part of the onboarding workflow to a certain role. Now we don't need to haggle over whether an individual needs that access, if they are being placed in a role we assume the appropriate vetting for that role has taken place. The ticket to file for access is as simple as this person is in this role and they get this access while they are in this role. Since all the vetting has taken place, we just need to find a more efficient way to verify E-Qip or something similar.

We shouldn't be granting granular, one-off access to our environments and any replacing process needs to acknowledge that. I don't like the toil that the temporary access creates and the purpose of is more to keep the progress moving forward on a better process. I want a path to ongoing access for anyone serving a role that needs it and this will nee da combination of being way less permissions of who gets access, and a rational outline for what roles are being filed and what access they need.

[ryan-mcneil]

@BillChapmanUSDS This all makes sense and I completely agree with your long term goal. Role based permissions are the way to go, and every "good" modern platform utilizes them in some shape or another.

I do, however, want to circle back to SOCKS (with the understanding that documentation is what you're striving for). If having SOCKS is NEEDED (meaning they cannot physically access ArgoCD without it), is it necessary for it to be a requirement? It seems redundant. My goal here is to remove a step so that users don't spend weeks waiting to get access.

Would adding a checkbox towards the top of the form containing something along the lines of the following suffice?: [ ] I certify that have SOCKS and understand I cannot access ArgoCD without it (I would also include a note telling them that if they don't have SOCKS to cancel this request, and link them to the SOCKS procedure/request form)

Worst case scenario, they mistakenly/malevolently check this box, get added to the respective Github teams (for lower Envs only, as they'd still need your approval for prod) and they still CANNOT access the ArgoCD UI because they don't have SOCKS.

If we instead require a lead, COR, etc... to confirm they have SOCKS, it's really not much more efficient (perhaps less convenient) than having them add a screen shot of an email that they may or may not have. We currently have contractors that have been on various projects with the VA for years, have SOCKS/PIV but don't have this email/know their date (which is understandable).

If your answer is unwavering, in that they either need eQIP or someone to explicitly confirm their SOCKS access, does their TRANSMITTAL DATE (vs screenshot) suffice? I understand that they can reach out to Amber/their VOR and get this. I would ultimately just replace the current checkbox at the top of the form with a text input for this date. This, I'll add leaves room for adding an arbitrary date, and therefore adds little value, as the Support rep has no way of verifying in an efficient manner.

Not to beat a dead horse, I just ultimately want as little back and forth as possible so we can get these people moving.

Thanks for your input!

[BillChapmanUSDS]

@ryan-mcneil I'm never unwavering, and thank you for your concern for making the process easier. The worry I had with the individual certifying is there is nothing in the ticket that serves as a documented check and balance against it. Part of the problem is that on the platform side we were spending a lot of time going back and forth with CORS and other leadership to verify, and while that work is not seen by the requestor, it greatly delays the time it takes to make a determination and ensure the documentation is in order. I'm ok with a checkbox but the federal validation still needs to take place. I was suggesting we just understand it has to happen and have it done up front instead of having it done in the background.

But you've sold me on this point:

"Worst case scenario, they mistakenly/malevolently check this box, get added to the respective Github teams (for lower Envs only, as they'd still need your approval for prod) and they still CANNOT access the ArgoCD UI because they don't have SOCKS."

I think the checkbox is acceptable in this case.

[ ] I certify that have SOCKS and understand I cannot access ArgoCD without it.

That's a good short term solution.

[rmtolmach]

You could have them add a link to their SOCKS (or AWS) request. Not to confuse things more, but I think we've only been requiring E-QIP for SOCKS since March 24 (in this commit).

[ryan-mcneil]

@BillChapmanUSDS well Rebecca has shaken my confidence in my approach. What are you thoughts?

I can ask for them to provide 1 of these many things to prove their eQIP Transmittal, which adds back some complexity/potentially confusing verbiage. I also feel like there's a chance that users (like the ones that drove me to spin up this ticket) won't be able to check any of these boxes but still be qualified, as they've been around these programs for years, have clearance, but don't have these emails/issues.

Should we be worried that there might be users that have been around since before March 24 that might have SOCKS but have somehow flown under the radar and don't have clearance? They still would have had to be on the VFS Roster and have provided all of the other details.

[BillChapmanUSDS]

Yes, The list of folks without E-Qip documented was another concern I had. My hope was that since we are still doing 3 month access limits, The fix for that needs to be rolled in to the bigger long term solution that grants ongoing access.

Thinking out loud: @rmtolmach would it be possible for us to start a separate initiative, let's get EQuip documented for everyone that existed prior to the current process date. I can help with this but I'd like your thoughts. For those folks many of them have actually been adjudicated already and the EQip submission is not really relevant anymore, we just use that for expedience.

I wonder if we should start collecting proof of adjudication, then we only need EQip dates for new folks.

But either way, if we come up with a sperate process for accounting for the "legacy" access folks we can simplify this process and the other is a one-off.

[rmtolmach]

Just for our understanding, why has EQIP proof become the gold standard for access approval? Is it because it's an ATO requirement?

Clarifications:

let's get EQuip documented for everyone that existed prior to the current process date

Who is "everyone"? Everyone who has SOCKS access?

For those folks many of them have actually been adjudicated already and the EQip submission is not really relevant anymore

By "adjudicated" you mean that they not only have submitted their EQIP, but it's been processed (and "approved" as well)? Is there any way to determine this? Can we assume you have the ability to confirm if someone has been adjudicated?

Proposed way forward:

If you need a list of users who have SOCKS access, we can provide that for you. We can identify the users who have gotten access since March 24 (proof of EQIP in the request form). If you can confirm the status of the "legacy" (pre-March 24) SOCKS users and comment on their SOCKS access request, then that would basically put everyone who has socks access on an allow-list of sorts. This would give us a paper trail, which is a requirement (correct me if I'm wrong on that). After this is done, everyone requesting ArgoCD terminal access going forward would not need to share their EQIP screenshot because they've already passed through the SOCKS gate. We'd be using SOCKS access request as an implicit ticket to get Argo terminal access. How does this sound?

[alyssagallion]

@ryan-mcneil Do we need to do anything else here or are there any updates? Saw it was still on our board and want to make sure it still should be.