Spike: Prep for Vets-API to Lighthouse v1 updates

[mmiddaugh]

Description

The Facilities team is currently responsible for maintaining Vets API which is used by the Facility Locator and by other teams in the VA.gov ecosystem (such as VA Online Scheduling). To enable v1 in the Facility Locator and for us by other teams, we need to update Vets API.

Details in SPIKE: Become familiar with the facility locator data architecture #12990 may be helpful for reference.

V1 documentation

https://developer.va.gov/explore/api/va-facilities/docs?version=current Release notes: https://developer.va.gov/explore/api/va-facilities/release-notes

Resources

Slack thread with 10-10CG team who are consumers

ACs

• [x] Review #12990
• [x] Consumers of the vets-api are identified and noted in the comments of this ticket. If further exploration is needed beyond current sprint, work with PM to stub a new ticket.
• [x] Impact on consumers are documented in the comments of this ticket.
• [x] Subsequent tickets are created for change management with PM - #16778
• [x] Subsequent tickets are created for access to v1 or PM is tagged to create tickets

[xiongjaneg]

vets-api can surface both apis at the same time, we can't just point to the new one because not everyone will be ready right away

need namespacing

[xiongjaneg]

By default on vets-api, queries go to v1 of vets-api. Current assumption is that vets-api v1 goes to Lighthouse v0.

[xiongjaneg]

Notes from daily scrum: Who are our consumers of the vets-api then? We could check in vets-website to confirm who is calling those vets-api endpoints, in order to know who’s using it?

[xiongjaneg]

Steve: If we are namespacing the facilities to V1 then we don't have to worry about identifying consumers. Both should coexist for a while.

[xiongjaneg]

@xiongjaneg This is a system change that requires research prior to engineering work. Change this to a spike.

[xiongjaneg]

Change management has to start in advance of sunset.

[xiongjaneg]

We didn't version our vets-api rout currently. So we don't have an easy way to just change a version number in our rout and direct that to the next version of Lighthouse. We need to create something moving forward that can differentiate between version 1 and version 2 going forward (so we don't end up in the same spot we are now).

[xiongjaneg]

In 12/20 Lighthouse discussion, the PII on vets-website issue came up. Current status:

They say the PII issue has been resolved, and have steps in this thread for how to deal with your locals: https://dsva.slack.com/archives/CE4304QPK/p1702570927613969?thread_ts=1702568834.825019&cid=CE4304QPK (I was thinking about some of the notes in the epic body here, but nothing substantive is there for this discussion: https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14507)

[jilladams]

@xiongjaneg looking over the VBA / Police / V1 FE Q1 question things, it seems to make sense to get this spike into Sprint 101 if we can, just so we can better understand the scope of what all will be left to do to get this out the door?

[xiongjaneg]

@jilladams Thanks, I'll refine with Eli next week.

[eselkin]

Currently most apps version themselves via module in vets-api. e.g. the check_in module has v0,v1,v2. The pieces themselves, such as serializers are in a directory for the module version and the class is defined within a two level module (CheckIn -> V2).

We currently have a v1 only but have not made 2 nested levels of module, we only have the module FacilitiesApi. We identify the class with a namespace of V1, but it's not so cleanly done.

Basically, what this means is we just made it less clear how the modules will be separated. Not a big deal.

We will, in all likelihood, have to make a V2 set of directories, V2 class prefixed components (since we started that way), and expand the routes.rb to differentiate the requests based on version. I say V2 because we already used V1 instead of starting from 0 and matching the lighthouse number.

The hardest thing will be getting the configuration right, since I believe the serializers and that models don't really change in the move to Lighthouse V1, but I'd need some confirmation on that.

We'll probably need to have settings updated if lighthouse v0 and v1 URLs are different, unless they are only different by a path. Not sure if there is a path in the config in production (I think @maxx1128 can answer this). Does anyone know the lighthouse v0 vs v1 url structure differences? @swirtSJW ?

[jilladams]

@maxx1128 @swirtSJW some questions for you in comment above, that I think were added after initial post so notifications may not have been sent.

@eselkin a question for you: long-term, it would be nice to align with the Lighthouse versions to prevent future confusion, but I understand we can just change what V1 means to api consumers here. Is there potentially a way to abstract it somehow, so our version numbers align with Lighthouse for clarity, but consumers are still going from V1 to V2? Or is that more work that is useful / would make it less clean in the code, in general? (I could see the latter being true, but for due diligence figured it's worth asking.)

[maxx1128]

@eselkin I'm not as familiar with the Lighthouse URLs and how they connect to the API, but I've been looking at the code and it looks like there's no Production-specific URL. lib/lighthouse/facilities/configuration.rb references the Lighthouse URL from the settings at config/settings.yml. That file also gets uploaded to Production, not with any environment-specific variables, so the URL would be the same across environments.

It also looks like the version differences are specified after the root of the URL (like in lib/lighthouse/facilities/client.rb:22), reinforcing that they're only different by the path.

[eselkin]

@jilladams @xiongjaneg How should we send out a request to ask who is using vets-api /facilities_api/v1/va or /facilities_api/v1/ccp/*?

I'm trying on datadog to see the origins of requests to those endpoints, but it's a little complicated.

[jilladams]

This might be a good question for the backend COP channel in DSVA. They support everyone building on vets-api, so I'm sure someone in there has had to do something similar?

[eselkin]

This is my current work in identifying logs with unique referrers. Logs search in Datadog

source:vets-api @payload.controller:"FacilitiesApi::V1::VAController" -status:(error OR warn) @http.status_code:200 -@http.referer:*staging* 

group by felds Show count all logs by Referrer

Unfortunately we get duplicates like pre-post login for secure-messages (top 2 users of 1.34 million requests in the last month 111k, vs 22k for 3rd find-locations) This was for the Facilities controller not our FacilitiesApi VA/CCP controllers

[eselkin]

In the past month except for find-locations I get 1073 unspecified requests referred from https://www.va.gov/ So, who knows which team is sending maybe some javascript requests from a page and not adding a referrer. There's no information on the individual logs for the requests other than the bounding boxes for the requests.

See: https://vagov.ddog-gov.com/logs?saved-view-id=1540

[xiongjaneg]

Posted in Product channel

[jilladams]

From scrum:

• We will also put out an ask in Team of Teams next week re; consumers of the API.
• So far, all the references Eli found in vets-website are from Facilities team apps, as consumers of our own API.
• Jane has cut the follow up ticket re: CM during API rollout

[jilladams]

@eselkin getting ready for Tuesday end of sprint - it seems likely this could close, but wasn't sure if you feel there's more work to do on the SPIKE end, or if we are really just waiting to hear about other possible API consumers at this point?

I also wondered: for the requests that Facilities team as a consumer is making without a "referrer" I'm guessing we need a ticket to follow up and clean those up to add referrer?

[eselkin]

There's nothing more I can do to get information about potential users. We have a pretty good understanding of the module structure of the API ruby and can update to v2 with v1 Lighthouse without too much difficulty other than the numbers. There's no real way to make a module change except to change the v1 to v2 considering the overall structure of other version changes in the project as a whole.

As far as the referrer situation, I'm not sure JS respects the referrer when making direct requests to the API from script.

Making a V2 will not impact the V1 users. We should make V2 as soon as possible so that 1010 can start testing with it so when EOL of v0 of lighthouse comes around we can remove access to v1.

So, we can close.

[jilladams]

Announced the ask for any other API consumers to come forward today in ToT, for coverage.

@xiongjaneg do you feel ok with the findings here / closing ticket?

[xiongjaneg]

My ask in Product channel produced no other results than 1010 team and they're not in production yet.

[jilladams]

Ok - and just confirming before I close bc there's one unchecked AC remaining: follow up tickets that are necessary for next steps are created?

[eselkin]

Is VAOS team on that 1010 spectrum of people or are they completely different? Yesterday we found one use of staging /ccp. However they don't seem to be using /va so it might be that they are untouched by lighthouse v1.

[xiongjaneg]

@eselkin That VAOS team, according to their Github page is the My HealtheVet team with Jeff Roof as the PM. They were the ones who responded in the Product channel that they don't use our endpoint. Thanks for double-checking!