Closed lbeaufort closed 1 day ago
Blocked by #1925
I'm putting this back to Sprint Backlog. I'm not getting anywhere with this. I'm going to work on some other stuff and come back to this, but if someone else wants to take a crack at it, feel free.
What I've discovered/tried:
I'm starting to suspect it has something to do with cypress and how it interacts with docker in this instance, so maybe there's some configuration there we can change.
Also, I'd like to understand how we're using docker.io/jwilder/dockerize and if that's a factor.
After doing a little digging and experimentation, we're using dockerize for the -wait
and -timeout
commands. We use these commands to ensure that the API docker container has fully started before starting e2e testing. You can find the documentation for dockerize here
Elaine Krauss commented: I also have to put this back into Sprint Backlog since I’ve been unable to crack this either.
Some things that I’ve learned:
/authenticate
}} endpoint. For the sake of documentation, the error received is {{Error: connect ECONNREFUSED ::1:8080}}. See the first included screenshot for more.If there’s one area that I think would be a good opportunity for further investigation, it would be to somehow gain access to and investigate the fecfile-web-api docker container’s NGINX and network logs. -The error that Cypress is showing- -suggests- -that the request is being rejected, and that should show in the logs somewhere.- Edit: I double-checked this, and Cypress gives the same error even when it makes a request to a nonsensical port (e.g, localhost:1001). Still, the point stands that this seems to be a problem of networking, and looking at the docker containers' logs might give us further insight.
!image-20240724-151038.png|width=622,height=423,alt="image-20240724-151038.png"!
!image-20240724-154204.png|width=945,height=749,alt="image-20240724-154204.png"!
David Heitzer commented: mostly passing tests https://app.circleci.com/pipelines/github/fecgov/fecfile-web-app/6150/workflows/1cb3f9e0-7483-4f59-bb5e-8e9077f35a66/jobs/21189
David Heitzer commented: It looks like [+this change+|https://github.com/moby/moby/pull/47062] in docker version 26 is causing IPv6 to be enabled in our containers and the /etc/hosts file to be updated accordingly. According to [+their release notes+|https://docs.docker.com/engine/release-notes/26.0/#bug-fixes-and-enhancements], this can be disabled if the host stack doesn't support it (although it is supposed to happen automatically).
I suspect that something in CircleCI's {{setup_remote_docker}} configuration doesn't support this. I opened a CircleCI issue since this is breaking container-to-container networking using {{--network container:fecfile-api}}. Using the {{net.ipv6.conf.all.disable_ipv6=1}} disables it and gets our tests running again.
{noformat}Thank you for contacting CircleCI Support. Your ticket reference ID is: 153108{noformat}
Todd Lees commented: Passes CR moving to QA
Shelly Wise commented: QA review verified e2e test running and passing successfully.
!image-20240808-151739.png|width=1825,height=1029,alt="image-20240808-151739.png"!
QA Review Completed. Moved to Stage Ready.
Sprint accepted by Paul Clark during sprint review on 8/20/2024.
Business Reason As a developer, I will be able to run end-to-end tests with the latest docker engine versions so that I can keep the underlying technology up to date.
QA Notes
See successful E2E tests: [https://app.circleci.com/pipelines/github/fecgov/fecfile-web-app/6199/workflows/aecaf364-6ea0-41d7-aee0-163f26a0ae5c/jobs/21335|https://app.circleci.com/pipelines/github/fecgov/fecfile-web-app/6199/workflows/aecaf364-6ea0-41d7-aee0-163f26a0ae5c/jobs/21335]
!image-20240808-131844.png|width=1077,height=142,alt="image-20240808-131844.png"! See image in Jira
!image-20240808-131953.png|width=1031,height=603,alt="image-20240808-131953.png"!
DEV Notes
On May 2, the e2e tests passed for the
release/sprint-41
branch. When I re-ran them today (5/15), they failed for the same commit: https://app.circleci.com/pipelines/github/fecgov/fecfile-web-app?branch=release%2Fsprint-41. One difference between the failing and passing tests:On the
Setup a remote Docker engine
step: PassingFailing
Pinning the version to 24 in the Circle config resulted in passing e2e tests. The best guess I have so far is that the Docker API had a breaking change: https://docs.docker.com/engine/api/version-history/#v145-api-changes. Hotfix PRs in progress,
Also, I'd like to understand how we're using docker.io/jwilder/dockerize and if that's a factor.
Originally posted by @lbeaufort in https://github.com/fecgov/fecfile-web-app/issues/1928#issuecomment-2113503624
NOTE: To speed up debugging, try limiting the number of e2e tests to one. Also, check for latest release version of Docker 26
QA Notes
Passes QA if merge into develop runs successfully in nightly run.
Design
null
FECFILE-390