Less than two MFA types available on a white screen. No dashboards. Fresh aws acc. Fresh deployment (terraform)

[linkrage]

🐛 Summary

What's wrong? Please be specific.

When I login with my cognito newly created user I see "Less than two MFA types available" on a white screen.

No dashboards etc.

This happens right after entering the correct auth code from the authenticator mobile app.

Please see the screenshots for details and let me know what additional debug info I should bring in (and possibly how to get that for you).

To reproduce

Using the Terraform scripts I deployed Crossfeed (modified prod vars/confs) on a fresh/empty aws account with the following ssms set to a random string as they were required by the Terraform scripts:

/crossfeed/prod/DATABASE_USER
/crossfeed/prod/DATABASE_PASSWORD
/crossfeed/prod/APP_JWT_SECRET
/crossfeed/prod/REACT_APP_TERMS_VERSION
/crossfeed/prod/CENSYS_API_ID
/crossfeed/prod/CENSYS_API_SECRET
/crossfeed/prod/SHODAN_API_KEY
/crossfeed/prod/HIBP_API_KEY
/crossfeed/prod/LG_API_KEY
/crossfeed/prod/LG_WORKSPACE_NAME

And these ssms were set correctly (proper keys & user-agent string) as in the docs:

/crossfeed/prod/WORKER_SIGNATURE_PRIVATE_KEY
/crossfeed/prod/WORKER_SIGNATURE_PUBLIC_KEY
/crossfeed/prod/WORKER_USER_AGENT

Terraform deployed successfully without any erorrs etc. I've tested the deployment on a different fresh aws account before the final no-error deployment mentioned here.

This ssm was manually corrected:

/crossfeed/prod/DOMAIN

Steps to reproduce the behavior:

What am I missing? Everything was done according to the docs and yet you can see what bothers me in the screenshots.

Expected behavior

What did you expect to happen that didn't?

Crossfeed to be working properly when installed via the Terraform scripts on a brand new/fresh aws account.

Any helpful log output or screenshots

Paste the results here:

git status:

    modified:   backend/env.yml
    modified:   backend/package.json
    modified:   backend/tools/deploy-worker.sh
    modified:   infrastructure/Makefile
    modified:   infrastructure/main.tf
    modified:   infrastructure/prod.config
    modified:   infrastructure/prod.tfvars
    modified:   infrastructure/users.tf 

git diff backend/env.yml backend/package.json backend/tools/deploy-worker.sh infrastructure/Makefile infrastructure/main.tf infrastructure/prod.config infrastructure/prod.tfvars infrastructure/users.tf

diff --git a/backend/env.yml b/backend/env.yml
index 6ee72eb2..d2f8def1 100644
--- a/backend/env.yml
+++ b/backend/env.yml
@@ -48,21 +48,21 @@ prod:
   FARGATE_SUBNET_ID: ${ssm:/crossfeed/prod/WORKER_SUBNET_ID}
   FARGATE_MAX_CONCURRENCY: 300
   SCHEDULER_ORGS_PER_SCANTASK: 50
-  FARGATE_CLUSTER_NAME: 'crossfeed-prod-worker'
-  FARGATE_TASK_DEFINITION_NAME: 'crossfeed-prod-worker'
-  FARGATE_LOG_GROUP_NAME: 'crossfeed-prod-worker'
-  CROSSFEED_SUPPORT_EMAIL_SENDER: 'noreply@cr'
-  CROSSFEED_SUPPORT_EMAIL_REPLYTO: 'vulnerability@ci'
-  FRONTEND_DOMAIN: 'https://crossfeed.cyber.dhs.gov'
+  FARGATE_CLUSTER_NAME: 'crossfeed-prod-worker-4444'
+  FARGATE_TASK_DEFINITION_NAME: 'crossfeed-prod-worker-4444'
+  FARGATE_LOG_GROUP_NAME: 'crossfeed-prod-worker-4444'
+  CROSSFEED_SUPPORT_EMAIL_SENDER: 'real-email@example.com'
+  CROSSFEED_SUPPORT_EMAIL_REPLYTO: 'real-email@example.com'
+  FRONTEND_DOMAIN: 'https://example.com'
   SLS_LAMBDA_PREFIX: '${self:service}-${self:provider.stage}'
   USE_COGNITO: 1
-  REACT_APP_USER_POOL_ID: us-east-1_MZgKoBmkN
+  REACT_APP_USER_POOL_ID: us-east-1_7S161LlK0
   WORKER_USER_AGENT: ${ssm:/crossfeed/prod/WORKER_USER_AGENT}
   WORKER_SIGNATURE_PUBLIC_KEY: ${ssm:/crossfeed/prod/WORKER_SIGNATURE_PUBLIC_KEY}
   ELASTICSEARCH_ENDPOINT: ${ssm:/crossfeed/prod/ELASTICSEARCH_ENDPOINT}
   REACT_APP_TERMS_VERSION: ${ssm:/crossfeed/prod/REACT_APP_TERMS_VERSION}
   MATOMO_URL: http://matomo.crossfeed.local
-  EXPORT_BUCKET_NAME: cisa-crossfeed-prod-exports
+  EXPORT_BUCKET_NAME: cisa-crossfeed-prod-exports-4444

 staging-vpc:
   securityGroupIds:
diff --git a/backend/package.json b/backend/package.json
index 8703d1fc..cdc49154 100644
--- a/backend/package.json
+++ b/backend/package.json
@@ -83,7 +83,7 @@
     "codegen": "ts-node src/tools/generate-censys-types.ts",
     "build-worker": "sh ./tools/build-worker.sh",
     "deploy-worker-staging": "./tools/deploy-worker.sh",
-    "deploy-worker-prod": "./tools/deploy-worker.sh crossfeed-prod-worker",
+    "deploy-worker-prod": "./tools/deploy-worker.sh crossfeed-prod-worker-4444",
     "syncdb": "docker-compose exec -T backend npx ts-node src/tools/run-syncdb.ts"
   },
   "resolutions": {
diff --git a/backend/tools/deploy-worker.sh b/backend/tools/deploy-worker.sh
index 03689696..f0f4c566 100755
--- a/backend/tools/deploy-worker.sh
+++ b/backend/tools/deploy-worker.sh
@@ -6,9 +6,9 @@

 set -e

-AWS_ECR_DOMAIN=957221700844.dkr.ecr.us-east-1.amazonaws.com
+AWS_ECR_DOMAIN=111111111111.dkr.ecr.us-east-1.amazonaws.com

-WORKER_TAG=${1:-crossfeed-staging-worker}
+WORKER_TAG=crossfeed-prod-worker-4444

 ./tools/build-worker.sh
 aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin $AWS_ECR_DOMAIN
diff --git a/infrastructure/Makefile b/infrastructure/Makefile
index 435017d8..f04c7b12 100644
--- a/infrastructure/Makefile
+++ b/infrastructure/Makefile
@@ -1,10 +1,10 @@
 .PHONY: init plan apply

 init:
-   dotenv -- terraform init -backend-config=stage.config -input=false
+   dotenv -- terraform init -backend-config=prod.config -input=false
 plan:
-   dotenv -- terraform plan -var-file=stage.tfvars -out plan
+   dotenv -- terraform plan -var-file=prod.tfvars -out plan
 apply:
    dotenv -- terraform apply plan
 format:
-   terraform validate && terraform fmt -recursive
\ No newline at end of file
+   terraform validate && terraform fmt -recursive
diff --git a/infrastructure/main.tf b/infrastructure/main.tf
index 119c0625..b6ad1f45 100644
--- a/infrastructure/main.tf
+++ b/infrastructure/main.tf
@@ -5,7 +5,7 @@ data "aws_availability_zones" "available" {
 resource "aws_ssm_parameter" "prod_api_domain" {
   name      = "/crossfeed/prod/DOMAIN"
   type      = "String"
-  value     = "api.crossfeed.cyber.dhs.gov"
+  value     = "example.com"
   overwrite = true

   tags = {
@@ -16,7 +16,7 @@ resource "aws_ssm_parameter" "prod_api_domain" {
 resource "aws_ssm_parameter" "stage_api_domain" {
   name      = "/crossfeed/staging/DOMAIN"
   type      = "String"
-  value     = "api.staging.crossfeed.cyber.dhs.gov"
+  value     = "example.com"
   overwrite = true

   tags = {
diff --git a/infrastructure/prod.config b/infrastructure/prod.config
index 578a2db6..f59b7f28 100644
--- a/infrastructure/prod.config
+++ b/infrastructure/prod.config
@@ -3,6 +3,6 @@

 key="PROD/frontend-prod.tfstate"
-bucket="cisa-cd-crossfeed-terraform-state-prod"
+bucket="crossfeed-tf-4444"
 region="us-east-1"
 profile="default"
diff --git a/infrastructure/prod.tfvars b/infrastructure/prod.tfvars
index 956c8e97..d282c542 100644
--- a/infrastructure/prod.tfvars
+++ b/infrastructure/prod.tfvars
@@ -1,12 +1,12 @@
 aws_region                        = "us-east-1"
-project                           = "Crossfeed"
+project                           = "Crossfeed-4444"
 stage                             = "prod"
-frontend_domain                   = "crossfeed.cyber.dhs.gov"
-frontend_lambda_function          = "crossfeed-security-headers-prod"
-frontend_bucket                   = "crossfeed.cyber.dhs.gov"
-api_domain                        = "api.crossfeed.cyber.dhs.gov"
-frontend_cert_arn                 = "arn:aws:acm:us-east-1:957221700844:certificate/6c26ea05-4806-460a-a639-ee204ebde894"
-db_name                           = "crossfeed-prod-db2"
+frontend_domain                   = "example.com"
+frontend_lambda_function          = "crossfeed-security-headers-prod-4444"
+frontend_bucket                   = "example.com"
+api_domain                        = "example.com"
+frontend_cert_arn                 = "arn:aws:acm:us-east-1:111111111111:certificate/00001f71-2467-4590-0000-305755b30000"
+db_name                           = "crossfeed-prod-db2-4444"
 db_port                           = 5432
 db_table_name                     = "cfproddb"
 db_instance_class                 = "db.t3.xlarge"
@@ -28,27 +28,27 @@ ssm_shodan_api_key                = "/crossfeed/prod/SHODAN_API_KEY"
 ssm_hibp_api_key                  = "/crossfeed/prod/HIBP_API_KEY"
 ssm_lg_api_key                    = "/crossfeed/prod/LG_API_KEY"
 ssm_lg_workspace_name             = "/crossfeed/prod/LG_WORKSPACE_NAME"
-cloudfront_name                   = "Crossfeed Prod Frontend"
-db_group_name                     = "crossfeed-prod-db-group"
-worker_ecs_repository_name        = "crossfeed-prod-worker"
-worker_ecs_cluster_name           = "crossfeed-prod-worker"
-worker_ecs_task_definition_family = "crossfeed-prod-worker"
-worker_ecs_log_group_name         = "crossfeed-prod-worker"
-worker_ecs_role_name              = "crossfeed-prod-worker"
-logging_bucket_name               = "cisa-crossfeed-prod-logging"
-export_bucket_name                = "cisa-crossfeed-prod-exports"
-user_pool_name                    = "crossfeed-prod"
-user_pool_domain                  = "crossfeed"
+cloudfront_name                   = "Crossfeed Prod Frontend-333"
+db_group_name                     = "crossfeed-prod-db-group-4444"
+worker_ecs_repository_name        = "crossfeed-prod-worker-4444"
+worker_ecs_cluster_name           = "crossfeed-prod-worker-4444"
+worker_ecs_task_definition_family = "crossfeed-prod-worker-4444"
+worker_ecs_log_group_name         = "crossfeed-prod-worker-4444"
+worker_ecs_role_name              = "crossfeed-prod-worker-4444"
+logging_bucket_name               = "cisa-crossfeed-prod-logging-4444"
+export_bucket_name                = "cisa-crossfeed-prod-exports-4444"
+user_pool_name                    = "crossfeed-prod-4444"
+user_pool_domain                  = "crossfeed-4444"
 ssm_user_pool_id                  = "/crossfeed/prod/USER_POOL_ID"
 ssm_user_pool_client_id           = "/crossfeed/prod/USER_POOL_CLIENT_ID"
-ses_support_email_sender          = "noreply@cr"
-ses_support_email_replyto         = "vulnerability@ci"
-matomo_ecs_cluster_name           = "crossfeed-matomo-prod"
-matomo_ecs_task_definition_family = "crossfeed-matomo-prod"
-matomo_ecs_log_group_name         = "crossfeed-matomo-prod"
-matomo_db_name                    = "crossfeed-matomo-prod"
+ses_support_email_sender          = "real-email@example.com"
+ses_support_email_replyto         = "real-email@example.com"
+matomo_ecs_cluster_name           = "crossfeed-matomo-prod-4444"
+matomo_ecs_task_definition_family = "crossfeed-matomo-prod-4444"
+matomo_ecs_log_group_name         = "crossfeed-matomo-prod-4444"
+matomo_db_name                    = "crossfeed-matomo-prod-4444"
 matomo_db_instance_class          = "db.t3.micro"
-matomo_ecs_role_name              = "crossfeed-matomo-prod"
+matomo_ecs_role_name              = "crossfeed-matomo-prod-4444"
 es_instance_type                  = "m4.large.elasticsearch"
 es_instance_count                 = 2
 es_instance_volume_size           = 512
diff --git a/infrastructure/users.tf b/infrastructure/users.tf
index 69b9e2a9..6e799e5c 100644
--- a/infrastructure/users.tf
+++ b/infrastructure/users.tf
@@ -64,4 +64,4 @@ resource "aws_ssm_parameter" "user_pool_client_id" {
   tags = {
     Project = var.project
   }
-}
\ No newline at end of file
+}

Add any screenshots of the problem here.

[epicfaace]

I see you set /crossfeed/prod/DOMAIN in terraform to example.com? It should be set to the value of a custom domain that your API is hosted on.

[linkrage]

@epicfaace it is actually set to the custom domain - it's the same domain I use to login to this newly installed instance of Crossfeed. Everywhere you see example.com, some IDs or anything similar is changed before pasting it here just for some privacy etc.

Is there anything I can check etc?

[epicfaace]

Can you try manually navigating to the / URL in your API endpoint? You should get a 200 response with a blank body. Let me know what you get. (this endpoint: https://github.com/cisagov/crossfeed/blob/a70e552308996d7d396c101e331372122ab2fb0c/backend/src/api/app.ts#L62)

[linkrage]

@epicfaace I see what you mean - like it is on https://api.crossfeed.cyber.dhs.gov/

In my case however the output is the HTML generated by the frontend code (not the API) - exactly like it would be if you query https://crossfeed.cyber.dhs.gov/

So I've now changed the ssm "/crossfeed/prod/DOMAIN" from "example.com" to "api.example.com"

Now I think that the following commands did nothing during the deployment (and continue to do so) cd backend npx sls create_domain --stage=prod npx sls deploy --stage=prod

In AWS Lambda I keep having only one function - the crossfeed-security-headers-prod one.

[epicfaace]

Now I think that the following commands did nothing during the deployment (and continue to do so) cd backend npx sls create_domain --stage=prod npx sls deploy --stage=prod

What output do you get from these commands?

[linkrage]

@epicfaace thank you for your time helping me on all this!

The output is the same for both commands - please see the screenshots bellow.

[epicfaace]

Hmm, it looks like you might have another command installed that is running when you call sls instead of serverless framework?

You could try replacing "sls" with "serverless", for example, npx serverless create_domain --stage=prod

[linkrage]

@epicfaace thank you!

On macos sudo npm install -g serverless fixed both sls & serverless commands

Now I have this error: An error occurred: SchedulerLambdaFunction - Resource handler returned message: "Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of [50].

So I have to request a quota increate at https://console.aws.amazon.com/servicequotas/home/services/lambda/quotas/L-B99A9384

In your experience what number would you recommend for running just the Crossfeed project on a fresh aws account specifically?

[epicfaace]

I'm not sure, this is probably specific to your AWS account and what else you're running on it? Are you sure you're using a fresh AWS account (is it possible it was contaminated by previous deploy attempts)?

You could also alternatively just comment out this line (https://github.com/cisagov/crossfeed/blob/6f2582c8fb713aba396fe328ba2968d1161a75ab/backend/src/tasks/functions.yml#L6) and it should still work.

[linkrage]

Thank you. Yes I'm sure it's a fresh account because I intentionally contaminated a previous AWS account with Crossfeed installs so I can do a much cleaner install on the new account :)

I've requested the quota increase to 200 - that solved that particular problem. API service was finally deployed and now it returns the blank page you @epicfaace asked about earlier etc. :)

However that did not solve the initial error message from the Title/screenshot but I'll investigate it later and post updates here. Thank you for your time!

[linkrage]

@epicfaace can you please help me debug these specific 502 errors?

I confirm that I'm using revision 6f2582c8fb713aba396fe328ba2968d1161a75ab If you think issues might be related to the revision please let me know about a working one if possible.

[epicfaace]

Hi @linkrage , unfortunately it's hard to tell what's going on without more logs / context. Can you try:

• running from the latest master branch
• checking the logs of API Gateway and lambda functions in the AWS console to see what's going wrong?

[linkrage]

@epicfaace I just deployed the latest master - no change in behavior at all on my end.

Here are some logs on the topic:

/aws/lambda/crossfeed-prod-scheduler:

2022-02-22T18:37:05.730+02:00   START RequestId: 0409b0d3-f0de-4545-90c9-365d03fe190b Version: $LATEST
2022-02-22T18:37:05.735+02:00   2022-02-22T16:37:05.735Z 0409b0d3-f0de-4545-90c9-365d03fe190b INFO Running scheduler...
2022-02-22T18:37:05.750+02:00   2022-02-22T16:37:05.749Z 0409b0d3-f0de-4545-90c9-365d03fe190b ERROR Invoke Error {"errorType":"QueryFailedError","errorMessage":"relation \"scan\" does not exist","code":"42P01","query":"SELECT \"Scan\".\"id\" AS \"Scan_id\", \"Scan\".\"createdAt\" AS \"Scan_createdAt\", \"Scan\".\"updatedAt\" AS \"Scan_updatedAt\", \"Scan\".\"name\" AS \"Scan_name\", \"Scan\".\"arguments\" AS \"Scan_arguments\", \"Scan\".\"frequency\" AS \"Scan_frequency\", \"Scan\".\"lastRun\" AS \"Scan_lastRun\", \"Scan\".\"isGranular\" AS \"Scan_isGranular\", \"Scan\".\"isUserModifiable\" AS \"Scan_isUserModifiable\", \"Scan\".\"isSingleScan\" AS \"Scan_isSingleScan\", \"Scan\".\"manualRunPending\" AS \"Scan_manualRunPending\", \"Scan\".\"createdById\" AS \"Scan_createdById\", \"Scan__organizations\".\"id\" AS \"Scan__organizations_id\", \"Scan__organizations\".\"createdAt\" AS \"Scan__organizations_createdAt\", \"Scan__organizations\".\"updatedAt\" AS \"Scan__organizations_updatedAt\", \"Scan__organizations\".\"name\" AS \"Scan__organizations_name\", \"Scan__organizations\".\"rootDomains\" AS \"Scan__organizations_rootDomains\", \"Scan__organizations\".\"ipBlocks\" AS \"Scan__organizations_ipBlocks\", \"Scan__organizations\".\"isPassive\" AS \"Scan__organizations_isPassive\", \"Scan__organizations\".\"parentId\" AS \"Scan__organizations_parentId\", \"Scan__organizations\".\"createdById\" AS \"Scan__organizations_createdById\", \"Scan__tags\".\"id\" AS \"Scan__tags_id\", \"Scan__tags\".\"createdAt\" AS \"Scan__tags_createdAt\", \"Scan__tags\".\"updatedAt\" AS \"Scan__tags_updatedAt\", \"Scan__tags\".\"name\" AS \"Scan__tags_name\", \"Scan__tags__organizations\".\"id\" AS \"Scan__tags__organizations_id\", \"Scan__tags__organizations\".\"createdAt\" AS \"Scan__tags__organizations_createdAt\", \"Scan__tags__organizations\".\"updatedAt\" AS \"Scan__tags__organizations_updatedAt\", \"Scan__tags__organizations\".\"name\" AS \"Scan__tags__organizations_name\", \"Scan__tags__organizations\".\"rootDomains\" AS \"Scan__tags__organizations_rootDomains\", \"Scan__tags__organizations\".\"ipBlocks\" AS \"Scan__tags__organizations_ipBlocks\", \"Scan__tags__organizations\".\"isPassive\" AS \"Scan__tags__organizations_isPassive\", \"Scan__tags__organizations\".\"parentId\" AS \"Scan__tags__organizations_parentId\", \"Scan__tags__organizations\".\"createdById\" AS \"Scan__tags__organizations_createdById\" FROM \"scan\" \"Scan\" LEFT JOIN \"scan_organizations_organization\" \"Scan_Scan__organizations\" ON \"Scan_Scan__organizations\".\"scanId\"=\"Scan\".\"id\" LEFT JOIN \"organization\" \"Scan__organizations\" ON \"Scan__organizations\".\"id\"=\"Scan_Scan__organizations\".\"organizationId\" LEFT JOIN \"scan_tags_organization_tag\" \"Scan_Scan__tags\" ON \"Scan_Scan__tags\".\"scanId\"=\"Scan\".\"id\" LEFT JOIN \"organization_tag\" \"Scan__tags\" ON \"Scan__tags\".\"id\"=\"Scan_Scan__tags\".\"organizationTagId\" LEFT JOIN \"organization_tag_organizations_organization\" \"Scan__tags_Scan__tags__organizations\" ON \"Scan__tags_Scan__tags__organizations\".\"organizationTagId\"=\"Scan__tags\".\"id\" LEFT JOIN \"organization\" \"Scan__tags__organizations\" ON \"Scan__tags__organizations\".\"id\"=\"Scan__tags_Scan__tags__organizations\".\"organizationId\"","parameters":[],"driverError":{"errorType":"error","errorMessage":"relation \"scan\" does not exist","code":"42P01","length":105,"name":"error","severity":"ERROR","position":"2016","file":"parse_relation.c","line":"1376","routine":"parserOpenTable","stack":["error: relation \"scan\" does not exist"," at Parser.parseErrorMessage (/var/task/src/tasks/scheduler.js:142675:98)"," at Parser.handlePacket (/var/task/src/tasks/scheduler.js:142514:29)"," at Parser.parse (/var/task/src/tasks/scheduler.js:142427:38)"," at Socket.<anonymous> (/var/task/src/tasks/scheduler.js:41488:42)"," at Socket.emit (events.js:400:28)"," at Socket.emit (domain.js:475:12)"," at addChunk (internal/streams/readable.js:293:12)"," at readableAddChunk (internal/streams/readable.js:267:9)"," at Socket.Readable.push (internal/streams/readable.js:206:10)"," at TCP.onStreamRead (internal/stream_base_commons.js:188:23)"]},"length":105,"severity":"ERROR","position":"2016","file":"parse_relation.c","line":"1376","routine":"parserOpenTable","stack":["QueryFailedError: relation \"scan\" does not exist"," at QueryFailedError.TypeORMError [as constructor] (/var/task/src/tasks/scheduler.js:678:28)"," at new QueryFailedError (/var/task/src/tasks/scheduler.js:2908:28)"," at PostgresQueryRunner.<anonymous> (/var/task/src/tasks/scheduler.js:46423:31)"," at step (/var/task/src/tasks/scheduler.js:218:23)"," at Object.throw (/var/task/src/tasks/scheduler.js:199:53)"," at rejected (/var/task/src/tasks/scheduler.js:190:65)"," at processTicksAndRejections (internal/process/task_queues.js:95:5)"]}
2022-02-22T18:37:05.750+02:00   END RequestId: 0409b0d3-f0de-4545-90c9-365d03fe190b
2022-02-22T18:37:05.750+02:00   REPORT RequestId: 0409b0d3-f0de-4545-90c9-365d03fe190b Duration: 15.83 ms Billed Duration: 16 ms Memory Size: 4096 MB Max Memory Used: 177 MB

/aws/api-gateway/crossfeed-prod:

2022-02-22T19:06:15.938+02:00   requestId: 81cab78f-e316-4bb0-9429-cbbec0a284fa, ip: xxx.xxx.xxx.xxx, caller: -, user: -, requestTime: 22/Feb/2022:17:06:15 +0000, httpMethod: POST, resourcePath: /{any+}, status: 502, protocol: HTTP/1.1, responseLength: 36

/aws/lambda/crossfeed-prod-api:

2022-02-22T18:45:59.663+02:00   START RequestId: 685f109f-3a3c-44e0-a102-451090aace17 Version: $LATEST
2022-02-22T18:45:59.754+02:00   2022-02-22T16:45:59.753Z 685f109f-3a3c-44e0-a102-451090aace17 ERROR [HPM] Error occurred while trying to proxy request /matomo.js from api.example.com to http://matomo.crossfeed.local (ENOTFOUND) (https://nodejs.org/api/errors.html#errors_common_system_errors)
2022-02-22T18:45:59.756+02:00   END RequestId: 685f109f-3a3c-44e0-a102-451090aace17
2022-02-22T18:45:59.756+02:00   REPORT RequestId: 685f109f-3a3c-44e0-a102-451090aace17 Duration: 92.41 ms Billed Duration: 93 ms Memory Size: 1024 MB Max Memory Used: 183 MB

BTW since the beginning I have 2 ECS clusters: crossfeed-prod-worker - 0 Services crossfeed-matomo-prod - 1 Service

[epicfaace]

Oh -- you need to invoke the lambda function "crossfeed-prod-syncdb" through the AWS console in the AWS lambda tab. Can you try that first?

This function actually creates the database tables.

If that works, then if you don't mind updating the documentation, I'd really appreciate it!

-- Ashwin Ramaswami

On Tue, Feb 22, 2022 at 12:30 PM linkrage @.***> wrote:

@epicfaace https://github.com/epicfaace I just deployed the latest master - no change in behavior at all on my end.

Here are some logs on the topic:

/aws/lambda/crossfeed-prod-scheduler:

2022-02-22T18:37:05.730+02:00 START RequestId: 0409b0d3-f0de-4545-90c9-365d03fe190b Version: $LATEST 2022-02-22T18:37:05.735+02:00 2022-02-22T16:37:05.735Z 0409b0d3-f0de-4545-90c9-365d03fe190b INFO Running scheduler... 2022-02-22T18:37:05.750+02:00 2022-02-22T16:37:05.749Z 0409b0d3-f0de-4545-90c9-365d03fe190b ERROR Invoke Error {"errorType":"QueryFailedError","errorMessage":"relation \"scan\" does not exist","code":"42P01","query":"SELECT \"Scan\".\"id\" AS \"Scan_id\", \"Scan\".\"createdAt\" AS \"Scan_createdAt\", \"Scan\".\"updatedAt\" AS \"Scan_updatedAt\", \"Scan\".\"name\" AS \"Scan_name\", \"Scan\".\"arguments\" AS \"Scan_arguments\", \"Scan\".\"frequency\" AS \"Scan_frequency\", \"Scan\".\"lastRun\" AS \"Scan_lastRun\", \"Scan\".\"isGranular\" AS \"Scan_isGranular\", \"Scan\".\"isUserModifiable\" AS \"Scan_isUserModifiable\", \"Scan\".\"isSingleScan\" AS \"Scan_isSingleScan\", \"Scan\".\"manualRunPending\" AS \"Scan_manualRunPending\", \"Scan\".\"createdById\" AS \"Scan_createdById\", \"Scanorganizations\".\"id\" AS \"Scanorganizations_id\", \"Scanorganizations\".\"createdAt\" AS \"Scanorganizations_createdAt\", \"Scanorganizations\".\"updatedAt\" AS \"Scanorganizations_updatedAt\", \"Scanorganizations\".\"name\" AS \"Scanorganizations_name\", \"Scanorganizations\".\"rootDomains\" AS \"Scanorganizations_rootDomains\", \"Scanorganizations\".\"ipBlocks\" AS \"Scanorganizations_ipBlocks\", \"Scanorganizations\".\"isPassive\" AS \"Scanorganizations_isPassive\", \"Scanorganizations\".\"parentId\" AS \"Scanorganizations_parentId\", \"Scanorganizations\".\"createdById\" AS \"Scanorganizations_createdById\", \"Scantags\".\"id\" AS \"Scantags_id\", \"Scantags\".\"createdAt\" AS \"Scantags_createdAt\", \"Scantags\".\"updatedAt\" AS \"Scantags_updatedAt\", \"Scantags\".\"name\" AS \"Scantags_name\", \"Scantagsorganizations\".\"id\" AS \"Scantagsorganizations_id\", \"Scantagsorganizations\".\"createdAt\" AS \"Scantagsorganizations_createdAt\", \"Scantagsorganizations\".\"updatedAt\" AS \"Scantagsorganizations_updatedAt\", \"Scantagsorganizations\".\"name\" AS \"Scantagsorganizations_name\", \"Scantagsorganizations\".\"rootDomains\" AS \"Scantagsorganizations_rootDomains\", \"Scantagsorganizations\".\"ipBlocks\" AS \"Scantagsorganizations_ipBlocks\", \"Scantagsorganizations\".\"isPassive\" AS \"Scantagsorganizations_isPassive\", \"Scantagsorganizations\".\"parentId\" AS \"Scantagsorganizations_parentId\", \"Scantagsorganizations\".\"createdById\" AS \"Scantagsorganizations_createdById\" FROM \"scan\" \"Scan\" LEFT JOIN \"scan_organizations_organization\" \"Scan_Scanorganizations\" ON \"Scan_Scanorganizations\".\"scanId\"=\"Scan\".\"id\" LEFT JOIN \"organization\" \"Scanorganizations\" ON \"Scan__organizations\".\"id\"=\"Scan_Scanorganizations\".\"organizationId\" LEFT JOIN \"scan_tags_organization_tag\" \"Scan_Scantags\" ON \"Scan_Scan__tags\".\"scanId\"=\"Scan\".\"id\" LEFT JOIN \"organization_tag\" \"Scantags\" ON \"Scantags\".\"id\"=\"Scan_Scan__tags\".\"organizationTagId\" LEFT JOIN \"organization_tag_organizations_organization\" \"Scantags_Scantagsorganizations\" ON \"Scantags_Scantagsorganizations\".\"organizationTagId\"=\"Scantags\".\"id\" LEFT JOIN \"organization\" \"Scantagsorganizations\" ON \"Scantagsorganizations\".\"id\"=\"Scantags_Scantags__organizations\".\"organizationId\"","parameters":[],"driverError":{"errorType":"error","errorMessage":"relation \"scan\" does not exist","code":"42P01","length":105,"name":"error","severity":"ERROR","position":"2016","file":"parse_relation.c","line":"1376","routine":"parserOpenTable","stack":["error: relation \"scan\" does not exist"," at Parser.parseErrorMessage (/var/task/src/tasks/scheduler.js:142675:98)"," at Parser.handlePacket (/var/task/src/tasks/scheduler.js:142514:29)"," at Parser.parse (/var/task/src/tasks/scheduler.js:142427:38)"," at Socket. (/var/task/src/tasks/scheduler.js:41488:42)"," at Socket.emit (events.js:400:28)"," at Socket.emit (domain.js:475:12)"," at addChunk (internal/streams/readable.js:293:12)"," at readableAddChunk (internal/streams/readable.js:267:9)"," at Socket.Readable.push (internal/streams/readable.js:206:10)"," at TCP.onStreamRead (internal/stream_base_commons.js:188:23)"]},"length":105,"severity":"ERROR","position":"2016","file":"parse_relation.c","line":"1376","routine":"parserOpenTable","stack":["QueryFailedError: relation \"scan\" does not exist"," at QueryFailedError.TypeORMError [as constructor] (/var/task/src/tasks/scheduler.js:678:28)"," at new QueryFailedError (/var/task/src/tasks/scheduler.js:2908:28)"," at PostgresQueryRunner. (/var/task/src/tasks/scheduler.js:46423:31)"," at step (/var/task/src/tasks/scheduler.js:218:23)"," at Object.throw (/var/task/src/tasks/scheduler.js:199:53)"," at rejected (/var/task/src/tasks/scheduler.js:190:65)"," at processTicksAndRejections (internal/process/task_queues.js:95:5)"]} 2022-02-22T18:37:05.750+02:00 END RequestId: 0409b0d3-f0de-4545-90c9-365d03fe190b 2022-02-22T18:37:05.750+02:00 REPORT RequestId: 0409b0d3-f0de-4545-90c9-365d03fe190b Duration: 15.83 ms Billed Duration: 16 ms Memory Size: 4096 MB Max Memory Used: 177 MB

/aws/api-gateway/crossfeed-prod:

2022-02-22T19:06:15.938+02:00 requestId: 81cab78f-e316-4bb0-9429-cbbec0a284fa, ip: xxx.xxx.xxx.xxx, caller: -, user: -, requestTime: 22/Feb/2022:17:06:15 +0000, httpMethod: POST, resourcePath: /{any+}, status: 502, protocol: HTTP/1.1, responseLength: 36

/aws/lambda/crossfeed-prod-api:

2022-02-22T18:45:59.663+02:00 START RequestId: 685f109f-3a3c-44e0-a102-451090aace17 Version: $LATEST 2022-02-22T18:45:59.754+02:00 2022-02-22T16:45:59.753Z 685f109f-3a3c-44e0-a102-451090aace17 ERROR [HPM] Error occurred while trying to proxy request /matomo.js from api.example.com to http://matomo.crossfeed.local (ENOTFOUND) (https://nodejs.org/api/errors.html#errors_common_system_errors) 2022-02-22T18:45:59.756+02:00 END RequestId: 685f109f-3a3c-44e0-a102-451090aace17 2022-02-22T18:45:59.756+02:00 REPORT RequestId: 685f109f-3a3c-44e0-a102-451090aace17 Duration: 92.41 ms Billed Duration: 93 ms Memory Size: 1024 MB Max Memory Used: 183 MB

BTW since the beginning I have 2 ECS clusters: crossfeed-prod-worker - 0 Services crossfeed-matomo-prod - 1 Service

— Reply to this email directly, view it on GitHub https://github.com/cisagov/crossfeed/issues/1388#issuecomment-1048039606, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4MX6KBFL6IFQ5MMAV4TDU4PB3RANCNFSM5NT25EBQ . You are receiving this because you were mentioned.Message ID: @.***>

[linkrage]

@epicfaace

Thanks! That fixed all the mentioned errors so far!

What's the most elegant way of dealing with this part in "pages/AuthCreateAccount/AuthCreateAccount.tsx" ?

I've used stage=prod (etc.) for everything during deployment.

Request URL: https://api.example.com/organizations/public
Request Method: GET
Status Code: 403

[epicfaace]

You need to 1) make an account, then 2) call the crossfeed-prod-makeGlobalAdmin lambda function through the AWS console with the payload {email: "[your email address]"}, then 3) go back to Crossfeed and log in. This is just one-time setup required for the first global admin account you create on a deployed Crossfeed instance.

-- Ashwin Ramaswami

On Tue, Feb 22, 2022 at 1:08 PM linkrage @.***> wrote:

@epicfaace https://github.com/epicfaace

Thanks! That fixed all the mentioned errors so far!

What's the most elegant way of dealing with this part in "pages/AuthCreateAccount/AuthCreateAccount.tsx" ?

I've used stage=prod (etc.) for everything during deployment.

Request URL: https://api.example.com/organizations/public Request Method: GET Status Code: 403

[image: screenshot-1] https://user-images.githubusercontent.com/2017450/155191597-e23e6868-f40e-4dfd-b708-f64d2ec91844.png

— Reply to this email directly, view it on GitHub https://github.com/cisagov/crossfeed/issues/1388#issuecomment-1048072686, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4MX5YOXJKGOSX53TU373U4PGKDANCNFSM5NT25EBQ . You are receiving this because you were mentioned.Message ID: @.***>

[linkrage]

@epicfaace I managed to login after doing that.

Now I get 403 HTTP error when I try to do something useful - e.g. create an "organization" in Crossfeed, invite user etc. I see zero errors on the topic in the logs.

The API response is "User must accept terms of use"

Is there a better way to proceed with this other than editing the backend/src/api/app.ts ?

[epicfaace]

Try going to [frontend url]/terms to see if you can sign the terms?

Note that the TOU are really only specific to CISA's crossfeed instance, which we should probably remove if you're standing up your own instance.

[linkrage]

Thanks, this one did not work unfortunatelly.

Whitelisting the desired domains in backend/src/api/app.ts worked.

[epicfaace]

Okay, that's great! probably a bug that needs to be fixed in the future.