All / deny lists for publishing

[WillBeebe]

For apps and bundles

• build -> builds in same directory
• publish -> builds, then copies to a tmp dir, tars the dir and sends to s3

During the copy, all files / dirs at the same level of the massdriver.yaml are put through an allowlist. We only copy files or folders in this allowlist.

Once copying one of these allowed folders, we explicitly ignore some files and do some other checks on things like filesize.

Max single file size: 1MB Max entire bundle size: 10MB

[WillBeebe]

I locally ran this against GKE directing output to a _build directory so I could inspect it.

ls -al core-services/
total 88
drwxr-xr-x  13 wbeebe  staff   416 Jul 14 11:39 .
drwxr-xr-x  18 wbeebe  staff   576 Aug 19 22:22 ..
drwxr-xr-x   4 wbeebe  staff   128 Jun 11 11:29 .terraform
-rw-r--r--   1 wbeebe  staff  5682 Jun 11 11:29 .terraform.lock.hcl
-rw-r--r--@  1 wbeebe  staff  1285 Jul  8 12:51 _artifacts.tf
-rw-r--r--@  1 wbeebe  staff   157 Aug 21 13:00 _connections_variables.tf.json
-rw-r--r--@  1 wbeebe  staff    89 Aug 21 13:00 _md_variables.tf.json
-rw-r--r--@  1 wbeebe  staff   359 Aug 21 13:00 _params_variables.tf.json
-rw-r--r--@  1 wbeebe  staff  1491 Jul  8 12:51 _providers.tf
-rw-r--r--   1 wbeebe  staff  3181 Jun 17 17:39 core_services.tf
-rw-r--r--   1 wbeebe  staff   400 Jun  9 21:25 locals.tf
-rw-r--r--   1 wbeebe  staff   369 Jun 17 17:39 observability.tf
-rw-r--r--   1 wbeebe  staff   943 Jun  9 21:25 service_account.tf

ls _build/
core-services           massdriver.yaml         schema-connections.json schema-ui.json
custom-resources        schema-artifacts.json   schema-params.json      src

ls _build/core-services/
_artifacts.tf      _providers.tf      core_services.tf   locals.tf          observability.tf   service_account.tf

[coryodaniel]

I commented those out locally and the ~build~ PUBLISH worked.

[WillBeebe]

I commented those out locally and the ~build~ PUBLISH worked.

I lost my way a bit in the allow / ignore and order of operations but 🤦 duh we need those! I commented them out w/ a danger note for future devs, and added a test.

[coryodaniel]

If anything we should add a list of files and fail if they are missing.

^ lets add a ticket to verify required files are present in any publish operation. I think we can merge this as is

[WillBeebe]

Ticket here and I updated the file in the PR. 💪

[coryodaniel]

PS, ive packaged w/ this branch like 10 times today and have had no issues.