artsy / README

:wave: - The documentation for being an Artsy Engineer
Creative Commons Attribution 4.0 International
1.1k stars 120 forks source link

doc: update preferred S3 path for shared configuration #530

Closed joeyAghion closed 10 months ago

joeyAghion commented 10 months ago

Back in https://github.com/artsy/README/pull/368, I proposed a convention for storing shared configuration for development environments in S3. That convention has been working, but I regret choosing the particular path (artsy-citadel/dev/.env.<project>) for a few reasons:

Instead, I think we should source shared config from paths like artsy-citadel/<project>/.env.shared. Projects that depend on multiple files have already started using that pattern, and moving the existing (<40) files should be easy, like:

aws s3 cp s3://artsy-citadel/dev/.env.admin_metadata s3://artsy-citadel/admin_metadata/.env.shared

Then each project's setup script would need an update to pull from the new location (also easy, but tedious). Finally, the files under dev/* can be cleaned up to minimize confusion.

I'll start this process, but wanted to solicit feedback since it's a slight change to our playbook.

joeyAghion commented 10 months ago

I actually did a similar search, but wasn't sure about this one. Do you know why the gravity IAM profile needs access to the development configuration?

mc-jones commented 10 months ago

I actually did a similar search, but wasn't sure about this one. Do you know why the gravity IAM profile needs access to the development configuration?

Tracked down to https://github.com/artsy/infrastructure/pull/464. I think it may have been needed for a jenkins job.

joeyAghion commented 10 months ago

Since k8s crons and Batch jobs get their config in other ways these days, I'm inclined to drop that Allow block. Let me know if you think otherwise.