Create env files from templates in a change-preserving way

dhruvkb commented 4 weeks ago

Fixes

Fixes #2938 by @stacimc

Description

This PR updates the env recipe to make it "smarter". Instead of copying over the file and overwriting changes, the Python script now keeps changed env values and even extra env key-value pairs in the file.

Assume the following env.template file:

key_one=value_one
key_two=value_two

Assume the following .env file:

key_one=new_value_one
key_three=value_three

After just env the .env file would be as follows.

key_one=new_value_one
key_two=value_two

# Preserved variables
key_three=value_three

Testing Instructions

Make some changes to the catalog/.env file. You could change some values, delete some keys or add some new keys.
Run just env.
See that the changes are preserved, deleted keys are restored and custom extra keys are kept at the bottom of the file.
Confirm this against the printed logs.

Shortcomings

This approach does not preserve any extra comments in the .env file. Those will be deleted.

Checklist

[x] My pull request has a descriptive title (not a vague title likeUpdate index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow [best practices][best_practices].
[x] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[x] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no visible errors.
[ ] I ran the DAG documentation generator (ov just catalog/generate-docs for catalog PRs) or the media properties generator (ov just catalog/generate-docs media-props for the catalog or ov just api/generate-docs for the API) where applicable.

[best_practices]: https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project#_commit_guidelines

Developer Certificate of Origin

``` Developer Certificate of Origin Version 1.1 Copyright (C) 2004, 2006 The Linux Foundation and its contributors. 1 Letterman Drive Suite D4700 San Francisco, CA, 94129 Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it. (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. ```

AetherUnbound commented 4 weeks ago

This approach does not preserve any extra comments in the .env file. Those will be deleted.

Is there any way to preserve those comments without ballooning this code? 😅 I've often left stuff commented out (like variables that would reference production resources) for testing and it would be ideal to have those retained. Perhaps the script could make a .env.backup-YYYY-MM-DD file as well which is added to the .gitignore, but will preserve the old file in its entirety?

dhruvkb commented 3 weeks ago

@AetherUnbound that's interesting! I feel like a backup would be simpler but it does need one to compare the old and new file and manually move any keys that were changed or removed. At that point I'm not sure there is that much difference between diffing the new .env with its backup vs. not using this script and diffing the current .env with the template.

Are you suggesting the backup as an addition/improvement over this PR or does the use-case for preserving comments occur frequent enough to invalidate this PR?

AetherUnbound commented 3 weeks ago

Thanks for adding that backup creation step! That's exactly what I was thinking 😄

WordPress / openverse