JSON appender script

[niccolopaganini]

• python script to append anything to a JSON file in any given directory (configs in this case)
• environment.yml file for the aforementioned python script

[niccolopaganini]

issue here: https://github.com/e-mission/op-admin-dashboard/issues/75

Added a template for the time being

[niccolopaganini]

@shankari please let me know if I understood what you said #70 here correctly.

[shankari]

@niccolopaganini please see my response. I don't want all the files to be changed. Also, as you can see, this code has introduced several extraneous changes. The only changes to the files should be the ones that we want. You probably want to write in UTF-8 format with appropriate indentation to avoid extraneous changes.

[shankari]

At this point, you may want to submit a separate PR by making the changes manually so we can get this done today, and work on cleaning up the script later.

[niccolopaganini]

checklist:

1. encoding format ✅
2. only target files ✅
3. see if it has any "side effects" - I could not find anything after reverting the changed I made

[niccolopaganini]

troubleshooted for like the last 25 minutes and had to force-push the changes. revert all the config files that don't need any change. Changes only applied by script to the JSON files that need change. Hopefully, this meets all the criteria.

[shankari]

@niccolopaganini this is much better - the files are in UTF-8. However, as I said in https://github.com/e-mission/op-admin-dashboard/issues/70#issuecomment-1744049354

The list of programs that should not exclude the user_id is:

Instead, you have excluded the user_id from just those programs. You need to flip this around.

There are still extraneous formatting changes.

As I already said:

The only changes to the files should be the ones that we want.

This means that the only change should be the line for data_trajectories_columns_exclude. Everything else should be unchanged. Otherwise this is hard to review, and git blame gets too messed up. If you can't figure out the JSON dump arguments to enable this, feel free to use jq to edit the files directly instead. If there is a reason why this doesn't work, please document it

[niccolopaganini]

I was thinking about how to tackle this problem and I found this to be a possible path:

• Work on/ Write a JSON Schema Validation script something like

  {
  k: val
  k: val
  {
   k: val
   k: val
  ...
  }
  }

• Validate them

  import jsonschema
  # open schema
  ...
  # iterate through the JSON files and then validate them

  And these can go on an exception list or have their own schema:

  user_id is: ca-ebike, stage-program, stage-study, uprm-civic, uprm-nicr, usaid-laos-ev, uue, washingtoncommons.

I went through the actions in the repo to peek at what's happening there (I wanted to see if there is anything relatable but I couldn't). Is this a path I could take that would be fruitful @shankari?