[REQUEST] Immutable Config

[dcplaya]

I have this script (in docker format) running in my kubernetes cluster and it works great! There is one downside, I have to store the config inside of RW volume, which makes updating it a pain in the ass! Ideally, I would use a configMap to allow me to keep the config inside of my git tree. Unfortunately, this requires the config to be immutable since the configMap mount is RO. This, then, breaks the Trakt integration since it stores the authorization inside of the config file.

Is there anyway to pull that part out of the config and store it in a separate file alongside of the config file? This would still live in my persistent volume claim as RW but the actual config would be static.

[meisnate12]

so if we put the collections attribute and everything under it in a separate file from all the config attributes would that work for you? @mza921 what do you think? it could also be beneficial if you're running multiple collections files and dont want to update every file with a change to the basic authorization.

[dcplaya]

As far as I can tell, the only thing that gets update via the script is the trakt authorization. The other auth stuff (Radarr/Plex/ect) is set just once (unless API keys get changed).

Specifically this section of the trakt config (copied from the example config in the repo)

  # Below is filled in automatically when the script is run
  authorization:
    access_token:
    token_type:
    expires_in:
    refresh_token:
    scope:
    created_at:

That data gets updated whenever the script re-authenticates with Trakt and will periodically update the values (at every run of the script if #50 is implemented)

[meisnate12]

The script will auto add missing attributes if they're not there so anything outside of the collection attribute has a chance to update. Plus if we move any part of the config it just makes sense to separate all the config attributes vs the collection attribute

[dcplaya]

How does the script add missing values (outside of the collections attributes) without knowing the proper value? If I leave out the plex URL, how does the script know the correct URL? Or does it add a placeholder value and tell the user to go substitute the correct value?

The config attributes (except for the authorization for trakt) are a "set and forget" value. Once the value is entered into the config.yml, the value never gets changes (unless the user personally changes it) while the trakt authorization will be updated periodically since its a bearer token. The write once part isn't the issue (since I would be entering in the API keys per each service anyway), its the periodic, unattended, writes to the config file that causes the issue.

Another possibility would be to add ENV to the docker for the auth stuff, but I noticed the README says extra docker features are coming but nothing really planned yet.

I can live with how it currently is, this request is more of a "nice to have" type of thing, especially since I can then create a helm chart for it and submit it to the k8s-at-home project.

[mza921]

The plex section would also need some extra consideration because of the library and library_type attributes. If splitting up auth from collection configuration, it makes sense to include these attributes in each of the collection configs.

[dcplaya]

The plex section would also need some extra consideration because of the library and library_type attributes. If splitting up auth from collection configuration, it makes sense to include these attributes in each of the collection configs.

I actually wondered why I had to run a separate instance, one for TV and one for Movies. Ideally a single instance to do both would be nice. But that is a whole separate feature and really not hard to deal with (copy+paste, change some values and I have a cronjob running both).

[meisnate12]

under the plex section we could do something like this

plex:
  libraries:
    Movies:    #Library Name
      library_config_path: /movie_collection.yml
      library_type: movie
    TV Shows:    #Library Name
      library_config_path: /show_collection.yml
      library_type: show
  token: ###################
  url: http://192.168.1.5:32400
  sync_mode: append

[burkasaurusrex]

Is there anyway to pull that part out of the config and store it in a separate file alongside of the config file? This would still live in my persistent volume claim as RW but the actual config would be static.

This feels like a bit of edge case to me / feels like lower priority. Not sure I completely understand the upside in generating another config if it needs to be RW anyways. A middle ground might be creating a setting like update_config or something where users can set it to false if they don't want the script to update the config.

I actually wondered why I had to run a separate instance, one for TV and one for Movies. Ideally a single instance to do both would be nice. But that is a whole separate feature and really not hard to deal with (copy+paste, change some values and I have a cronjob running both).

under the plex section we could do something like this

plex:
  libraries:
    Movies:    #Library Name
      library_config_path: /movie_collection.yml
      library_type: movie
    TV Shows:    #Library Name
      library_config_path: /show_collection.yml
      library_type: show
  token: ###################
  url: http://192.168.1.5:32400
  sync_mode: append

We should find a way to address this I think. The most logical to me would be set this by each collection element? Users might have multiple Movies and TV Shows libraries, so being able to map group of collections to Plex libraries might be useful. We could probably do it by naming collections or something.

[dcplaya]

This feels like a bit of edge case to me / feels like lower priority. Not sure I completely understand the upside in generating another config if it needs to be RW anyways. A middle ground might be creating a setting like update_config or something where users can set it to false if they don't want the script to update the config.

The config doesn't need to be RW, it needs to be RWO (Read/Write Once) except for the Trakt authorization bearer keys (which get updated either every script run or when the bearer key expires). In Kubernetes, I can make what is called a configMap which will mount data as a RO file. This allows me to store my Plex Auto Collection config as code that is backed up to my git (and thus easily restored/deployed/copied and a record of changes). This only works if the data does not need to be written to. Since the Plex Auto Collection config (again, except for Trakt auth section) is a "write once, read multiple" file (writing once is me filling in all of the API keys and the collection section and then pushing it to my git account which then would be sent to my cluster and deployed), a configMap is a perfect solution. Currently, if I want to make a collections change or an API change, I have to manually mount the volume the config is stored in, drop into a terminal and modify the config, then unmount the volume. Separating the collections section is helpful since the collection section is the section that would be modified the most, but I would still have to keep a volume setup and active and at that point, its no different than keeping the collection section where it is currently. For an example, check out my AdGuard config, which is mounted as a configMap for that container. All of that data is technically RW but since the data never changes unless I concensiously want to change it, I can control it via gitops. It might make things a bit clearer? It also might make it more confusing though.

We should find a way to address this I think. The most logical to me would be set this by each collection element? Users might have multiple Movies and TV Shows libraries, so being able to map group of collections to Plex libraries might be useful. We could probably do it by naming collections or something.

As for this, I do have multiple "Movie" libraries (4K vs non 4K for example). Maybe keep what you suggested above, which defines the library name and the type, then for each collection entry, you define the name of the library too or there are sub-keys under collection that defines the library name and then nested in that are the actual collection data? This is if you want to keep the collections and authentication data all in a single config file.

[JoeIzzard]

I would also like to see API config split out from my collection config, as I generally would push my collections to git but have the API config not in git for example, something like config_api.yml and config_collection.yml.

Also good to see confirmation that currently you need multiple instances for Movies and TV. I couldn't see anything in the README confirming that (Although I tend to do this sort of stuff later at night so may have missed it). Another option could be you specify a directory in the main config, which can contain as many config files as you like that are specifically collections. May make it easier to manage a large number of collections as you would be able to group similar collections my file, for example:

collections.d/
   --> decades.yml
   --> years.yml
   --> studios.yml