[Bug]*.override.yml overwrites volumes instead of merging

[oranblackwell]

Describe the bug I created 2 override files to alter the directory structure on the mounted volumes. From your wiki page on Overrides I expected the 'volumes' section to be merged with the pre-configured volumes section (expected 4 volumes per service) but the overrides became the only volumes when viewing the re-generated docker.compose.yml

Steps to Reproduce Create sonar.overwride.yml and radarr.override.yml in their .apps/<service> directory as follows:

sonar.overwride.yml

services:
  sonarr:
    volumes:
      - ${MEDIADIR_TV}:/media/tv

radarr.overwride.yml

services:
  radarr:
    volumes:
      - ${MEDIADIR_TV}:/media/movies

Regenerate docker-compose.yml with: $ sudo bash ~/.docker/main.sh -g

Result:

sonarr:
    container_name: sonarr
    environment:
    - PGID=${PGID}
    - PUID=${PUID}
    - TZ=${TZ}
    image: lsioarmhf/sonarr
    ports:
    - ${SONARR_PORT_8989}:8989
    restart: always
    volumes:
    - ${MEDIADIR_TV}:/media/tv
 radarr:
    container_name: radarr
    environment:
    - PGID=${PGID}
    - PUID=${PUID}
    - TZ=${TZ}
    image: lsioarmhf/radarr
    ports:
    - ${RADARR_PORT_7878}:7878
    restart: always
    volumes:
    - ${MEDIADIR_MOVIES}:/media/movies

Expected result:

sonarr:
    container_name: sonarr
    environment:
    - PGID=${PGID}
    - PUID=${PUID}
    - TZ=${TZ}
    image: lsioarmhf/sonarr
    ports:
    - ${SONARR_PORT_8989}:8989
    restart: always
    volumes:
    - ${DOCKERCONFDIR}/sonarr:/config
    - ${DOWNLOADSDIR}:/downloads
    - ${MEDIADIR_TV}:/tv
    - ${MEDIADIR_TV}:/media/tv
radarr:
    container_name: radarr
    environment:
    - PGID=${PGID}
    - PUID=${PUID}
    - TZ=${TZ}
    image: lsioarmhf/radarr
    ports:
    - ${RADARR_PORT_7878}:7878
    restart: always
    volumes:
    - ${DOCKERCONFDIR}/radarr:/config
    - ${DOWNLOADSDIR}:/downloads
    - ${MEDIADIR_MOVIES}:/movies
    - ${MEDIADIR_MOVIES}:/media/movies

System (please complete the following information): Linux raspberrypi 4.14.34-v7+ #1110 SMP Mon Apr 16 15:18:51 BST 2018 armv7l GNU/Linux Raspbian GNU/Linux 9 (stretch)

[nemchik]

We will probably just adjust the wiki to match this behavior. I thought the tool we're using for yml merge would merge volumes, but I guess I misread the documentation for it. I'll do some tests to confirm and then change the wiki

[JonathonSunshine]

@oranblackwell Can I ask what you were trying to do? Did you want to mount your ${MEDIADIR_MOVIES} in two separate locations?

[oranblackwell]

@JonathonSunshine basically, I wanted to nest the tv and movie folders under /media as opposed to directly in the root directory.

If merging isnt an option I think being able to add additional volume(s) via the .env would be very handy.

Possibly create faux-arrays in .env, and then process them accordingly. SONARR_ADDITIONAL_VOLUMES="/first/path/to/host|/first/path/to/service,/second/path/to/host|/second/path/to/service" RADARR_ADDITIONAL_VOLUMES="/another/path/to/host|/another/path/to/service" COMMON_ADDITIONAL_VOLUMES="/mnt|/mnt"

[nemchik]

So you have MEDIADIR_TV=/some/folder/media/tv in your .env and you want sonarr to be able to use either /tv or /media/tv but both actually point to /some/folder/media/tv ?

[JonathonSunshine]

@oranblackwell Hi, I messed around with my over ride like this

version:             "3.6"
services:
  sonarr:
    image:           linuxserver/sonarr
    container_name:  sonarr
    restart:         always
    environment:
      - PGID=${PGID}
      - PUID=${PUID}
      - TZ=${TZ}
    volumes:
      - ${DOCKERCONFDIR}/sonarr:/config
      - ${DOWNLOADSDIR}:/downloads
      - ${MEDIADIR_TV}:/tv
      - ${MEDIADIR_TV}:/testsecondmount

It was just a straight copy of the sonarr.yml with the testsecondmount line added and I was able to browse the TV mnt from both locations within Sonarr.

In your override, you only had the one line, and it lived up to its name by override ing the default value, if you want to map the same location to two places, you have it both times I guess

[JonathonSunshine]

@oranblackwell but to "to nest the tv and movie folders under /media", in the .environment file, just set the global TV mnt to /media. But in Sonarr,

My most recent install, when I started with DockSTARTer, I formatted my SSD in 2 partitions, 32gb and 200GB, install on the 32, mounted the 200gb as /mnt/p2pDownloads

Then,

/mnt/TV                  # points to root of my tv hdd
/mnt/Movies           # this is an empty folder but inside i have mounted two more HDD, movies1 and 2

The default behaviour in .env suggests a mediafolder but I just put them in but I didn't do that. So if you want to run it that way, just a make a folder with all your TV and Movies and mount that as the TV and Movies locations.

[oranblackwell]

@nemchik @JonathonSunshine So, I didn't actually need to keep ${MEDIADIR_TV}:/tv but I didnt see an easy way to remove it without modifying base files. From reading the override's wiki entry I thought it would be better to keep the base version of the yml file to get any future release changes, with only my personal preferences in the override, but as @nemchik mentioned earlier the merge tool may not allow for this.

For what its worth, before I switched over to DockSTARTer I was running docker from a 16Gb SD Card on a Pi, with a 500Gb USB drive for downloading and extracting of files, and then moving them to a 3tb network drive for playback (ie. via Plex Media Server). My focus had been much less ambitious then DockSTARTer, so I chose to structure things a small bit differently, which gave me control of the container directories from the .env

I had mapped out the NZBGet and Deluge configs like below and my intention was to figure out how to automatically inject these variables directly into the programs config files but I discovered DockSTARTer before I had a chance.

NZBGet Structure

#  MainDir            /downloads
#  DestDir            ${MainDir}/completed
#  InterDir           ${MainDir}/intermediate
#  NZBs Dir           ${MainDir}/black-hole
#  NZBs Queue Dir     ${MainDir}/nzb-queue

Deluge Structure

#  DownloadTo         /downloads/intermediate
#  Move Completed To  /downloads/completed
#  Autoadd from       /downloads/black-hole

.env

DOWNLOADSDIR_USB=/mnt/usb/downloads
DOWNLOADSDIR_CONTAINER=/downloads

MEDIADIR_NET=/mnt/net
MEDIADIR_CONTAINER=/media

docker-compose.yml

 # --------------------------------------------------------------------------------
  # Sonarr – TV Show Download and Management
  # --------------------------------------------------------------------------------
  sonarr:
    container_name: sonarr
    image: lsioarmhf/sonarr:latest
    restart: unless-stopped
    ports:
      - '${IP_ADDRESS}:8989:8989'    
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
    volumes:
      - '${CONFIGDIR}/sonarr:/config'
      - '${DOWNLOADSDIR_USB}:${DOWNLOADSDIR_CONTAINER}'
      - '${MEDIADIR_NET}/tv:${MEDIADIR_CONTAINER}/tv'

  # --------------------------------------------------------------------------------
  # Radarr – Movie Download and Management
  # --------------------------------------------------------------------------------
  radarr:
    container_name: radarr
    image: lsioarmhf/radarr:latest
    restart: unless-stopped
    ports:
      - '${IP_ADDRESS}:7878:7878'    
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
    volumes:
      - '${CONFIGDIR}/radarr:/config'
      - '${DOWNLOADSDIR_USB}:${DOWNLOADSDIR_CONTAINER}'
      - '${MEDIADIR_NET}/movies:${MEDIADIR_CONTAINER}/movies'

[nemchik]

@oranblackwell would you be willing to join us on gitter? https://gitter.im/GhostWriters/DockSTARTer I've got questions about your former setup that will be easier to discuss in chat. I'd like to discuss your usage scenarios and see if it's something that makes sense to include in DockSTARTer.

[nemchik]

What I think makes sense for you with DockSTARTer is to set .env with these:

DOWNLOADSDIR=/mnt/usb/downloads
MEDIADIR_MOVIES=/mnt/net/movies
MEDIADIR_TV=/mnt/net/tv

And then in the containers use /downloads just like your already are, and instead of /media/movies in radarr just use /movies, and instead of /media/tv in sonarr just use /tv. This should have you set almost exactly like you were before DockSTARTer. Both sonarr and radarr have options to mass move (edit) the root directories so you can make the switch above very easily.

Still wouldn't mind to chat on gitter, you can let me know if I'm overlooking something :)

[nemchik]

@oranblackwell After our discussion on Gitter it sounds like you just want the folders to be different on the inside of the containers because of your personal preference, and I outlined briefly that you can do this with overrides. Do you need any additional help with this? (trying to figure out if we need to do anything else to resolve this issue before closing)

[nemchik]

I'm going to close this for now as I believe the answer above resolves the issue, but if anyone is still having trouble with this just reply here and we'll reopen and see if there's another way to address it.

[oranblackwell]

@nemchik only seeing this now. Thanks for your help.