search

PGBlitz / PGClone

Visit the PG Repos @ https://github.com/PGBlitz - PGClone (PG's Mount & RClone Execution)
https://pgblitz.com
GNU General Public License v3.0
23 stars 72 forks source link

Add option to use the default media folders #8

Closed Haulien closed 5 years ago

Haulien commented 5 years ago

I've seen a few sparse comments on the forums and whatnot where some are annoyed that the default media folders (abooks, ebooks, movies, music, tv) keep getting created when they get deleted (eg: if people are using their own file structure).

This change adds an option to pgclone that, should the user wish, will allow them to set an option so those folders are not created by default.

The default option is set to Yes, create folders. And I've added a warning in there for those who are unsure. See https://github.com/Haulien/PlexGuide-PGClone/blob/a40b04500ec6fdb140d12dccdbab0e882ab3bd35/gdrive.sh#L49 for the warning and the function.

It auto-sets /var/plexguide/pgclone.defaultFolders which is then read by the ansible playbooks for gdrive, gcrypt, tdrive and tcrypt. If set to Yes (default) it proceeds as normal. If set to No by the user then just the main directory (/mnt/tcrypt, etc) is created.

ghost commented 5 years ago

While I think it's silly to be annoyed by a few empty folders, this is done for a reason.

You should take this a step further and instead do a 'category' setup.

Prompt for a movies category, default to 'movies' etc, store each category as a var. Then modify the core downloaders, for they setup cats on a new install. Swap out the hard coded cat strings with the vars.

Even though you add a warning people will still do it.

Even if you just do this PR, those folders will eventually exist because the downloaders use these categories.

So you'll need to work on this more before we can accept it

Haulien commented 5 years ago

Category setup would be a good idea and I might dig into that next. That however is separate, larger beast to tackle!

That's the thing though, the only thing hard coding those specific directories are pgclone itself.

Downloaders (Nzbget, rtorrent etc) are setup to use those folders only in the /mnt/downloads/[software] locations. The download software aren't setup by default to move finished downloads into gdrive as such the folders never show up on g/tdrive/crypt due to the download manager. They pass the files onto the media manager to handle it.

Then you look at the media managers (sonarr, radarr etc). By default the user has to configure the directory they move/copy files to after they get them from the download manager which makes the default folders moot as they're not default for the manager either.

Then you have the front end software to your library. Plex and emby aren't configured on with any media directory at all. It's up to the user to choose the directory for their library.

A big deal keeps being made about the default folders being a necessity yet the only thing that references them in regards to gdrive/tdrive in the default config are pgclone.

Downloaders download to /mnt/downloads and pass the file off to your library manager. Library manager renames and moves/copies the file into the directory they're configured for, and mergerfs/rclone handles getting it to google. Library frontend uses whichever directory you point it at.

ghost commented 5 years ago

That's not true.

The downloaders set up categories with those folders too, they will still make their way into gdrive that way during the upload process.

Test your change on a clean install without using existing appdata and you will find those folders still get created.

It's a big deal because it's out of the box configuration that just works. All of the wiki assumes that these folders are used and exist.

If you want to change that, then you have to finish the PR and then when are ready to merge, you will need to update the wiki in several areas.

This change as is will cause nothing but more support and confusion.

On Mon, Feb 4, 2019, 10:47 AM Steven Saric <notifications@github.com wrote:

Category setup would be a good idea and I might dig into that next. That however is separate, larger beast to tackle!

That's the thing though, the only thing hard coding those specific directories are pgclone itself.

Downloaders (Nzbget, rtorrent etc) are setup to use those folders only in the /mnt/downloads/[software] locations. The download software aren't setup by default to move finished downloads into gdrive as such the folders never show up on g/tdrive/crypt due to the download manager. They pass the files onto the media manager to handle it.

Then you look at the media managers (sonarr, radarr etc). By default the user has to configure the directory they move/copy files to after they get them from the download manager which makes the default folders moot as they're not default for the manager either.

Then you have the front end software to your library. Plex and emby aren't configured on with any media directory at all. It's up to the user to choose the directory for their library.

A big deal keeps being made about the default folders being a necessity yet the only thing that references them in regards to gdrive/tdrive in the default config are pgclone.

Downloaders download to /mnt/downloads and pass the file off to your library manager. Library manager renames and moves/copies the file into the directory they're configured for, and mergerfs/rclone handles getting it to google. Library frontend uses whichever directory you point it at.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Admin9705/PlexGuide-PGClone/pull/8#issuecomment-460343624, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3YJSx9NBTaETwFdNTafia6FCma5cXVks5vKHI9gaJpZM4ahbYm .

Haulien commented 5 years ago

Just eyed through the entire code base, there are a few instances that reference these directories in relation to the google drive mounts.

They are:

As mentioned before the download clients all use /mnt/downloads/[client]/tv,music,etc however those paths have no effect on the google drive directory.

I deployed a droplet and set it up quickly with a clean isntall of PlexGuide v8.3.13. After a quick test with nzbget and radarr, here are my observations.

1) Radarr's default movie category when adding an NZBGet server is to search for Movies and as such it fails as the default category for nzbget was changed to movies. 2) The first time Radarr asks you to add a movie it asks you to set a download path, I set mine to /mnt/unionfs/Movies3 (screenshot of: rename settings, movie settings, remote path mappings, drive/Movies3, top level gdrive) 3) My observation is that the default directory of movies is not created by either nzbget or radarr.

If you're able to replicate my findings the only other changes that would need to be made to remove dependencies on predefined folders are:

ghost commented 5 years ago

Nzbget and sab will auto create the category folders. Those folders are in /mnt/downloads/sabnzbd/....

They will eventually show up because the blitz/move process moves everything not explicitly ignored.

Regardless, my decision still stands. You need to do further work and have it so users can define their category folders names, which will be used inside _downloaders.yml to create the folders.

The changes I'm requesting are not unreasonable. This is a good start but in it's current state is not providing the best experience for the use-case.

On Mon, Feb 4, 2019, 3:29 PM Steven Saric <notifications@github.com wrote:

Just eyed through the entire code base, there are a few instances that reference these directories in relation to the google drive mounts.

They are:

As mentioned before the download clients all use /mnt/downloads/[client]/tv,music,etc however those paths have no effect on the google drive directory.

I deployed a droplet and set it up quickly with a clean isntall of PlexGuide v8.3.13. After a quick test with nzbget and radarr, here are my observations.

If you're able to replicate my findings the only other changes that would need to be made to remove dependencies on predefined folders are:

  • Cleanup in main.yml (having it in there is pointless if PGClone makes the directories too),
  • Modifying the api reading of config.py for pgtrakt to instead use the API instead of a hardcoded path
  • Add a wiki entry detailing how the user can use custom paths

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Admin9705/PlexGuide-PGClone/pull/8#issuecomment-460439054, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3YJbEnWImDBkZqoCVI3_n0vQqb_63oks5vKLROgaJpZM4ahbYm .

Haulien commented 5 years ago

Nzbget and sab will auto create the category folders. Those folders are in /mnt/downloads/sabnzbd/.... They will eventually show up because the blitz/move process moves everything not explicitly ignored.

Except /mnt/downloads/sabnzbd, /mnt/downloads/nzbget, .../rtorrent etc are all explicitly ignored from uploading in pgmove.sh.

As I said the folders created inside /mnt/downloads/[dlclient]/[category] could be anything as they're not touched by the blitz/move process because they are explicitly ignored. The files get passed to Sonarr, Sadarr, etc. for renaming and hardlinked into /mnt/unionfs/<chosen download path> from /mnt/downloads/[dlclient]/[category]

Admin9705 commented 5 years ago

i'm off testing other things, but looks interesting. thanks for submitting; apologize not pulled yet but want to ensure it works well :D; including reviewing 7's comments

ghost commented 5 years ago

Nzbget and sab will auto create the category folders. Those folders are in /mnt/downloads/sabnzbd/.... They will eventually show up because the blitz/move process moves everything not explicitly ignored.

Except /mnt/downloads/sabnzbd, /mnt/downloads/nzbget, .../rtorrent etc are all explicitly ignored from uploading in pgmove.sh.

As I said the folders created inside /mnt/downloads/[dlclient]/[category] could be anything as they're not touched by the blitz/move process because they are explicitly ignored. The files get passed to Sonarr, Sadarr, etc. for renaming and hardlinked into /mnt/unionfs/<chosen download path> from /mnt/downloads/[dlclient]/[category]

You are correct that they wont be made, but that requires the person to edit the downloader categories and then change the library location in arr. You may think that is obvious...but to others who barely understand any of this, it won't be.

For example, LazyLibrarian and other apps are setup by PG to use abooks/ebooks as library folders. We would do the same for sonarr/radarr if the configuration wasn't inside a sql db. We may find a good solution one day for configuring sonarr/radarr. If we merged the PR as is, someone not understanding the ins and outs (most people) may choose not to create them, then file a bug that LazyLibrarian is broken because abooks and ebooks are missing.

So let's provide the best experience instead of just a shortcut.

You have time to update the PR as we are not merging anything in besides quick fixes for 8.3 at this point.

Please update the PR so it prompts the user to setup categories, provide the defaults, but allow them to override it, if they put in an empty folder name, let's assume they don't want to use the category.

You just need to create some vars, set them to the defaults when making them. Then your prompt should say, would you like to change the default categories. Then prompt for each one (provide current value), then they can change it or just hit enter to accept. It sets the vars. Then prep a PR to update Core _downloaders.yml and the downloaders themselves to use the vars.

This will provide the best experience while addressing your issue.

Admin9705 commented 5 years ago

closing this until a solution is generated.

ghost commented 5 years ago

@haulien the community would really appreciate the ability to set custom categories....

This change may take you an hour or two. Do you have any intention on doing this?

Haulien commented 5 years ago

I've found myself stuck on a tropical island where it's sunny all year round. If someone hasn't done it before I get back I might have a look at it in March when I've made my escape.

ghost commented 5 years ago

@Haulien I pushed a change to no longer create the folders.

This PR won't be needed anymore.