Don't get prompted for rclone mount name after configuring rclone (and other issues)

thunderstorm654 commented 1 year ago

Hi, upgrading from Gooby to OmniStream today :)

When I do the rclone step, I say 'yes' to configure, then 'q' to quit the rclone config since I already have a 'gdrive' mount there.

However it doesn't then ask me to confirm the name of the mount, so OmniStream seems to know nothing about it. I've tried going back through a few times, but it definitely isn't prompting me after 'q' the rclone config to confirm my mount name :(

The omnimount status is constantly 'restarting'.

Any ideas?

I've only re-enabled Plex (the hw one as I have hw transcoding available), but this isn't working either - I chose Cloudflare as all my DNS is through them, but I just get a privacy error on visiting my plex domain address - invalid SSL certificate. Grateful for any ideas, thank you for all the hard work!

Not sure if it's related to anything above, but I also see an error every time I use any of the omni commands: jq: error (at <stdin>:0): Cannot iterate over null (null) error

TechPerplexed commented 1 year ago

Right, so Plex won't work until your OmniMount is healthy - to prevent it from "removing" all your content :) So that part is easy to answer.

Rclone: if you type omni configs and check the content of the folder omnimount, does it contain the rclone.conf file and is it populated with the mount information?

Oh and welcome to Omni - I know that was a big decision for you and we aim for you not to be disappointed, haha 😆

thunderstorm654 commented 1 year ago

Haha thank you!

rclone.conf is there, but if I try to open it as my user account that's running OmniStream it gives me access denied.

If I check the contents as root, it has my gdrive one listed, with client ID, secret and access token.

I took a bit of a cheeky guess and did a chmod 777 on rclone.conf, though I can now see it as my "Omni" user it's still saying restarting on the omnimount (tried Omni down and up). Doh :)

thunderstorm654 commented 1 year ago

I'm hoping these logs help - looks like the same thing over and over, but pasted the lot in case ha! It does seem like the fact I never got asked to confirm my mount name for rclone may be the missing link?

kelinger commented 1 year ago

First thing: OmniMount definely won't work with rcloneservice and rclonemount undefined. So, let's start there.

Can you run:

omni down
sudo chown -R $USER:$USER ~/.config
sudo chown -R $USER:$USER $OMNIHOME/configs

This will make sure you, the user, own these files and not root. The rclone.conf that is under the home/.config directory is used solely for setup purposes though it is copied into the $OMNIHOME/configs/omnimount/ path for use by OmniStream.

Since you already have an rclone file, run omni menu and choose 3 Setups Menu then rerun rclone setup then choose no and OK.. Verify that this is correct and that there aren't any additional characters or lines added from a previous cut and paste job. Save and exit the editor.

You should now be presented with the choice of the services you saw in the editor. Choose the one that contains your media. After that, exit the menus and see if you can start OmniStream now with omni up.

thunderstorm654 commented 1 year ago

Thank you for your reply @kelinger . I've followed all your steps to the letter and ran into the same issue I had originally - I never get promoted to choose the services I see in the editor. My rclone.conf loads, but once I save and exit (I use ctrl + x as nothing needs changing, I can see my remote with client ID/secret etc), I just get bumped back to the Omni setup menu. There's still a weird jq error coming up e.g. when running Omni down:

Screenshot_20221009-081718~2

Then when I do the re run of rclone, this is what happens.

Choose 'no' to check file: Screenshot_20221009-082117~2

Editing file: Screenshot_20221009-082120~2

Press ctrl+x to exit and then I just get bumped immediately here, no prompt to select mount name: Screenshot_20221009-082124~2

So close and yet so far! Something seems funky with the menus not quite working right - I know the exact prompt you mention as I remember Gooby doing it as well.

thunderstorm654 commented 1 year ago

Ok so I thought let's just do it all again from scratch. Wiped server, restored Gooby, verified all is working (rclone included!). Then started the upgrade again.

Still going to have the same issue I think - when I get to open up existing rclone.conf then exit editor, the installer never asks me to confirm the mount name.

I checked your install guide and it's specifically this that never comes up for me: Screenshot_20221009-100055

It just progresses onto the next steps.

Is there something about my rclone.conf it can't process?! There's only my gdrive in there and a mostly empty looking "local" (screenshot of my conf file in previous reply above).

Edit: removing the local one doesn't help.

TechPerplexed commented 1 year ago

Can you post a screenshot of your rclone.conf file? Of COURSE with the actual codes masked but it might give us an idea of what could be going on... ^_^

thunderstorm654 commented 1 year ago

@TechPerplexed Sure it was in my previous response above, but here it is :)

Screenshot_20221010-053533

Ps. Sorry all my images come out big, I'm replying on mobile with screenshots taken on phone.

The empty team_drive variable - I have tried getting rid of that as well and that makes no difference either way. On exiting the editor, I just immediately get this screen; Screenshot_20221010-054029~2

Is there a way I can directly tell OmniStream about this remote/file as the menu isn't working/coming up? Don't want to give up with the install but Plex down a couple days already and have some users missing it :) feels like so close to getting it going!

thunderstorm654 commented 1 year ago

Ok I managed to update them manually via Omni edit. As soon as I set the missing RCLONESERVICE and RCLONEMOUNT variables to gdrive, moved my rclone.conf manually to the omnimount folder, then did an omni down / omni up the mount was working :) I did have to open up the scripts to figure out what it would normally do after closing the conf file to figure these bits out!

Anyway hooray I now have Plex back (and Emby and Tautulli!). One question I hope is an easier answer - I use cloudflare for DNS so set that up with the API key. However I have found that I needed to turn off proxying through Cloudflare and only use DNS for Plex and Emby, as otherwise had intermittent buffering issues with 4K playback. I know this is less secure but I'm ok with that.

With Gooby and managing DNS manually I could just turn off proxying in the Cloudflare dashboard and all still good and working. But Omni is overwriting that setting as it seems to default to proxying. Please tell me there's a way I can toggle that :) I see in the DNS script to create records there's an input variable for it, but I don't know where that gets called from or how to change the input - is it in the docker files? Divert to you experts as I don't want to break stuff - a variables file is one thing but this feels more likely to go wrong if I'm guessing ;)

TechPerplexed commented 1 year ago

Sorry, I missed that information. Very weird indeed!!! But at least it now works for you...

As for managing the DNS, of course @kelinger facilitated that ^_^ All you need to do is copy the content of your Plex yaml to the 8xx range and change one line:

- "omni=${PLEXNAME},${MYDOMAIN},yes,yes,yes"

This tells the start/stop process the name and domain used and whether it should be added to Cloudflare on startup, removed from Cloudflare on shutdown, and whether it should be cached (“orange cloud”) by Cloudflare.

So, in your case, change it to

- "omni=${PLEXNAME},${MYDOMAIN},yes,yes,no"

That should create the entry in CloudFlare without the cache feature!

kelinger commented 1 year ago

Thank you for the info. I'll definitely be exploring the "why" part of why your Rclone configuration didn't pop up in the confirmation part of the setup when you are choosing the volume to mount. The expectation is that you should see another dialog box with a list of the drive(s) defined in your rclone.conf under ~/.config/rclone/rclone.conf. If the setup script cannot read or access this file (or it is missing) that would be the first obvious thing to fix but it sounds like this is not the case.

As for disabling the cache, what @TechPerplexed described is accurate. In its current implementation, the 800-900 range of service definitions is reserved for any customization you want. So disable the Plex service, copy it to a new definition in the 801-899 range, edit it accordingly, and then activate it. From the perspective of OmniStream, this will act as any other component that was included. But, when OmniStream updates, it won't run into issues seeing a different "101-plex-hw.yaml" file than the one included.

Future potential: I've seen several other requests regarding the enabling/disabling of CF caching and may look to find a way to split that off so that it can be more readily tweaked by the end user. There's a fine line between "customization" and "config files that are hundreds of lines" and when we talk about each service having variable authentication, caching, GPU hardware-enabling/disabling, the ability to preconfigure all combinations gets rather lengthy. So, for now, this is the best way.

thunderstorm654 commented 1 year ago

Thank you both, really appreciate your help. There's definitely something odd with the lack of prompt to confirm the mount - I have reset my server three times and installed Omni via migration from Gooby and also from scratch and all cases it wouldn't prompt for it. I'm on a Hetzner dedi, just using Debian 10.

I spoke too soon about Plex working, it's ... Sort of!! I copied the yaml file and made a new one under 801, then brought that up ok.

I think I have some hangover things to fix with remote access. I followed an old tutorial a very long time ago from @TechPerplexed about adding an origin certificate with Cloudflare. I think some of this is still in my Plex settings under Remote access and Network. But for the life of me I don't know what to correct these to now. I don't want anything Cloudflare except the DNS.

Remote access is somewhat buggered, since the Omni upgrade it keeps showing indirect only (takes a long time to load then goes indirect) and the remote access green tick keeps disappearing and going red (then back again). I've attached some screenshots of the current config migrated from the Gooby days (this was working fine). Do I still need to specify a public port? Is it still 8443 or something else - I remember that used to be in the docker file.

Remote access: Screenshot_20221010-201538

Network: Screenshot_20221010-201634~3 Continued: Screenshot_20221010-201943

I wondered about creating a second Plex instance to check the defaults but obviously that won't work without getting clever about not using same data directories :)

Thanks again both. Really close now!

kelinger commented 1 year ago

No, do NOT create a public port in the YAML. Traefik takes care of all that. However, Plex, in its infinite wisdom, does NOT assume that https://mydomain.com = port 443 so you have to specify it in "remote access"

While under network/advanced, you just need https://plex.mydomain.com (no port)

Since Traefik handles the certificates, you do not need to specify one in Plex.

TechPerplexed commented 1 year ago

To add, my settings almost match yours - with the exception of "enable relay" (it's unchecked on mine).

Edit: sorry, I missed that he mentioned this custom certificate is no longer needed. I just removed it on my server as well - obviously a leftover from the olden days in both our cases 😄

thunderstorm654 commented 1 year ago

I don't know if I've blocked myself out of something by trying too many times with Omni down and up- my Plex domain won't work at all now. After I remove the certificate and changed the port to 443, app.plex.tv lost my server completely and apps don't find it either :( says it's offline, and I can't go to plex.mydomain.com either it just gives me this site can't be reached immediately.

Would recreating the containers and DNS stuff everytime on down/up have done such a thing?

I'm completely confused by this, as domain aside, it definitely didn't like 443! Is that the same as you too @TechPerplexed - do you also have 443?

Edit: using a different domain name has brought that back and I can sign in with Plex credentials, but it's lost the Plex server itself - it is listed "unavailable" - FFS Plex :) any ideas before I give up in despair? Can't believe Plex is being so funky.

thunderstorm654 commented 1 year ago

Man, Plex is horrid. I got things working in the end, but I firstly go with the new domain, then had to delete my Plex Preferences.xml file to force a reclaim of the server. Then I managed to sneakily recompile Preferences.xml with the old server IDs to make it think it was the original one (otherwise after a reclaim, have to reshare with everyone..!).

The remote access is now correctly configured with 443, and the Plex apps are working everywhere on all devices thank god - in Plex web, it does still seem to flit between green and red, but as it's now working and not sat there hanging and then showing 'indirect' I think we're good!

Thank you for your assistance with stuff by the way!

In a similar fashion to the rclone setup not prompting for the remote name used, I also noticed if you choose to edit advanced config during install, the config file doesn't open either. Maybe a similar problem. Or perhaps it's just these Hetzner boxes :)

One final question - (sorry!) - what is the default user/pass for Prowlarr?

I added it via the standard OmniStream config YAML file selected through the menus, and it's been created, but it prompts (in a generic looking pop up) for user/pass. Thanks and sorry for my frustrated ramblings - I think I was battling some obscure Plex-isms unique to my app data.

TechPerplexed commented 1 year ago

Heh, yes kelinger and I often joke that if we have an issue, anywhere, any problem at all, first blame Plex. Then blame Plex some more and after that, look further 🤣

But yes, I remember that I had that 443 issue too. You'd think that opening Plex to the correct port would make things better, not worse, yet here we are. In fact when we started troubleshooting your issues here, I had external access set to disabled. However I opened it yesterday in conjunction with removing the certificate and Plex stayed up, so I think you might have been right that the frequent up/down might have triggered some sort of ban temporarily.

Prowlarr: I use it too and I don't remember having to enter an initial password for that. I'll try a fresh installation a little later and will get back to you on that one.

No worries about the frustrated ramblings - setting up a new system that involves Plex in any way tends to have that effect of wanting to pull out all your hair - been there, done that🧑‍🦲

thunderstorm654 commented 1 year ago

I couldn't believe how painful Plex was. Everything else works 😂. I discovered that Prowlarr wanted "admin" and the Traefik password I set up on install. Noticed this container and some others (like Radarr) have an extra Traefik line in the compose yaml files for auth so guessing that's it!

shadowsbane0 commented 1 year ago

I'm sorry I'm late on this. For my Plex server remote access, I personally just manually set it to port 32400. It will initially say fully connected then it often changes to say it isn't but I've not had any connection issues - local or remote to the service. Unfortunately I'm getting to the point where I'm not a fan of all the bloat Plex has been adding as they grow. I was extremely pissed when visiting my son and connecting to my Plex with his account and had to suffer f'ing ads with my content. I'm primarily Emby now and thanks to @kelinger and @TechPerplexed the recent watchstate server keeps everything in line if I need to switch over for any reason to Jellyfin or Plex.

thunderstorm654 commented 1 year ago

I'm sorry I'm late on this. For my Plex server remote access, I personally just manually set it to port 32400. It will initially say fully connected then it often changes to say it isn't but I've not had any connection issues - local or remote to the service. Unfortunately I'm getting to the point where I'm not a fan of all the bloat Plex has been adding as they grow. I was extremely pissed when visiting my son and connecting to my Plex with his account and had to suffer f'ing ads with my content. I'm primarily Emby now and thanks to @kelinger and @TechPerplexed the recent watchstate server keeps everything in line if I need to switch over for any reason to Jellyfin or Plex.

Thanks for your comments @shadowsbane0 . I completely agree with the direction Plex has been taking, it's infuriating having to explain to users that no, "Movies and TV on Plex" which is a pile of ad driven crap isn't my stuff 😂. I run Emby and Jellyfin as well alongside and personally use Plex and Emby quite equally. The main thing I wish for in Emby is something like PlexMetaManager, as I really like having most popular movies/TV collection rows on front page alongside a bunch of others that can it generate. If they ever make Emby support or someone else does I'm off for good! :)

TechPerplexed commented 1 year ago

Make that three for jumping between Plex, Emby and (to a much lesser degree), Jellyfin. Thank goodness for Watchstate which makes switching virtually seamless!

And with that, let me archive this thread since I get the impression everything is sorted... for now. As always, feel free to reopen/add as you guys see fit ^_^

kelinger / OmniStream

Don't get prompted for rclone mount name after configuring rclone (and other issues) #21