madslundt / cloud-media-scripts

Upload and stream media from the cloud with or without encryption. Cache all new and recently streamed media locally to access quickly and reduce API calls
MIT License
88 stars 11 forks source link

How can I make /root/cloud-media-scripts/.cloud-decrypt sync simultaneously? #43

Closed lowprofileusername closed 6 years ago

lowprofileusername commented 6 years ago

I have only 20g space on vps. so I would like to stream on /root/cloud-media-scripts/.cloud-decrypt dir. My idea is download file to /root/cloud-media-scripts/.local-decrypt and upload it right after download completed and remove the file. and stream the file on /root/cloud-media-scripts/.cloud-decrypt dir. is this the correct way to use this script? every time I execute /root/cloud-media-scripts/scripts/cloudupload the script will upload file from /root/cloud-media-scripts/.local-decrypt. However after upload completed. I still can not find the newly uploaded file under /root/cloud-media-scripts/.cloud-decrypt. Only I can found it when I reboot the server.

How can I make the file appear in /root/cloud-media-scripts/.cloud-decrypt immediately right after I upload the file?

Thanks.

Dulanic commented 6 years ago

Have you tried unmount and remount? Also, for plexdrive issues, I would refer to that programs issues. https://github.com/dweidenfeld/plexdrive

But as an idea, you may want to alter the script and change the code in cloudupload to move instead of copy then it will do it all for you and delete the file.

Should just be changing line 65 from "${rclone_bin}" copy to "${rclone_bin}" move

Another idea might be to use https://github.com/rvoicilas/inotify-tools to fire off the cloudupload whenever a new files gets added? I don';t do it, but it's an idea.

lowprofileusername commented 6 years ago

Thanks for your answer. Remove files is not the problem. as you 've mention the solution as well I can also use /root/cloud-media-scripts/scripts/rmlocal to remove it.

My main question is how to keep /root/cloud-media-scripts/.cloud-decrypt up to date as file already been change on google drive. Say googledrive /Media has newly added one file. but When I check /root/cloud-media-scripts/.cloud-decrypt there still nothing in it. or does it have some sort of cache and need to be refreshed in a certain way.

I do believe unmount and remount would solve the problem but that's not the perfect way to solve the problem.

"Rclone is used to encrypt, decrypt and upload files to the cloud. Rclone is used to mount and decrypt Plexdrive to a different folder (cloud_decrypt_dir). Rclone encrypts and uploads from a local folder (local_decrypt_dir) to the cloud."

btw, isn't Rclone that mounted drive to cloud_decrypt_dir instead of plexdrive? Also what does ${media_dir}/.cloud-encrypt do? when I enter this folder it list root of my google drive. what is it use for?

Best Regards.

Dulanic commented 6 years ago

The cache is all done through plexdrive, there may be a delay or some other settings you can change to get it updated faster but I am not sure what it may be. madslundt may know, but I am not sure off hand. I did test a manual upload myself and it does not show right away for me either. It is plexdrive cache thing, just not sure why.

EX: I uploaded 75113f82ad1a461ba9d628d69400471f.mkv...

If I do find . -name 75113f82ad1a461ba9d628d69400471f.mkvI get no results but if I do rclone I see it.

sudo rclone ls gd-crypt: | grep 75113f82ad1a461ba9d628d69400471f.mkv
2534592468 75113f82ad1a461ba9d628d69400471f.mkv

FYI, it did show up in plex drive within 10 minutes. Also, dont recommend ls grep to find a file if you have many many TB of data lol just earned myself a API ban.

lowprofileusername commented 6 years ago

Sorry for causing that ban for you.

I tried replace plexdrive with 5.0 instald of 4.0 to it if it would update the cache more quickly. however because of MongoDB replaced by BoltDB the script does not work.

Do you have time for a update version of script that use plexdrive 5.0?

thanks

Dulanic commented 6 years ago

This script will stay with 4.0 for lower memory usage. You could fork it and make a 5.0 version. That is unless madslundt changed his mind, which I doubt.

On Mon, Feb 26, 2018, 10:41 AM randon name notifications@github.com wrote:

Sorry for causing that ban for you.

I tried replace plexdrive with 5.0 instald of 4.0 to it if it would update the cache more quickly. however because of MongoDB replaced by BoltDB the script does not work.

Do you have time for a update version of script that use plexdrive 5.0?

thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/madslundt/cloud-media-scripts/issues/43#issuecomment-368565191, or mute the thread https://github.com/notifications/unsubscribe-auth/ACz8bs_Mr_WaEgpCFusBvESM3FoODDhmks5tYt6vgaJpZM4STaBV .

lowprofileusername commented 6 years ago

in order to run 5.0 I need to make some change. So everything is ok but I have problem edit mount.remote script. I have to execute /root/cloud-media-scripts/plexdrive/plexdrive-linux-amd64 mount -c /root/cloud-media-scripts/plexdrive/.plexdrive -o allow_other /root/cloud-media-scripts/.cloud-encrypt before I run mount.remote because I dont know how to modify screen -dmS plexdrive "$plexdrive_bin" --config="${plexdrive_dir}" $mongo $plexdrive_options "${cloud_dir}" in a proper way. Can you lend me a hand?

I tried with screen -dmS plexdrive "$plexdrive_bin" mount -c "${plexdrive_dir}/.plexdrive" $plexdrive_options "${cloud_dir}" but it doesn't work.

Thank you very much

Dulanic commented 6 years ago

Check those values in config file, thats where you need to change the options and dir etc... plex_dir etc... all comes from cloud-media-scripts/config.

madslundt commented 6 years ago

@lowprofileusername you are of course welcome to create a PR with Plexdrive v5 that I can merge into a new branch, so it will separate v5 and v4. Some users have asked for Plexdrive v5 so when I get the time I'll see if I can do something about it and make the install to install v5 if the user wants it. The reason that I want to stay with v4 is because v5 only uses memory cache

lowprofileusername commented 6 years ago

I'm willing to contribute something here. But I'm not good at coding. I will post what I did here. so others can make a v5 script base on it.

  1. comment out all the string related to mongo in setup.sh since v5 do not need it anymore.

2.change _plexdrive_url from 4.0.0 to 5.0.0 in setup.sh.

3.plexdrive_options= in config remove --temp=${plexdrive_temp_dir} --clear-chunk-max-size=300G --clear-chunk-age=24h --chunk-size=10M since it store small chunk in memory and release very quickly according to manual.

  1. make change for mount.remote script. screen -dmS plexdrive "$plexdrive_bin" mount --config="${plexdrive_dir}" $plexdrive_options "${cloud_dir}" by add mount .

that's all the thing I can remember for what I did to make 5.0.0 work with the script. Maybe something is missing here:P

btw . this maybe a little bit off topic. but how long did you guys wait for ""Libraries are currently being updated" " when you add a google drive mount folder as libary? seems something wrong with my settings. with only one movie it takes forever to upload. seems permissions problem.