Sonarr unable to import downloads

[imran0]

Setup

Package Name: Sonarr Package Version: 20180303-13 (inside app this is version 2.0.0.5163)

NAS Model: Synology DS212+ NAS Architecture: ? DSM version: DSM 5.0-4482

Expected behavior

Sonarr should import folder from /volume1/Downloads/Completed -> /volumeUSB2/usbshare/video/TV Shows

Actual behavior

The following error is displayed for a show: Couldn't import episode /volume1/Downloads/Completed/Series/Billions.S03E01.720p.WEB.H264-DEFLATE/f798e91152b843858196fcf16b2a30b0.mkv: Access to the path is denied.

In debug logs this error is logged:

18-3-26 00:40:18.0|Debug|EpisodeFileMovingService|Moving episode file: /volume1/Downloads/Completed/Series/Billions.S03E01.720p.WEB.H264-DEFLATE/f798e91152b843858196fcf16b2a30b0.mkv to /volumeUSB2/usbshare/video/TV Shows/Billions/3x01 - Tie Goes to the Runner WEBDL-720p.mkv
18-3-26 00:40:18.0|Debug|DiskTransferService|Move [/volume1/Downloads/Completed/Series/Billions.S03E01.720p.WEB.H264-DEFLATE/f798e91152b843858196fcf16b2a30b0.mkv] > [/volumeUSB2/usbshare/video/TV Shows/Billions/3x01 - Tie Goes to the Runner WEBDL-720p.mkv]
18-3-26 00:40:18.1|Warn|ImportApprovedEpisodes|Couldn't import episode /volume1/Downloads/Completed/Series/Billions.S03E01.720p.WEB.H264-DEFLATE/f798e91152b843858196fcf16b2a30b0.mkv

[v2.0.0.5163] System.UnauthorizedAccessException: Access to the path is denied.
  at System.IO.File.Move (System.String sourceFileName, System.String destFileName) [0x00116] in <cb410c64c7fa4e8d9841a0cbe4173d66>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFileInternal (System.String source, System.String destination) [0x00000] in <6d548036160a49ed8e2657c617163f50>:0 
  at NzbDrone.Mono.Disk.DiskProvider.MoveFileInternal (System.String source, System.String destination) [0x00076] in <a4f6bcacc2ea4724bcba67562d4e7206>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFile (System.String source, System.String destination, System.Boolean overwrite) [0x000e3] in <6d548036160a49ed8e2657c617163f50>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileTransactional (System.String sourcePath, System.String targetPath, System.Int64 originalSize, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x0008f] in <6d548036160a49ed8e2657c617163f50>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x003ce] in <6d548036160a49ed8e2657c617163f50>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, System.Boolean verified) [0x0000e] in <6d548036160a49ed8e2657c617163f50>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.TransferFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Tv.Series series, System.Collections.Generic.List`1[T] episodes, System.String destinationFilePath, NzbDrone.Common.Disk.TransferMode mode) [0x0012c] in <9b8bbe29888f49229914613e26af4aa5>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.MoveEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode) [0x0006c] in <9b8bbe29888f49229914613e26af4aa5>:0 
  at NzbDrone.Core.MediaFiles.UpgradeMediaFileService.UpgradeEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode, System.Boolean copyOnly) [0x0017c] in <9b8bbe29888f49229914613e26af4aa5>:0 
  at NzbDrone.Core.MediaFiles.EpisodeImport.ImportApprovedEpisodes.Import (System.Collections.Generic.List`1[T] decisions, System.Boolean newDownload, NzbDrone.Core.Download.DownloadClientItem downloadClientItem, NzbDrone.Core.MediaFiles.EpisodeImport.ImportMode importMode) [0x00277] in <9b8bbe29888f49229914613e26af4aa5>:0 

--

Steps I've taken: I have have given access to volumeUSB2 and Downloads to both sc-downloads and sc-media groups. I have even gone into the individual users (sc-nzbdrone) and given it access to those folders.

Please advise

[brandonscript]

I've also tried sudo synogroup --member sc-downloads sc-nzbdrone, per https://www.reddit.com/r/synology/comments/7bg9wp/synology_and_sonarr_permissions_issue/dpiispb/, but no dice.

[HansGr00ber]

Is this with all downloads, or just some of them? I find mine imports most downloads fine, but gets stuck on others.

I'm not 100% but it may just be the downloads that have names such as "1326ac2e980647708b9960e9952e8b15.mkv" instead of "show.episodenumber.mkv" that get the permission denied error.

[brandonscript]

@HansGr00ber for me it's doing it for everything. Has been since March 4.

[imran0]

@HansGr00ber Actually it appears to be only some downloads for me. I'm sure there were files named in this way before the update as I have never got this permission issue before.

Is there a way to fix this? (aside from trying another download)

[HansGr00ber]

I can confirm it isn't because of the filenames like 1326ac2e980647708b9960e9952e8b15.mkv as I saw it download and import something similar with no trouble a bit earlier. A bit stumped what it is that causes it.

[brandonscript]

Yeah. And I even tried chmod -R 0775 /path/to/parent and it didn't work. Only thing that's out of the ordinary for me is that source and destination are on two different partitions.

[imran0]

Ok so I was able to fix my issue, it looks like the folders for the series that were failing were owned by a weird "103" user. I have changed them back to admin and reset file permissions to 777 like the rest of my "working" series.

[brandonscript]

OK, WEIRD development, I just gave sc-media group recursive full control to the destination dir (not the source that's in the errors), and now things are working again.

[HansGr00ber]

I don't have a sc-media group on my machine, just sc-download

[malanden]

I too notice the issue is just with wr877524jsif8w3.filenames If its tvshow.s01e01.mkv its no problem. What I find is the weird filename remains with sc-nzbdrone as the owner, and the permission for the file is not given to the sc-download group

[malanden]

What I have also noticed is when NZBGET downloads the file and extracts it, it ends up with with folder/file owner as sc-nzbget and group is a weird number, in this case 219004

[ymartin59]

@brandonscript because of sudo synogroup --member sc-downloads sc-nzbdrone, you have removed sc-nzbdrone from its own default group nzbdrone - synogroup command requires to list of members, there is no "add" logic.

@malanden Service user account is now created by DSM "privilege" framework, each user has its own default group - group name is nzbget for sc-nzbget with GID same as UID.

[brandonscript]

Strange that it broke in the first place though? So should it be added back to sc-nzbget?

On Mar 29, 2018, 10:21 PM -0700, Yves Martin notifications@github.com, wrote:

@brandonscript because of sudo synogroup --member sc-downloads sc-nzbdrone, you have removed sc-nzbdrone from its own default group nzbdrone - synogroup command requires to list of members, there is no "add" logic. @malanden Service user account is now created by DSM "privilege" framework, each user has its own default group - group name is nzbget for sc-nzbget with GID same as UID. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[brandonscript]

Seems to be working again after a backup, reinstall, and restore. FYI docs here https://github.com/Sonarr/Sonarr/wiki/Backup-and-Restore still say to add permissions for users, but should be sudo chown -R sc-nzbdrone:sc-download * I suppose?

[HansGr00ber]

That's good to hear. What version of Sonarr are you running? I think my problems started when I installed the latest (2.0.0.5163).

@brandonscript I think it should be sudo chown -R sc-nzbdrone:nzbdrone *

[brandonscript]

Yep @HansGr00ber, that's the version I'm running, and that's when my problems started. I think it's related to the permissions system change in DSM6. Either it's my own ignorance, or.. ¯_(ツ)_/¯

[HansGr00ber]

I've just done a complete fresh install of my Synology. Factory reset it, wiped all drives, reinstalled all apps and reset all permissions in case anything was setup wrong from the past and first show downloaded and imported fine, second one:

Unable to parse media info from file: /volume1/downloads/completed/The.Crossing.S01E01.720p.WEB.x264-TBS/f96e2ac68d914e26864d1bbb8a1e7569.mkv: Access to the path "/volume1/downloads/completed/The.Crossing.S01E01.720p.WEB.x264-TBS/f96e2ac68d914e26864d1bbb8a1e7569.mkv" is denied.

System.UnauthorizedAccessException: Access to the path "/volume1/downloads/completed/The.Crossing.S01E01.720p.WEB.x264-TBS/f96e2ac68d914e26864d1bbb8a1e7569.mkv" is denied. at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x00259] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/corlib/System.IO/FileStream.cs:274 at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean isAsync, System.Boolean anonymous) [0x00000] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/corlib/System.IO/FileStream.cs:149 at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access) [0x00000] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/corlib/System.IO/FileStream.cs:86 at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor(string,System.IO.FileMode,System.IO.FileAccess) at NzbDrone.Common.Disk.DiskProviderBase.OpenReadStream (System.String path) [0x0001b] in M:\BuildAgent\work\b69c1fe19bfc2c38\src\NzbDrone.Common\Disk\DiskProviderBase.cs:405 at NzbDrone.Core.MediaFiles.MediaInfo.VideoFileInfoReader.GetMediaInfo (System.String filename) [0x0006e] in M:\BuildAgent\work\b69c1fe19bfc2c38\src\NzbDrone.Core\MediaFiles\MediaInfo\VideoFileInfoReader.cs:55

This is the permissions of the downloaded file as seen by Windows:

[brandonscript]

Ok now give the sc-downloads group full RW access to both the source (where the error is complaining) and the destination share where you’re trying to move it. What happens?

[HansGr00ber]

I tried that and it then imported ok. I then deleted the file and records from NZBGet and redownloaded it (from with Sonarr) and received the same error. So I was actually able to repeat the failure.

Next I tried downloading it again using SABnzbd, and it worked first time. Maybe the issue lies with NZBGet?

[brandonscript]

Maybe?? Could be this limbo between DSM 5 and 6-style permissions too?

[HansGr00ber]

I just thought I had something, same episode downloaded from different indexers, one failed and one successfully imported. Tried again on another episode and both failed to import. I'm stumped.

[malanden]

still having same issue. Its only the wierd filenames, not anything else

[HansGr00ber]

This is by no means conclusive but I've found NZBs downloaded from NZBGeek seem to work and NZBs downloaded from DrunkenSlug and NZB Finder can have permissions issues. The only difference I can see between these is that NZBGeek uses Newznab and the other two use nZEDb. Now is there some connection there? I'm not by any means techy enough to work out what's wrong but that info might be useful to someone.

[malanden]

Its definitely the random letters and numbers that are causing the issue. Where or how, I don't know. Anything that is properly named, works just fine. Anything obscure, doesn't work. I can manually import it with no problems through Sonarr, so I can't figure out how its a permissions issue.

[koenvanzuijlen]

Also getting this exact issue only on the random numbered files, sc-download has full r/w access to both source and destination but it still happens.

When a file has a "normal" name everything works fine, so doesn't seem to be a permission issue.

[ymartin59]

But error message is really "access is denied" (OK it comes from C#/.Net/Mono so...) What are this "random number" file properties? Please report from File Station and from command line with stat. We will probably have to increase application verbosity and maybe collect strace or libtrace to get understanding of what happens behind the scene.

[HansGr00ber]

Just to follow up my earlier post I've been keeping any eye on which indexer NZBs are coming from and can confirm 100% success rate downloaded from NZBGeek whether the file names are obfuscated or not, and NZBs from DrunkenSlug have been failing if they have an obfuscated name. I can only guess that's something to do with the back ends the sites use.

This is the properties of my latest failure

What would the exact command for stat be? I'm a bit of a Linux newbie.

Is this correct?

jake@wolfcola:/volume1/downloads/completed/LA.to.Vegas.S01E12.1080p.WEB.x264-TBS$ stat 83ce4df57e074378b7407bd4ddb223af.mkv
  File: ‘83ce4df57e074378b7407bd4ddb223af.mkv’
  Size: 809105444       Blocks: 1580296    IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 1375        Links: 1
Access: (0640/-rw-r-----)  Uid: (219004/sc-nzbget)   Gid: (219004/  nzbget)
Access: 2018-04-11 08:38:05.265912567 +0100
Modify: 2018-04-11 03:32:24.000000000 +0100
Change: 2018-04-11 08:38:05.235912986 +0100
 Birth: -

[koenvanzuijlen]

I also had a movie in Radarr with an obfuscated filename that didn't work in the exact same way.

Also have the stat of a file here, my nzb are coming from NZBFinder if that is relevant.

 File: ‘987450eaaa814c149500b5405ed0a274.mkv’
  Size: 5162147413      Blocks: 10082336   IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 9963165     Links: 1
Access: (0640/-rw-r-----)  Uid: (219004/sc-nzbget)   Gid: (219004/  nzbget)
Access: 2018-04-11 16:20:40.047818277 +0200
Modify: 2017-10-16 20:26:13.000000000 +0200
Change: 2018-04-11 16:20:39.687840386 +0200
 Birth: -

Keep in mind that a manual import works 100% of the time, so I don't see how the permissions are different?

[BenjV]

Maybe the obfuscated filename has characters that are not allowed as utf-8 characters. This can happen when a filename is created with a windows codepage and not with an utf-8 character set. Only Windows does this, the rest of the world uses utf-8 as a standard coding, so probably those sites uses windows servers to do the obfuscating.

Normal names mostly have only ascii characters and they are the same in both windows code page and utf-8 coding so they don't have this problem.

[ymartin59]

@BenjV Do you understand well: you mean "filename" known in application no longer matches when file has been written into DSM file system, right? If so, application developers may confirm this trouble and probably propose a patch to normalize filename to avoid such issue.

[BenjV]

It is most likely not done by the application but by the creators of the nzb's If you create an NZB on a windows system then the filename's are created with an 8 bit windows code page. If you unpar the nzb on a system that uses utf-8 the bytes are interpreted as utf-8 code. This can always be a problem if characters with accents or special characters are used, but an obfuscated filename has random characters as filename so the change it is going wrong is bigger.

The same will happen if you crate a zip or rar on a windows system, put a special characters in the filename en unzip or unrar it on a utf-8 system. Some character will even create filenames that hardly can be removed from the linux filesystem.

Normal ascii characters don't have a problem because they are the same on both systems (both 1 byte), but special characters can create problems because on a utf-8 system they are represented by 2x 8bit(= 2 bytes) and on a windows system still as an one 8 bit(=1byte) character witch the means something else when different code pages are used.

So you see this has nothing to do with packages nor with applications

[ymartin59]

@BenjV OK... As nzb does not specify what file name encoding/code page is, it make sense application may check file names with expected file system encoding. I wonder if Mono/C# API expects file names to be provided as UTF-8... probably no, or else an exception would be thrown.

[BenjV]

NZB's must be unparred. The filename is inside the compressed file and during the unpar applications (just as unzip and unrar) there is no way to do this, because the unpar function does not has this functionality.

In my own python application (which can download .zip files) I created the following solution for this. I unzip the filename as a byte stream into memory and analyse it with chardet to see if it is utf-8 or windows coded and convert it depending on the outcome to utf-8. Unparren cannot do this, it just creates file from the bytestream and that is the problem if it is a windows coded name it will just write the bytes to the filesystem and the filesystem interpret them as utf-8 and so it can contain not allowed chars.

There is no unpar application who can do that.

This is a well known probleem for compressed files created on windows and uncompressed on utf-8 systems. The uncompress utils just uncompress because they do not know how the compressed file is created.

They should of course check if the filename they create is allowed on the system but they just do not do that.

[LaChoz]

Hey guys,

Updating the post. I've also got a permission issue on my Sonarr. I tried several things :

uninstall / reinstall with backup restore convert ACL download and video folder check permission and allow read/write to sc-download on download and video folders uninstall / reinstall without restoring backup no success, I still get in my log things like : ExtraService failed while processing "access to path /Volume1..../show folder" is denied

I'm desperate for a solution ...

Thx for any help

[BenjV]

Giving access right via sc-download does nothing for existing folders and files. You have to check what the owner of those folders and files are and add that owner to the group sc-download.

[LaChoz]

Okayyyy it worked !!

Thx very much and sorry for the multiple post question I was hoping for a better chance of getting a solution !

Thx again.

[malanden]

I have found that if I go into FileStation, select the folder, right click, select properties, go to the permission tab, select SC-Download, click Apply to this folder and sub folders, this ends up working, and within a few seconds, Sonarr sees the file is complete and moves it.

Even though it already has read/write access, forcing this again solves the issue. Does that help anyone figure out whats going on?

[BenjV]

That's the same solution.

[LtMarx]

I have the exact same problem and am getting a bit desparate. What i tried sofar:

• Gave sc-download and sc-media permissions to all my download folders and destination folders
• sabnzbd is owner of download, complete and incomplete, sc-download has group permissions
• Got a folder for series, sc-download has group permissions, there seem to be multiple owners
• tried adding nzbdrone, sabnzbd user to sc-download and sc-media via telnet session (synogroup command)
• by doing this sonarr now does see the files if i select to manually import them but gives the access denied error

anyone can point me in the right direction?

[BenjV]

Can you post the output of these commands here?

cat /etc/passwd cat /etc/group

[koenvanzuijlen]

Hi @BenjV ,

Does adding the owner of the folders to sc-download fix this problem for future imports or only for current ones?

[BenjV]

Depend on how the files are created. The file owner is the application that creates the files on the NAS. So if you put that owner in the group all new files will be fine.

But normally files are created with the same privileges as the folder they are created in (inherite privs)unless the application explicite change those privs.

So if the top folder and all subfolders are given access rights to the sc-download group normally all will be ok for new folders and files.

[yarez0]

Same here, get desperate

nzbget creates folders with the rights permission et those permitions are inherited from the top folder but when nzbget wrote the final file when completed, all the inherited permissions are broken, juste sc-nzbget et nzbget user / group have rights on it.

I'm confused because some times downloads, transfert, all process go fine and sometimes everything goes wrong and sonarr can't process any download

[BenjV]

No nzbget does not change the inherited permissions unless you changed the umask which you should no do. You have to give the sc-download and sc-media groups permissions on the top level folder and set for existing folders and files this with FileStation recusively. You cannot work with the users themself anymore.

[ymartin59]

As a reference, please read documentation about permission management: https://github.com/SynoCommunity/spksrc/wiki/Permission-Management

[yarez0]

already did, I didn't change the umask.

just now, I received an email from sonarr

xxx downloaded and sorted

fine, so I decided to check if some files are locked and yes it is.

Unable to parse media info from file yyy

I don't undertand why some files are good and some are not. as I said, if I check permission on top shared folder sc-nzbget is the owner, sc-media and sc-download has full RW permission.

under top folder there is a COMPLETED foder, same, sc-nzbget is the owner, sc-media and sc-download has RW permission.

under, SERIES folder, same again

under, XXXXX folder, same again

finaly under XXXXX folder, XXXXX.mk file, sc-nzbget is the owner, all previous permissions droped, only sc-nzbget and nzbget have personnalized permissions :

sc-nzbget : READ permission without specific "cross folder / execute file" WRITE permission without specific "delete"

nzbget : READ permission without specific "cross folder / execute file" no WRITE permission

[koenvanzuijlen]

Changed my folder owners as follows:

downloads -> group: sc-download, user:sc-nzbget movies -> group: sc-download, user:sc-radarr tvshows -> group: sc-download, user:sc-nzbdrone

But imports are still failing.

[the-lazy-fox]

Just jumping in that thread.

Keep in mind that a manual import works 100% of the time, so I don't see how the permissions are different?

I had a kind of similar issue in the past and it's important to remember that it's not just an import (automatic or manual). Depending of your configuration, it might be a copy, a move, an hardlink or even a move to trash if the file is an upgrade of another one. In my case, the account didn't had permissions on the recycle folder... to move previous versions inside :)

[LtMarx]

Right, i solved my problem. I did these things: https://github.com/SynoCommunity/spksrc/issues/3256#issuecomment-383360008

I added the group sc-download to my movie and series folder (and had to do so manually via Filestation). But sc-download didn't have RW rights to all those folders. Once i gave sc-download RW right to all (sub)folders sonarr started crunching releases

[koenvanzuijlen]

@LtMarx Does this also work for new imports?