'no rar files extracted' error

[mkendrick]

Hi. Found your script and have been attempting to get it to work on a FreeBSD-based NAS (FreeNAS).

Having some trouble, the script doesn't seem to find any rar files. Sample output:

[root@freenas] /mnt/vol1/tv/show# /mnt/vol1/backup/unrarall -d -v -s --clean=all /mnt/vol1/tv/show/test Detecting clean up hooks... Found unrarall-clean-covers_folders Found unrarall-clean-empty_folders Found unrarall-clean-nfo Found unrarall-clean-osx_junk Found unrarall-clean-rar Found unrarall-clean-sample_folders Found unrarall-clean-sample_videos Found unrarall-clean-windows_junk Using virtual clean-up hook all Using "echo" to extract rar files Working over directory "/mnt/vol1/tv/show/test" no rar files extracted

There are, of course, many subfolders in my 'test' dir, all of which contain multi-part rar archives:

[root@freenas /mnt/vol1/tv/show/test/show.S07E13.HDTV.XviD]# ls show.s07e13.hdtv.xvid.r03 show.s07e13.hdtv.xvid.r08 show.s07e13.hdtv.xvid.r13 show.s07e13.hdtv.xvid.nfo show.s07e13.hdtv.xvid.r04 show.s07e13.hdtv.xvid.r09 show.s07e13.hdtv.xvid.r14 show.s07e13.hdtv.xvid.rar ... etc (yes, it's a complete rar set.)

Grateful any assistance. The default shell for FreeNAS is csh, however I get the same result when I run bash and then run the script. Note that if I run without the -d switch, I get the same end result, however it does report that it's found the unrar executable:

[root@freenas] /mnt/vol1/tv/show# /mnt/vol1/backup/unrarall -v -s --clean=all /mnt/vol1/tv/show/test Detecting clean up hooks... Found unrarall-clean-covers_folders Found unrarall-clean-empty_folders Found unrarall-clean-nfo Found unrarall-clean-osx_junk Found unrarall-clean-rar Found unrarall-clean-sample_folders Found unrarall-clean-sample_videos Found unrarall-clean-windows_junk Using virtual clean-up hook all Looking for unrar...found Using "unrar" to extract rar files Working over directory "/mnt/vol1/tv/show/test" no rar files extracted

other miscellanea: UNRAR 4.20 freeware Copyright (c) 1993-2012 Alexander Roshal unrarall script version 0.4.2

[delcypher]

@mkendrick That's weird. the -d flag shouldn't change what rar files are detected.

Does the output of this command show rar files?

$ find  /mnt/vol1/tv/show/test/ -depth -iregex '.*\.\(rar\|001\)$'

this is what we use to find the rar files in the script.

[mkendrick]

Note that regardless of whether I use -d or not, no rar files are detected.

The find command above does not return any results.

[mkendrick]

a normal 'find' does locate the files though: [root@freenas /mnt/vol1/tv/show/test]# find ./ -name '*.rar' ./show.S07E10.HDTV.XviD/show.s07e10.hdtv.xvid.rar ./show.S07E03.PROPER.HDTV.XviD/show.s07e03.proper.hdtv.xvid.rar ./show.S07E12.HDTV.XviD/show.712.hdtv.rar ... etc

[mkendrick]

perhaps the version of 'find' is different on freeBSD? unfortunately FreeNAS doesn't install any man pages, I can't tell you what switches are supported beyond what it gives me in an error message:

[root@freenas /mnt/vol1/tv/show/test]# find usage: find [-H | -L | -P] [-EXdsx] [-f path] path ... [expression] find [-H | -L | -P] [-EXdsx] -f path [path ...] [expression]

looks like it supports -depth and -iregex, maybe there's an issue with the regex itself? (i'm definitely not a regex guru.)

[mkendrick]

I can find rar files with the following:

find ./ -depth -iregex '.*\.\rar'

but not this:

'find ./ -depth -iregex '.*\.\001'

note that I don't have any *.001 files on my volume. Could I just change the script to look for rar instead of (rar|001)? Would that affect the cleanup of .rar/.r01/.r02 etc?

[delcypher]

The GNU version of find uses the emacs style of regex by default. Here's the man page for free BSD find

Could you try adding the -E flag and see if that fixes your regex issue?

[delcypher]

In fact it looks like in #17 someone else needed this. Is there a nice way detect the version of find being used?

I suppose we could look at the output of uname? What is the output of that command on your FreeNAS?

We could use that to add the -E flag if we detect FreeBSD.

[mkendrick]

unfortunately it didn't help.

 find -E ./ -depth -iregex '.*\.\(rar\|001\)$'

also returned no results

uname returned 'FreeBSD'.

[delcypher]

I think the "extended" regex syntax is different from the emacs style. I think the brackets shouldn't be quoted

Does this work?

find -E ./ -depth -iregex '.*\.(rar\|001)$'

[mkendrick]

Nope, still no results. On the plus side, changing the script function to only look for .rar instead of .rar or .001 seems to have done the trick. I'm happy to help you debug though if you like.

[mkendrick]

Found it - you were almost there. The pipe doesn't need to be escaped either:

[root@freenas /mnt/vol1/tv/show/test]# find -E ./ -depth -iregex '.*\.(rar|001)$'
./show.S07E10.HDTV.XviD-ASAP/show.s07e10.hdtv.xvid-asap.rar
./show.S07E03.PROPER.HDTV.XviD-ASAP/show.s07e03.proper.hdtv.xvid-asap.rar
./show.S07E12.HDTV.XviD-LOL/show.712.hdtv-lol.rar
...etc

nice one!

[mkendrick]

edited the script and ran it over the test directory, it extracted everything but didn't clean up the rar files. Will go back and check out the hook function and see what needs to be done there.

[delcypher]

Unfortunately you're going to run into more problems because we also use find to clean many other files up. see the unrarall-clean-rar, unrarall-clean-sample_videos and find-regex-escape functions.

One way of fixing this is to use the POSIX extended regex syntax always. In GNU find I think we would pass the command line option -regextype posix-extended.

@arfoll If we're going to make this change I think we should setup a test suite first to make sure all the hooks work so we don't accidently break something.

[mkendrick]

I did get it cleaning up rar files, but yeah, it's left behind some .nfo files, and presumably would have left behind other things if they had been there.

Happy to do a bit of testing for you if you like.

[arfoll]

@delcypher check the check_unrarall branch it's pretty dirty but should stop basic silly mistakes. The big issue I can see is that getting people to run this to detect problems will be hard as we require rar which on freebsd/busybox type stuff might be difficult. We could wget stuff down instead :/

Should make testing find options a bit easier.

[ashleyconnor]

If you are on OSX it might just be easier to install GNU findtools:

brew install findutils --with-default-names

Just make sure you unlink them after as it might cause issues.

[xieem]

@ashleyconnor how do you unlink them in the best way and what issues can they cause?

[ashleyconnor]

The only reason to unlink is that some other program that might not be expecting GNU find but find that comes on OSX.

To unlink run brew unlink findutils which will delete the symlinks out of your path. To relink just run brew link findutils again. You will probably need to reload your shell to make sure your paths are updated.

Unlinking shouldn't cause any issues unless another program depends on GNU find.

My output is below (note your homebrew path is likely to be different):

➜  aconnor  which find
/opt/boxen/homebrew/bin/find
➜  aconnor  brew unlink findutils
Unlinking /opt/boxen/homebrew/Cellar/findutils/4.4.2... 12 symlinks removed
➜  aconnor  which find
/usr/bin/find

[mattmackenzie]

In 10.10.3, I just replaced all of the 'find' commands with 'gfind' from Homebrew

[Roedy86]

Hi, i had the same issue as mentioned above. (Freebsd 10.0 amd64) I edited the file as described. added a -E after find and removed all backslashes in except the first in all find commands.

[alatas]

I tried @ashleyconnor solution to freebsd / freenas and succeed compiled findutils from ports ( https://www.freshports.org/misc/findutils/ )

cd /usr/ports/misc/findutils/ && make install clean

then change the script to gfind and it works :+1:

[delcypher]

Is this still an issue with the current master? If not I'll close this issue.

If someone would like to refactor the way we use find so that it works wtih macOS/*BSD find and GNU find that would be great. We have a test suite now so this should be way easier to test.

[delcypher]

In f00f48b09372734a048149178596ed9a90430b75 I taught unrarall to try and handle the different versions of find and use egrep style regular expressions so I'm closing. If this is still a problem please report the issue.

[xieem]

@delcypher seems to be working 100% here no more issue for me