arfoll / unrarall

bash script to unrar everything and cleanup in a given directory
GNU General Public License v3.0
260 stars 68 forks source link

'no rar files extracted' error #18

Closed mkendrick closed 7 years ago

mkendrick commented 9 years ago

Hi. Found your script and have been attempting to get it to work on a FreeBSD-based NAS (FreeNAS).

Having some trouble, the script doesn't seem to find any rar files. Sample output:

[root@freenas] /mnt/vol1/tv/show# /mnt/vol1/backup/unrarall -d -v -s --clean=all /mnt/vol1/tv/show/test Detecting clean up hooks... Found unrarall-clean-covers_folders Found unrarall-clean-empty_folders Found unrarall-clean-nfo Found unrarall-clean-osx_junk Found unrarall-clean-rar Found unrarall-clean-sample_folders Found unrarall-clean-sample_videos Found unrarall-clean-windows_junk Using virtual clean-up hook all Using "echo" to extract rar files Working over directory "/mnt/vol1/tv/show/test" no rar files extracted

There are, of course, many subfolders in my 'test' dir, all of which contain multi-part rar archives:

[root@freenas /mnt/vol1/tv/show/test/show.S07E13.HDTV.XviD]# ls show.s07e13.hdtv.xvid.r03 show.s07e13.hdtv.xvid.r08 show.s07e13.hdtv.xvid.r13 show.s07e13.hdtv.xvid.nfo show.s07e13.hdtv.xvid.r04 show.s07e13.hdtv.xvid.r09 show.s07e13.hdtv.xvid.r14 show.s07e13.hdtv.xvid.rar ... etc (yes, it's a complete rar set.)

Grateful any assistance. The default shell for FreeNAS is csh, however I get the same result when I run bash and then run the script. Note that if I run without the -d switch, I get the same end result, however it does report that it's found the unrar executable:

[root@freenas] /mnt/vol1/tv/show# /mnt/vol1/backup/unrarall -v -s --clean=all /mnt/vol1/tv/show/test Detecting clean up hooks... Found unrarall-clean-covers_folders Found unrarall-clean-empty_folders Found unrarall-clean-nfo Found unrarall-clean-osx_junk Found unrarall-clean-rar Found unrarall-clean-sample_folders Found unrarall-clean-sample_videos Found unrarall-clean-windows_junk Using virtual clean-up hook all Looking for unrar...found Using "unrar" to extract rar files Working over directory "/mnt/vol1/tv/show/test" no rar files extracted

other miscellanea: UNRAR 4.20 freeware Copyright (c) 1993-2012 Alexander Roshal unrarall script version 0.4.2

delcypher commented 9 years ago

@mkendrick That's weird. the -d flag shouldn't change what rar files are detected.

Does the output of this command show rar files?

$ find  /mnt/vol1/tv/show/test/ -depth -iregex '.*\.\(rar\|001\)$'

this is what we use to find the rar files in the script.

mkendrick commented 9 years ago

Note that regardless of whether I use -d or not, no rar files are detected.

The find command above does not return any results.

mkendrick commented 9 years ago

a normal 'find' does locate the files though: [root@freenas /mnt/vol1/tv/show/test]# find ./ -name '*.rar' ./show.S07E10.HDTV.XviD/show.s07e10.hdtv.xvid.rar ./show.S07E03.PROPER.HDTV.XviD/show.s07e03.proper.hdtv.xvid.rar ./show.S07E12.HDTV.XviD/show.712.hdtv.rar ... etc

mkendrick commented 9 years ago

perhaps the version of 'find' is different on freeBSD? unfortunately FreeNAS doesn't install any man pages, I can't tell you what switches are supported beyond what it gives me in an error message:

[root@freenas /mnt/vol1/tv/show/test]# find usage: find [-H | -L | -P] [-EXdsx] [-f path] path ... [expression] find [-H | -L | -P] [-EXdsx] -f path [path ...] [expression]

looks like it supports -depth and -iregex, maybe there's an issue with the regex itself? (i'm definitely not a regex guru.)

mkendrick commented 9 years ago

I can find rar files with the following:

find ./ -depth -iregex '.*\.\rar'

but not this:

'find ./ -depth -iregex '.*\.\001'

note that I don't have any *.001 files on my volume. Could I just change the script to look for rar instead of (rar|001)? Would that affect the cleanup of .rar/.r01/.r02 etc?

delcypher commented 9 years ago

The GNU version of find uses the emacs style of regex by default. Here's the man page for free BSD find

Could you try adding the -E flag and see if that fixes your regex issue?

delcypher commented 9 years ago

In fact it looks like in #17 someone else needed this. Is there a nice way detect the version of find being used?

I suppose we could look at the output of uname? What is the output of that command on your FreeNAS?

We could use that to add the -E flag if we detect FreeBSD.

mkendrick commented 9 years ago

unfortunately it didn't help.

 find -E ./ -depth -iregex '.*\.\(rar\|001\)$'

also returned no results

uname returned 'FreeBSD'.

delcypher commented 9 years ago

I think the "extended" regex syntax is different from the emacs style. I think the brackets shouldn't be quoted

Does this work?

find -E ./ -depth -iregex '.*\.(rar\|001)$'
mkendrick commented 9 years ago

Nope, still no results. On the plus side, changing the script function to only look for .rar instead of .rar or .001 seems to have done the trick. I'm happy to help you debug though if you like.

mkendrick commented 9 years ago

Found it - you were almost there. The pipe doesn't need to be escaped either:

[root@freenas /mnt/vol1/tv/show/test]# find -E ./ -depth -iregex '.*\.(rar|001)$'
./show.S07E10.HDTV.XviD-ASAP/show.s07e10.hdtv.xvid-asap.rar
./show.S07E03.PROPER.HDTV.XviD-ASAP/show.s07e03.proper.hdtv.xvid-asap.rar
./show.S07E12.HDTV.XviD-LOL/show.712.hdtv-lol.rar
...etc

nice one!

mkendrick commented 9 years ago

edited the script and ran it over the test directory, it extracted everything but didn't clean up the rar files. Will go back and check out the hook function and see what needs to be done there.

delcypher commented 9 years ago

Unfortunately you're going to run into more problems because we also use find to clean many other files up. see the unrarall-clean-rar, unrarall-clean-sample_videos and find-regex-escape functions.

One way of fixing this is to use the POSIX extended regex syntax always. In GNU find I think we would pass the command line option -regextype posix-extended.

@arfoll If we're going to make this change I think we should setup a test suite first to make sure all the hooks work so we don't accidently break something.

mkendrick commented 9 years ago

I did get it cleaning up rar files, but yeah, it's left behind some .nfo files, and presumably would have left behind other things if they had been there.

Happy to do a bit of testing for you if you like.

arfoll commented 9 years ago

@delcypher check the check_unrarall branch it's pretty dirty but should stop basic silly mistakes. The big issue I can see is that getting people to run this to detect problems will be hard as we require rar which on freebsd/busybox type stuff might be difficult. We could wget stuff down instead :/

Should make testing find options a bit easier.

ashleyconnor commented 9 years ago

If you are on OSX it might just be easier to install GNU findtools:

brew install findutils --with-default-names

Just make sure you unlink them after as it might cause issues.

xieem commented 9 years ago

@ashleyconnor how do you unlink them in the best way and what issues can they cause?

ashleyconnor commented 9 years ago

The only reason to unlink is that some other program that might not be expecting GNU find but find that comes on OSX.

To unlink run brew unlink findutils which will delete the symlinks out of your path. To relink just run brew link findutils again. You will probably need to reload your shell to make sure your paths are updated.

Unlinking shouldn't cause any issues unless another program depends on GNU find.

My output is below (note your homebrew path is likely to be different):

➜  aconnor  which find
/opt/boxen/homebrew/bin/find
➜  aconnor  brew unlink findutils
Unlinking /opt/boxen/homebrew/Cellar/findutils/4.4.2... 12 symlinks removed
➜  aconnor  which find
/usr/bin/find
mattmackenzie commented 9 years ago

In 10.10.3, I just replaced all of the 'find' commands with 'gfind' from Homebrew

Roedy86 commented 9 years ago

Hi, i had the same issue as mentioned above. (Freebsd 10.0 amd64) I edited the file as described. added a -E after find and removed all backslashes in except the first in all find commands.

alatas commented 9 years ago

I tried @ashleyconnor solution to freebsd / freenas and succeed compiled findutils from ports ( https://www.freshports.org/misc/findutils/ )

cd /usr/ports/misc/findutils/ && make install clean

then change the script to gfind and it works :+1:

delcypher commented 7 years ago

Is this still an issue with the current master? If not I'll close this issue.

If someone would like to refactor the way we use find so that it works wtih macOS/*BSD find and GNU find that would be great. We have a test suite now so this should be way easier to test.

delcypher commented 7 years ago

In f00f48b09372734a048149178596ed9a90430b75 I taught unrarall to try and handle the different versions of find and use egrep style regular expressions so I'm closing. If this is still a problem please report the issue.

xieem commented 7 years ago

@delcypher seems to be working 100% here no more issue for me