Understand why torrentexpander-CSI don't work

[GoogleCodeExporter]

Over 100 download of working version.
Only few search on scraper.
The users is not the problem. It can't be.
So...what's the problem?

1) First problem:
tar issue on A100 devices.
http://www.networkedmediatank.com/showthread.php?tid=56777&pid=527543#pid527543
I will solve it now. I know how to do.

2) (hypothesis) Missing depandancies.
How to solve?
Add self-check of all TorrentExpander required software (cat, cp, mv, unrar, 
unzip, ...). If something is missing, add to torrentexpander_debug.

3) Users are lazy
They don't report if the program don't works. We could add an 'send error 
report by email' future? (Enabled by default)

Original issue reported on code.google.com by login...@gmail.com on 14 Dec 2011 at 5:15

[GoogleCodeExporter]

First problem solved, i hope. Waiting confirmation.

Original comment by login...@gmail.com on 14 Dec 2011 at 5:49

[GoogleCodeExporter]

I am planning to go a bit further

I was already planning to check all binaries and fetch their path on first run.
Also, I was planning to replace all standard commands by the corresponding full 
path variable.

This should make torrentexpander cron friendly and should solve all common 
issues.
I will get on it this week-end

What if a script download script is already indicated in the transmission json 
script. It looks like your installer will fail to update the line.
Your sed line seems to look for : "script-torrent-done-filename": ""

Original comment by addicted...@gmail.com on 14 Dec 2011 at 6:17

[GoogleCodeExporter]

Can't understand last 3 lines.
My installer replace "script-torrent-done-filename": "" with full tx path on 
installation and restore it when removed. 

Updating torrentexpander through CSI is done by automatically removing old 
version and installing new. So the line were changed 3 times. 
Installing:
blanck --> full-path
Updating
full-path --> blanck
blanck --> full-path

Original comment by login...@gmail.com on 14 Dec 2011 at 7:59

[GoogleCodeExporter]

Here's my point :
What if  I already had a unrar script path in my transmission json settings 
path ?
Sed wouldn't find the line when looking for "script-torrent-done-filename": ""

Also, are you sure you shouldn't edit the line to 
"script-torrent-done-filename": 
"\/share\/Apps\/TorrentExpander\/bin\/torrentexpander.sh 
$TR_TORRENT_DIR/$TR_TORRENT_NAME" instead of "script-torrent-done-filename": 
"\/share\/Apps\/TorrentExpander\/bin\/torrentexpander.sh" only ?

Original comment by addicted...@gmail.com on 14 Dec 2011 at 10:47

[GoogleCodeExporter]

1) In this case setup don't work.
2) MmMmm....
I forget it...
But when i have tryed with a real torrent over transmission, program worked 
well.
I'm going to edit.

Original comment by login...@gmail.com on 14 Dec 2011 at 11:06

[GoogleCodeExporter]

Also, i'm not sure that is the right way:

    /share/Apps/Transmission/daemon.sh stop
    wait

I need to wait until application is fully stopped. Is there a better way?
I don't like this way:

    /share/Apps/Transmission/daemon.sh stop
    sleep 4

Original comment by login...@gmail.com on 14 Dec 2011 at 11:27

[GoogleCodeExporter]

About comment nr. 4, could you send me correct sed command?

this don't works:
cat /share/Apps/Transmission/config/settings.json | sed 
's/"script-torrent-done-enabled": false,/"script-torrent-done-enabled": 
true,/g' |  sed 's/"script-torrent-done-filename": 
"",/"script-torrent-done-filename": 
"\/share\/Apps\/TorrentExpander\/bin\/torrentexpander.sh 
$TR_TORRENT_DIR\/$TR_TORRENT_NAME",/g'> 
/share/Apps/Transmission/config/settings.json

Original comment by login...@gmail.com on 15 Dec 2011 at 1:04

[GoogleCodeExporter]

Hi

I was thinking of a line that looks a bit like this. Give it a try.
cat /share/Apps/Transmission/config/settings.json | sed 's/^\( 
*\)"script-torrent-done-enabled"\: .*$/\1"script-torrent-done-enabled": 
true,/g' |  sed 's/^\( *\)"script-torrent-done-filename": 
.*$/\1"script-torrent-done-filename": 
"\/share\/Apps\/TorrentExpander\/bin\/torrentexpander.sh 
\$TR_TORRENT_DIR\/\$TR_TORRENT_NAME",/g'> 
/share/Apps/Transmission/config/settings.json

Let me know how it works for you

Original comment by addicted...@gmail.com on 15 Dec 2011 at 6:21

[GoogleCodeExporter]

Regarding the issue you're facing with the daemon, I may have an idea.
Is there a file i could check against (availability, write permission, etc...) ?

Original comment by addicted...@gmail.com on 15 Dec 2011 at 7:19

[GoogleCodeExporter]

The line provided works well. Please send 'revert' line also.

No useful file found, could you use 'pgrep'? 
i have this idea, we could stop the daemon and use this string to confirm:

while [ $(pgrep transmission) != "" ]; then
  sleep 1

Original comment by login...@gmail.com on 15 Dec 2011 at 8:30

[GoogleCodeExporter]

Here are the lines I suggest

Install :
cat /share/Apps/Transmission/config/settings.json | sed 's/^\( 
*\)\"script-torrent-done-enabled\"\: .*$/\1\"script-torrent-done-enabled\"\: 
true\,/g' |  sed 's/^\( *\)\"script-torrent-done-filename\"\: 
.*$/\1\"script-torrent-done-filename\"\: 
\"\/share\/Apps\/TorrentExpander\/bin\/torrentexpander.sh 
\$TR_TORRENT_DIR\/\$TR_TORRENT_NAME\"\,/g' > 
/share/Apps/Transmission/config/settings.json

Uninstall :
cat /share/Apps/Transmission/config/settings.json | sed 's/^\( 
*\)\"script-torrent-done-enabled\"\: .*$/\1\"script-torrent-done-enabled\"\: 
false\,/g' |  sed 's/^\( *\)\"script-torrent-done-filename\"\: 
.*$/\1\"script-torrent-done-filename\"\: \"\",/g' > 
/share/Apps/Transmission/config/settings.json

Exit transmission :
/share/Apps/Transmission/daemon.sh stop
while [ $(pgrep transmission) ]; do sleep 1; done

Also, maybe you should make sure all users are running transmission at root 
level and not the nmt user.
If they run as nmt, they may face unexpected results with the permissions set 
in the settings.ini file included in the installer.

Original comment by addicted...@gmail.com on 16 Dec 2011 at 9:50

[GoogleCodeExporter]

Thanks. is it possible to have timeout on while command?

Original comment by login...@gmail.com on 17 Dec 2011 at 12:23

[GoogleCodeExporter]

Example with 60 seconds

count=0
while [ $(pgrep transmission) ] && [[ $count -lt 60 ]]; do
    sleep 1;
    count=$(( count + 1 )); 
done

Original comment by addicted...@gmail.com on 17 Dec 2011 at 9:40

[GoogleCodeExporter]

Please help me. It's a long time i'm studing this bug, but i have never found a 
solution. There is this random error when script edit the config file.

How to reproduce:
Remove Transmission & TorrentExpander
Restart the unit.
Install Transmission
put setup.sh where you want and run many times:
./setup.sh install
./setup.sh uninstall

After some times, you will see an empty config file with uncorrect permissions 
set.

Example:
http://pastebin.com/3EY5tPYV

Original comment by login...@gmail.com on 17 Dec 2011 at 3:07

[GoogleCodeExporter]

Original comment by login...@gmail.com on 17 Dec 2011 at 3:09

• Changed state: Accepted
• Added labels: Priority-Critical
• Removed labels: Priority-High

Attachments:

• setup.sh

[GoogleCodeExporter]

Hi
What config file ends up empty ?
torrentexpander_settings.ini or settings.json ?
Maybe the cleanest and safest way is to preserve the user's settings.json file 
(i.e. cp settings.json settings.json.old) and revert on uninstall.
Also, editing a file using cat and sed may not be the best idea (if cat fails 
to open the file, you may replace it with a blank file).
Maybe this is safer

Install :
if [ ! -f /share/Apps/Transmission/config/settings.json.bak ]; then
    cp /share/Apps/Transmission/config/settings.json /share/Apps/Transmission/config/settings.json.bak
fi
sed -i 's/^\( *\)\"script-torrent-done-enabled\"\: 
.*$/\1\"script-torrent-done-enabled\"\: true\,/g' 
/share/Apps/Transmission/config/settings.json
sed -i 's/^\( *\)\"script-torrent-done-filename\"\: 
.*$/\1\"script-torrent-done-filename\"\: 
\"\/share\/Apps\/TorrentExpander\/bin\/torrentexpander.sh 
\$TR_TORRENT_DIR\/\$TR_TORRENT_NAME\"\,/g' 
/share/Apps/Transmission/config/settings.json

Uninstall :
if [ -f /share/Apps/Transmission/config/settings.json ] && [ -f 
/share/Apps/Transmission/config/settings.json.bak ]; then
   rm /share/Apps/Transmission/config/settings.json && mv /share/Apps/Transmission/config/settings.json.bak /share/Apps/Transmission/config/settings.json
fi

Original comment by addicted...@gmail.com on 17 Dec 2011 at 11:43

[GoogleCodeExporter]

Thanks, it seems to work. Next CSI version will have a 'nmj_update.sh' in bin 
folder, ready for Issue 13 in the future.

Original comment by login...@gmail.com on 18 Dec 2011 at 12:56

[GoogleCodeExporter]

Hi
I'm glad it works now
I just committed a new SVN revision. You can add the following two lines to the 
torrentexpander_settings.ini file
post_run_script_enabled=yes
post_run_script=/share/Apps/TorrentExpander/bin/nmj_update.sh

This is a very basic implementation for now until I find a way to polish it a 
bit. It can only fetch a binary or script's name in your PATH directory (and a 
few additional others) or run a script or binary from a specific path and run 
it.
In the future, I'll need to compile a list of essential binaries and scripts 
users may want to run once torrentexpander is done and also find an smart way 
to pass variables.

Original comment by addicted...@gmail.com on 18 Dec 2011 at 9:30

[GoogleCodeExporter]

Very Good. For the future, have a look to oversight sample script:

# Scanning with Oversight
cd /share/Apps/oversight/bin
./oversight.sh NEWSCAN "$TR_TORRENT_DIR"

Original comment by login...@gmail.com on 18 Dec 2011 at 11:00

[GoogleCodeExporter]

released new csi version. I hope that cat command was the problem

Original comment by login...@gmail.com on 18 Dec 2011 at 11:41

• Added labels: Priority-Low
• Removed labels: Priority-Critical

[GoogleCodeExporter]

Original comment by login...@gmail.com on 18 Dec 2011 at 9:42

• Changed state: Fixed

[GoogleCodeExporter]

Another problem:
http://www.networkedmediatank.com/showthread.php?tid=56777&pid=529670#pid529670

Original comment by login...@gmail.com on 23 Dec 2011 at 11:20

• Changed state: Accepted
• Added labels: Priority-Critical
• Removed labels: Priority-Low

[GoogleCodeExporter]

Hi
Can you please ask this user what his download path is.
This will point us in the right direction
Thanks

Original comment by addicted...@gmail.com on 23 Dec 2011 at 12:19

[GoogleCodeExporter]

founded users problems:

1) debug log of torrentexpander says:
./setup.sh: line 17: [: too many arguments

2) Missing depandancies:
opkg install commands fails!!

Original comment by login...@gmail.com on 27 Dec 2011 at 9:42

[GoogleCodeExporter]

I don't see anything that could cause such an issue in line 17 of your setup 
script
I'm not familiar with case/esac structure. 
If you need help rewriting the setup script, I could do it, but I would then 
use some more basic commands (i.e. replace case/esac with some if/fi routines 
and assign 2 values to $1 : install or uninstall)

Thanks for finding the source of this issue, I would never have been able to 
track that down.

Original comment by addicted...@gmail.com on 27 Dec 2011 at 9:59

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

line 17:
while [ $(pgrep -f php5server) ] && [[ $count -lt 60 ]]; do

what happens if pgrep found more than one process? ;-)
Is this the origin of error?

Original comment by login...@gmail.com on 27 Dec 2011 at 11:32

[GoogleCodeExporter]

I must improve debug on setup.sh.
Please complete this script using wget command only (because it must not 
depends to busybox)

    if [ "$(wget ???????????????? ) ]; then
        internet="yes"
    else
        internet="no"
        echo 'Warning: no internet connection'
    fi

Original comment by login...@gmail.com on 28 Dec 2011 at 12:13

[GoogleCodeExporter]

if [ "$(echo "$(wget -q "http://sourceforge.net" -O - | grep -o 
"SourceForge.net"; wait)")" ]; then
        internet="yes"
    else
        internet="no"
        echo 'Warning: no internet connection'
    fi

But you could also use these lines prior to installing every package
if [ ! "$(opkg list-installed | grep "unzip")" ] && [ "$internet" == "yes" ]; 
then ...

Afterwards, 
if [ ! "$(opkg list-installed | grep "unzip")" ] || [ ! "$(opkg list-installed 
| grep "unrar")" ] || [ ! "$(opkg list-installed | grep "wget")" ] || [ ! 
"$(opkg list-installed | grep "busybox")" ]; then...
In this case, we can return an error message saying torrentexpander did not 
install correctly

Regarding php5server, all processes should be killed by the previous command
If not, we could force quit the process on timeout ?

Original comment by addicted...@gmail.com on 28 Dec 2011 at 9:17

[GoogleCodeExporter]

Sorry, we can not use grep command in this debug lines, because it is a part of 
busybox package. No busybox --> no debug output. See 'ls -al /bin/' on nmt 
device for avaiable commands (excluding busybox entryes).

Original comment by login...@gmail.com on 29 Dec 2011 at 12:27

[GoogleCodeExporter]

The main idea for now is about adding a new function on setup.sh, called 
'presetup'. This function do some things:

1) Check if opkg is installed.
2) Check if there is an internet connection.
3) Install depandancies
4) Check if depandancies are installed.
5) Start setup if [4] are OK.

Original comment by login...@gmail.com on 29 Dec 2011 at 12:35

[GoogleCodeExporter]

This is so frustrating
All I could think of in order to detect internet connection are ping and wget. 
Both are linked to busybox

if ! ping sourceforge.net -c 1 -w 5 > /dev/null 2>&1; then
if ! wget -q "http://sourceforge.net" -O - --timeout 5 > /dev/null 2>&1; then

In order to detect binaries, all I could think of was (and both find and which 
seem linked to busybox)
if [ ! -x "$(which unrar)" ]; then
if [ ! "$(find /bin /usr/bin /usr/local/bin -name "unrar")" ]; then

I'm scratching my head thinking of other solutions, hoping I'll find some new 
ideas soon.

Original comment by addicted...@gmail.com on 29 Dec 2011 at 7:47

[GoogleCodeExporter]

You should use wget! It's included in the firmware. Have a look:
ls -al /bin/wget
lrwxrwxrwx    1 root     root            21 Jan  1  2000 /bin/wget -> 
/opt/syb/app/bin/wget

If you need an idea, have a look to this:
http://repository.nmtinstaller.com/appinit.cgi

appinit_auto_upgrade() is a function that check for an internet connection.

Original comment by login...@gmail.com on 29 Dec 2011 at 11:36

[GoogleCodeExporter]

Don't use which for setup script:

ls -al /usr/bin/which
lrwxrwxrwx    1 516      root            17 Jan  1  2000 /usr/bin/which -> 
../../bin/busybox

Original comment by login...@gmail.com on 29 Dec 2011 at 11:39

[GoogleCodeExporter]

I think this is the right way:
wget -qO- http://repository.nmtinstaller.com/appinit_version

Original comment by login...@gmail.com on 29 Dec 2011 at 12:01

[GoogleCodeExporter]

Updated setup.sh
It works well for me.

Original comment by login...@gmail.com on 29 Dec 2011 at 2:24

Attachments:

• setup.sh

[GoogleCodeExporter]

New version is out. See forum for changes.

Original comment by login...@gmail.com on 29 Dec 2011 at 8:53

• Added labels: Priority-Medium
• Removed labels: Priority-Critical

[GoogleCodeExporter]

Tried to revert my NMT C200 to its out of the box setup then installed this 
package.
All dependencies have been installed correctly and everything seems to work 
fine.

Let's cross our fingers hoping it works fine for all users.

I'll now take care of issue 39

Original comment by addicted...@gmail.com on 29 Dec 2011 at 9:32

[GoogleCodeExporter]

As i suspect....
We need to replace all binaries with their proper path, as you have written at 
comment 2.

Look this output: the user has not correct $PATH, torrentexpander will never 
work without it. 
http://www.networkedmediatank.com/showthread.php?tid=56777&pid=531869#pid531869

Original comment by login...@gmail.com on 1 Jan 2012 at 11:20

• Added labels: Priority-Critical
• Removed labels: Priority-Medium

[GoogleCodeExporter]

I started doing that until I realized that this is not the correct way to do it
At the beginning of the script I would rely on some commands like find, ls and 
so on in order to guess the path to binaries. If those commands are not in 
$PATH either, then we're screwed

The smart option would be to save the user's $PATH then edit it and do the same 
in your install script

# SCRIPT START
PATH_BACKUP="$PATH"
for test_path in $(echo -e 
"/usr/local/sbin\n/usr/local/bin\n/usr/sbin\n/usr/bin\n/sbin\n/bin\n/usr/games\n
/usr/bin\n/bin\n/usr/sbin\n/sbin\n/usr/local/bin\n/usr/X11/bin\n/Applications\n/
opt/syb/sigma/bdj/jvm/bin\n/usr/bin/X11\n/opt/syb/app/bin\n/opt/syb/app/sbin\n/o
pt/syb/sigma/bin\n/nmt/apps"); do
if [[ -d "$test_path" && "$PATH" != *$test_path* ]]; then 
PATH="$PATH:$test_path"; fi;
done
export PATH="$PATH"
...
...
...
export PATH="$PATH_BACKUP"
# SCRIPT END

WHat do you think of that ?

Original comment by addicted...@gmail.com on 2 Jan 2012 at 5:58

[GoogleCodeExporter]

Looks good, but i have done dozen of times the command 'export PATH'. After 
restart, i have lost all changes. 

Original comment by login...@gmail.com on 2 Jan 2012 at 11:15

[GoogleCodeExporter]

That's why these lines have to be added to my script as well as your installer.
The path variable will be replaced then restored everytime the script runs
I'll commit a new SVN tonight with these lines as well as a few tweaks

Original comment by addicted...@gmail.com on 2 Jan 2012 at 12:20

[GoogleCodeExporter]

Ok, but i think that it must exist a way to permanently edit $PATH. 

Original comment by login...@gmail.com on 2 Jan 2012 at 3:35

[GoogleCodeExporter]

Siriam says:
"
Probably he was using A110/100 which does not contain the above path default. 
It can be added in the start_app.sh 
"

Original comment by login...@gmail.com on 2 Jan 2012 at 3:36

[GoogleCodeExporter]

In order to do that permanently, we'd need to edit each user's /etc/bashrc
Doing changing a variable on every run is a better option

Original comment by addicted...@gmail.com on 2 Jan 2012 at 3:59

[GoogleCodeExporter]

This feature has been added to the latest SVN build
We'll see if it works well from user feedback

Original comment by addicted...@gmail.com on 2 Jan 2012 at 6:45

[GoogleCodeExporter]

/etc/bashrc do not exist
/root/.bashrc exist
anyway nmt user do not have any .bashrc

Original comment by login...@gmail.com on 3 Jan 2012 at 11:06

[GoogleCodeExporter]

TEST CODE:

    # Try to correct paths
    PATH_BACKUP="$PATH"
    for test_path in $(echo -e "/usr/local/sbin\n/usr/local/bin\n/usr/sbin\n/usr/bin\n/sbin\n/bin\n/usr/games\n/usr/bin\n/bin\n/usr/sbin\n/sbin\n/usr/local/bin\n/usr/X11/bin\n/Applications\n/opt/syb/sigma/bdj/jvm/bin\n/usr/bin/X11\n/opt/syb/app/bin\n/opt/syb/app/sbin\n/opt/syb/sigma/bin\n/nmt/apps"); do
        if [[ -d "$test_path" && "$PATH" != *$test_path* ]]; then 
        PATH="$PATH:$test_path"; 
        STRING="Information: adding $test_path your paths list"
        echo $STRING
        fi
    done
    export PATH="$PATH"

Original comment by login...@gmail.com on 3 Jan 2012 at 12:39

[GoogleCodeExporter]

But there is an error: i have run it many times, but it always say:
Information: adding /nmt/apps your paths list

echo $PATH
/opt/syb/sigma/bdj/jvm/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin/X11:/usr/local
/bin:/opt/syb/app/bin:/opt/syb/app/sbin:/opt/syb/sigma/bin

Original comment by login...@gmail.com on 3 Jan 2012 at 12:43

[GoogleCodeExporter]

Another try, i have set $PATH=/bin, then i have run the script.

Information: adding /usr/local/bin to your paths list
Information: adding /usr/sbin to your paths list
Information: adding /usr/bin to your paths list
Information: adding /opt/syb/sigma/bdj/jvm/bin to your paths list
Information: adding /opt/syb/app/bin to your paths list
Information: adding /opt/syb/app/sbin to your paths list
Information: adding /opt/syb/sigma/bin to your paths list
Information: adding /nmt/apps to your paths list

but then... echo $PATH
/bin

Original comment by login...@gmail.com on 3 Jan 2012 at 12:59