[Solved] Photorec scripted run: session not working

mvasi90 commented 4 years ago

Hello everyone,

I'm testing with photorec scripted mode but the session not working. I can't resume the recovery. Note: In interactive mode it works.

Device and software info

Distro: ArchLinux
Kernel: Linux 5.3.13-arch1-1

Photorec version:
Version: 7.1
Compiler: GCC 9.2
ext2fs lib: 1.45.4, ntfs lib: libntfs-3g, ewf lib: none, libjpeg: libjpeg-turbo-2.0.3, curses lib: ncurses 6.1, zlib: 1.2.11

bash script command:

photorec /log /logname rec.log /debug /d ./ /cmd $device fileopt,everything,disable,pdf,enable,options,paranoid,keep_corrupted_file_no,freespace,$part,search

photorec.ses scripted mode, non interactive:

#1575671443
/dev/sdb partition_i386,1,blocksize,16384,fileopt,everything,disable,pdf,enable,options,paranoid,keep_corrupted_file_no,freespace,search,status=find_offset,inter
31808-60751871

photorec.ses interactive mode:

#1575668749                                                                                                                                                         
/dev/sdb partition_i386,1,blocksize,16384,fileopt,everything,disable,pdf,enable,options,paranoid,keep_corrpted_file_no,freespace,search,status=ext2_off,30421568,inter
31808-27551487                                                                                                                                                                  
29263840-30421567                                                                                                                                                                           
30421568-60751871

Both sessions files have the keyword inter (Interactive). The partition filesystem is FAT32, why appears status=ext2_off? Where is the documentation of status=?

The tool has undocumented code.

The session file photorec.ses is deleted if the device is removed while working. This should not happen. it is exactly the same as interrupting program execution. Like the ddrescue tool, photorec should keep the session in any circumstance.

Using photorec directly on the device, instead of using ddrescue first, is a significant time saver if it is known that the device has no bad sectors or failures.

cgsecurity commented 4 years ago

Duplicate of #18 Currently PhotoRec doesn't detect that the disk is disconnected. Once the disk is disconnected, all read failed, the recovery finished without new files been recovered. As the recovery is completed, the photorec.ses file is deleted.

mvasi90 commented 4 years ago

Why you are closing it? This is not a duplicate thread. The final content matches another, nothing more.

The main problem is that photorec executed in command mode (script) overwrites the session file. Another problem is that it does not update the session status file if it is not executed in interactive mode.

Note: I'm killing the process sending TERM signal (sudo kill -15 <pid>). In theory it is identical to gracefully finish the interactive execution by clicking Stop.

Please give me more information about the status parameter. I can change it? Or it belongs to the internal behavior of the execution.

What is the way to keep the session in non-interactive mode? Running only once with parameters?

mvasi90 commented 4 years ago

Temporary solution to resume session from script

Issues photorec does not detect os signals. If the process is killed, does not update photorec.ses file. If the input device (to recover) is removed, the process deletes photorec.ses file.

Culture for windows developers: Interactive mode: the user executes the binary file and interacts with the process. Script/batch mode: the system (or the user) runs the script (almost always in background). The user can't interact with photorec process (background) without cheating/hacking.

Windows parameters are using slash /. Unix parameters minus -. Photorec uses slash on all platforms.

The Unix Programming Environment (book)

Options follow the command name on the command line, and are usually made up of an initial minus sign (-) and a single letter.

The C Programming Language (book):

A common convention for C programs on UNIX systems is that an argument that begins with a minus sign introduces an optional flag.

Current photorec version (7.1) and all the previous versions is designed to be started from script but only in foreground interactively. If you can't interact with the current recovery process you can't stop it updating the session file.

Stop background photorec recovery process saving current session (without killing it): from root user

 echo -ne "\nYq" > /proc/`pidof photorec`/fd/0

from non-root user

 echo -ne "\nYq" | sudo tee /proc/`pidof photorec`/fd/0

The concept is: send Enter, Y and q to the input of photorec process from any terminal. -n: do not output the trailing newline. (without it, echo sends \n at the end of the string). -e: enable interpretation of backslash escapes (in this case: \n. Without it echo sends the string \n instead the Enter code 0x10). \nYq explained:

\n new line to stop process (Enter).
Y key to confirm Stopaction on photorec process.
q key to quit the photorec process.

Those keys are sended to the /proc/<photorec pid>/fd/0 file descriptor. The file descriptor 0 is the input (keyboard) STDIN. Writing to the 0 file descriptor is the same as typing interactively.

Recover the previous session from script mode (non-interactive mode. background process)

#!/bin/bash

DEV=/dev/sdb # device
PART_NUM=1 # partition to recover files from freespace (you can use testdisk to get all partitions)
WDIR="/home/user/work" # work directory where photorec stores recovered files
cd "$WDIR"

if [[ -f "photorec.ses" ]]; then # file photorec.ses exists (recover previous session)
   i=0 # index to read at most 10 lines of photorec.ses until to find the command parameters.
   while IFS="\n" read -r l; do # read photorec.ses line by line
      [[ "$i" -gt 9 ]] && break # break loop if number of line > 9
      [[ "$l" =~ "$PART_NUM" ]] && [[ "$l" =~ "search" ]] && params="$l" && break # set params and break loop if current line $l contains the $PART_NUM and the string search.
      ((i++)) # increase index counter and continue with the next line
   done < "photorec.ses"
   if [[ ! -z "$params" ]]; then # if params variable exists, start recovery from previous session
      params=`echo $params | sed 's/,inter//'` # First: remove ,inter from params. inter = interactive mode. photorec rewrite it any time...
      photorec /log /logname rec.log /d ./ /cmd $params # run photorec script mode from the last partition offset
   fi
else # file photorec.ses does not exists (start new session)
   photorec /log /logname rec.log /d ./ /cmd $DEV fileopt,everything,disable,pdf,enable,options,paranoid,keep_corrupted_file_no,freespace,$PART_NUM,search
fi

What is inside of photorec.ses file? What is the session format?

#1575732467
/dev/sdb partition_i386,1,blocksize,16384,fileopt,everything,disable,pdf,enable,options,paranoid,keep_corrupted_file_no,freespace,search,status=ext2_off,2898400,inter
31840-93151
309280-60751871

#1575732467: First line: unix timestamp. Time of last session update. /dev/sdb .. status=ext2_off,2898400: Second line: The command. The script looks for the command in the first 10 lines, to be compatible with future photorec updates. ext2_off,2898400: Offset (in sectors) of the last recovery process.

The next lines shows a range of scanned portions (and recovered files) to avoid recovering two times the same file. The last offset of the last line -60751871 is the last sector of the current partition.

This script is reusing the last session command but opening photorec in script mode. This means that photorec closes when it finishes. (In interactive mode it does not close when finished scanning).

That's all. I hope it is useful.

cgsecurity commented 4 years ago

Currently PhotoRec flushed the log file and exits immediately when SIGINT or SIGHUP is detected. If I understand correctly you are writing about several things:

photorec not handling SIGTERM (feature request)
photorec not detecting device that are disconnected during the scan (#18)
photorec writing the photorec.ses file in the current directory. If you want to run several PhotoRec sessions at the same time, use a new directory. To avoid to overwrite the old photorec.ses file, rename it before starting PhotoRec.

mvasi90 commented 4 years ago

Currently PhotoRec flushed the log file and exits immediately when SIGINT or SIGHUP is detected

Tested. It does not work. sudo kill -2 $(pidof photorec) -2 = SIGNINT:

kills parent process (atd.service). sudo kill -1 $(pidof photorec) SIGHUP does not kill the parent process. This happens because I don't have any mail command installed (atd[802751]: Exec failed for mail command: No such file or directory). And that is because the atd (parent) process detects that the photorec (child) process has come out unexpectedly and tries to inform.
the partially recovered file (the file being recovered) is not deleted, it is maintained.
the session file photorec.ses is not updated.
on resuming the session, it is starting again from the previous session (not the last because the offset is not updated). It's similar to SIGKILL (kill -9).

Note: I'm using atd.service to run the script in background from the udev rules on device connected.

photorec writing the photorec.ses file in the current directory.

Obviously photorec does that. Who said otherwise? The script change the shell working directory to the same every time to keep and resume the recovery session.

If you want to run several PhotoRec sessions at the same time, use a new directory. To avoid to overwrite the old photorec.ses file, rename it before starting PhotoRec.

No. I am saying that it is not possible to recover the session in script mode from background. Script mode = non-interactive mode. Above you have all those details that I have mentioned and it affects in some way the "resume recovery session in script mode (background)".

I recorded a terminal session with asciinema with two examples of the behavior:

Sending the keys: Enter (to stop recovery process), Y (to confirm) and q (to quit process). This works.
Sending SIGINT (kill -2 <pid>) not works. It kills the PhotoRec process without updating the session file.

mvasi90 commented 4 years ago

https://github.com/cgsecurity/testdisk/commit/6f24cf45f6f93f52b6bbb3ddb63118dd4a3d93cd Thank you.

photorec /cmd ... starts non interactive scripted mode. Now, I can kill/finish the process without losing the session. How can I restart it in non interactive scripted mode without the "session resume confirmation"? (In background mode the user can't interact).

You can implement what seems right to you. But for example it could be a menu /cmd option,resume or /resume allowing to restart the previous session from scripted mode.

I hope you understand me.

mvasi90 commented 4 years ago

Thank you very much.

cgsecurity / testdisk

[Solved] Photorec scripted run: session not working #73