search

LinearTapeFileSystem / ltfs

Reference implementation of the LTFS format Spec for stand alone tape drive
BSD 3-Clause "New" or "Revised" License
255 stars 76 forks source link

Previous mounted tape now fail to mount #431

Closed melop closed 7 months ago

melop commented 1 year ago

Hello. We have an IBM ULTRIUM-HH9 (N9B1 /dev/st0 /dev/sg3 ) tape drive. Previously we were able to use ltfs to format, mount, read and write data to the tape. But today when we tried to mount the same tape it gave the following error:

ltfs -o devname=/dev/sg3 /tape cad LTFS14000I LTFS starting, LTFS version 2.4.5.0 (10502), log level 2. cad LTFS14058I LTFS Format Specification version 2.4.0. cad LTFS14104I Launched by "ltfs -o devname=/dev/sg3 /tape". cad LTFS14105I This binary is built for Linux (x86_64). cad LTFS14106I GCC version is 11.3.0. cad LTFS17087I Kernel version: Linux version 5.15.0-76-generic (buildd@lcy02-amd64-028) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 i386. cad LTFS17089I Distribution: DISTRIB_ID=Ubuntu. cad LTFS17089I Distribution: PRETTY_NAME="Ubuntu 22.04.2 LTS". cad LTFS14063I Sync type is "time", Sync time is 300 sec. cad LTFS17085I Plugin: Loading "sg" tape backend. cad LTFS17085I Plugin: Loading "unified" iosched backend. cad LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection. cad LTFS30209I Opening a device through sg-ibmtape driver (/dev/sg3). cad LTFS30250I Opened the SCSI tape device 2.0.0.0 (/dev/sg3). cad LTFS30207I Vendor ID is IBM . cad LTFS30208I Product ID is ULTRIUM-HH9 . cad LTFS30214I Firmware revision is N9B1. cad LTFS30215I Drive serial is 10WT001089. cad LTFS30285I The reserved buffer size of /dev/sg3 is 1048576. cad LTFS30294I Setting up timeout values from RSOC. cad LTFS17160I Maximum device block size is 1048576. cad LTFS11330I Loading cartridge. cad LTFS30252I Logical block protection is disabled. cad LTFS11332I Load successful. cad LTFS17157I Changing the drive setting to write-anywhere mode. cad LTFS11005I Mounting the volume. cad LTFS30252I Logical block protection is disabled. cad LTFS30820I (no message found). cad LTFS30263I READ returns No Additional Sense Information (-20002) /dev/sg3. cad LTFS12049E Cannot read: backend call failed (-20002). cad LTFS11174E Cannot read ANSI label: read failed (-20002). cad LTFS11170E Failed to read label (-20002) from partition 0. cad LTFS11009E Cannot read volume: failed to read partition labels. cad LTFS14013E Cannot mount the volume. cad LTFS30252I Logical block protection is disabled.

It appears to be an HBA problem after looking through past issues, but how is it possible that we were able to mount and write to the tape before? Is there a chance that the data were corrupted?

Thanks!

melop commented 1 year ago

Update: we tested with an empty tape. We found that initial formatting, mounting and writing were successful. But after writing about 5TB of data, the tape again failed to mount with the same error. So at least this was repeatable.

piste-jp commented 1 year ago

It looks message definition of the LTFS30820I is missing. But the drive returned a longer record than LTFS requested. See below.

cad LTFS30820I (no message found).
cad LTFS30263I READ returns No Additional Sense Information (-20002) /dev/sg3.
cad LTFS12049E Cannot read: backend call failed (-20002).
cad LTFS11174E Cannot read ANSI label: read failed (-20002).
cad LTFS11170E Failed to read label (-20002) from partition 0.

It is little bit funny to me why first mount is successfully done and second mount (after writing a file) is failed. Because this overrun condition happens at the first record of a partition and it is read every mount...

piste-jp commented 1 year ago

I believe you compiled the LTFS from source code. Please try to run the test below and send back the log file. It would be helpful to us.

I assume the problem happens even if the file is not 5TB, for example it is 1MB. Is it correct?

  1. Rebuild the LTFS with applying the patch attached
  2. Format a tape
  3. Mount the tape by ltfs command with higher log level -o verbose=606 and write a short file
  4. Unmount the tape
  5. Mount the tape again with higher log level
diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c
index d91f4ad..6501081 100644
--- a/src/tape_drivers/linux/sg/sg_tape.c
+++ b/src/tape_drivers/linux/sg/sg_tape.c
@@ -1729,10 +1729,10 @@ static int _cdb_read(void *device, char *buf, size_t size, bool sili)
                         * In this case, LTFS will trust SCSI sense.
                         */
                        if (diff_len < 0) {
-                           ltfsmsg(LTFS_INFO, 30820I, diff_len, size - diff_len); // "Detect overrun condition"
+                           ltfsmsg(LTFS_INFO, 30217I, diff_len, size - diff_len); // "Detect overrun condition"
                            ret = -EDEV_OVERRUN;
                        } else {
-                           ltfsmsg(LTFS_DEBUG, 30821D, diff_len, size - diff_len); // "Detect underrun condition"
+                           ltfsmsg(LTFS_DEBUG, 30218D, diff_len, size - diff_len); // "Detect underrun condition"
                            length = size - diff_len;
                            ret = DEVICE_GOOD;
                        }
piste-jp commented 1 year ago

Hmm, one possibility comes to my mind.

Do you access to the st device, like just open, get some info and close, with another software include shell script while LTFS is running? (For example, monitor something on tape drive.)

This kind of side channel access might corrupt the LTFS format on tape because st device tries to rewind the tape at close(). Technically, LTFS has any way to detect such kind of silent position change at all (because it is done from same HBA port).

Now LTFS disallow the drive access fro same sg device by the feature on the sg driver but it cannnot disallow the access from st driver.

The scenario is

  1. LTFS mount the tape
  2. User writing a big file (let's assume LTFS has written to record-100)
  3. Now LTFS thinks the position is 100
  4. Another S/W open the drive via st device, and finally close the device
  5. The st device isse the rewind command and position is changed to 0
  6. LTFS doesn't know the position is changed at all and keep issuing WRITE command
    • In other words, LTFS thinks it is writing to position 101 but actually it is writing to position 0
  7. As a result, record at position 0 is 512KB record instead of the 80-bytes label
  8. At the second mount, LTFS reads 512KB record but it shall be 80 byte -> Overrun happens!
melop commented 1 year ago

We don't know of any side access through the st device while the tape is being written. The only time I used the st device was after unmounting and used the mt eject command to eject the tape from the drive.

Hmm, one possibility comes to my mind.

Do you access to the st device, like just open, get some info and close, with another software include shell script while LTFS is running? (For example, monitor something on tape drive.)

This kind of side channel access might corrupt the LTFS format on tape because st device tries to rewind the tape at close(). Technically, LTFS has any way to detect such kind of silent position change at all (because it is done from same HBA port).

Now LTFS disallow the drive access fro same sg device by the feature on the sg driver but it cannnot disallow the access from st driver.

The scenario is

  1. LTFS mount the tape
  2. User writing a big file (let's assume LTFS has written to record-100)
  3. Now LTFS thinks the position is 100
  4. Another S/W open the drive via st device, and finally close the device
  5. The st device isse the rewind command and position is changed to 0
  6. LTFS doesn't know the position is changed at all and keep issuing WRITE command
  • In other words, LTFS thinks it is writing to position 101 but actually it is writing to position 0
  1. As a result, record at position 0 is 512KB record instead of the 80-bytes label
  2. At the second mount, LTFS reads 512KB record but it shall be 80 byte -> Overrun happens!
melop commented 1 year ago

I will try this test. No - the problem did not occur after three initial mounting and writing operations ( each time a ~1TB tar.gz file was written ). We previously tested writing and reading small files (40-50Mb), which was all fine. The problem only happened after the fourth 1TB file was written. For the second test, we tried with a brand new LTO9 tape, formatted it and wrote all files (~5TB) to it. This tape also failed to mount with the same error messages.

I believe you compiled the LTFS from source code. Please try to run the test below and send back the log file. It would be helpful to us.

I assume the problem happens even if the file is not 5TB, for example it is 1MB. Is it correct?

  1. Rebuild the LTFS with applying the patch attached
  2. Format a tape
  3. Mount the tape by ltfs command with higher log level -o verbose=606 and write a short file
  4. Unmount the tape
  5. Mount the tape again with higher log level
diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c
index d91f4ad..6501081 100644
--- a/src/tape_drivers/linux/sg/sg_tape.c
+++ b/src/tape_drivers/linux/sg/sg_tape.c
@@ -1729,10 +1729,10 @@ static int _cdb_read(void *device, char *buf, size_t size, bool sili)
                       * In this case, LTFS will trust SCSI sense.
                       */
                      if (diff_len < 0) {
-                         ltfsmsg(LTFS_INFO, 30820I, diff_len, size - diff_len); // "Detect overrun condition"
+                         ltfsmsg(LTFS_INFO, 30217I, diff_len, size - diff_len); // "Detect overrun condition"
                          ret = -EDEV_OVERRUN;
                      } else {
-                         ltfsmsg(LTFS_DEBUG, 30821D, diff_len, size - diff_len); // "Detect underrun condition"
+                         ltfsmsg(LTFS_DEBUG, 30218D, diff_len, size - diff_len); // "Detect underrun condition"
                          length = size - diff_len;
                          ret = DEVICE_GOOD;
                      }
melop commented 1 year ago

I have now applied the patch, recompiled and performed the tests you described. I used the mt erase command to erase the tape, before starting the following test. Again, this 3.3Gb file was written ok, with successful re-mounting. The md5 on tape was also correct.

Should I proceed to test whether incorrectly issuing the mt eject command before umount would corrupt the data?

mkltfs -d /dev/sg3 > mkltfs.log 2>&1 &
ltfs -o verbose=606  -o  devname=/dev/sg3 /tape > mount.log 2>&1 &
cp ~/sra_cache/sra/SRR6313492.sra /tape
umount /tape
ltfs -o verbose=606  -o  devname=/dev/sg3 /tape > mount.2.log 2>&1 & #direct remounting after umount failed, see log
mt -f /dev/st0 eject  #eject with the mt command, then manually re-insert the tape into the tape drive
ltfs -o verbose=606  -o  devname=/dev/sg3 /tape > mount.3.log 2>&1 &

mkltfs.log:

LTFS15000I Starting mkltfs, LTFS version 2.4.5.1 (Prelim), log level 2.
LTFS15041I Launched by "mkltfs -d /dev/sg3".
LTFS15042I This binary is built for Linux (x86_64).
LTFS15043I GCC version is 11.3.0.
LTFS17087I Kernel version: Linux version 5.15.0-76-generic (buildd@lcy02-amd64-028) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 i386.
LTFS17089I Distribution: DISTRIB_ID=Ubuntu.
LTFS17089I Distribution: PRETTY_NAME="Ubuntu 22.04.2 LTS".
LTFS15003I Formatting device '/dev/sg3'.
LTFS15004I LTFS volume blocksize: 524288.
LTFS15005I Index partition placement policy: None.

LTFS11337I Update index-dirty flag (1) - NO_BARCODE (0x0x5630cb421980).
LTFS17085I Plugin: Loading "sg" tape backend.
LTFS30209I Opening a device through sg-ibmtape driver (/dev/sg3).
LTFS30250I Opened the SCSI tape device 6.0.0.0 (/dev/sg3).
LTFS30207I Vendor ID is IBM     .
LTFS30208I Product ID is ULTRIUM-HH9     .
LTFS30214I Firmware revision is N9B1.
LTFS30215I Drive serial is 10WT001089.
LTFS30285I The reserved buffer size of /dev/sg3 is 1048576.
LTFS30294I Setting up timeout values from RSOC.
LTFS17160I Maximum device block size is 1048576.
LTFS11330I Loading cartridge.
LTFS30252I Logical block protection is disabled.
LTFS11332I Load successful.
LTFS17157I Changing the drive setting to write-anywhere mode.
LTFS15049I Checking the medium (mount).
LTFS30252I Logical block protection is disabled.
LTFS30205I READ (0x08) returns -20801.
LTFS30263I READ returns End-of-Data (EOD) Detected (-20801) /dev/sg3.
LTFS30262I Forcing drive dump.
LTFS30253I Saving drive dump to /tmp/ltfs_10WT001089_2023_1101_131659_f.dmp.
LTFS12049E Cannot read: backend call failed (-20801).
LTFS11174E Cannot read ANSI label: read failed (-20801).
LTFS11009E Cannot read volume: failed to read partition labels.
LTFS15010I Creating data partition b on SCSI partition 1.
LTFS15011I Creating index partition a on SCSI partition 0.
LTFS17165I Resetting the medium's capacity proportion.
LTFS11097I Partitioning the medium.
LTFS30263I MODESELECT returns Mode Parameters Rounded (-20101) /dev/sg3.
LTFS11100I Writing label to partition b.
LTFS11278I Writing index to partition b.
LTFS30205I READ_ATTR (0x8c) returns -20501.
LTFS30263I READ_ATTR returns Invalid Field in CDB (-20501) /dev/sg3.
LTFS30233I Cannot read attribute (-20501).
LTFS11336I The attribute does not exist. Ignore the expected error.
LTFS17235I Writing index of NO_BARCODE to b (Reason: Format, 0 files) 10WT001089.
LTFS17236I Wrote index of NO_BARCODE (Gen = 1, Part = b, Pos = 5, 10WT001089).
LTFS11337I Update index-dirty flag (0) - NO_BARCODE (0x0x5630cb421980).
LTFS11100I Writing label to partition a.
LTFS11278I Writing index to partition a.
LTFS30205I READ_ATTR (0x8c) returns -20501.
LTFS30263I READ_ATTR returns Invalid Field in CDB (-20501) /dev/sg3.
LTFS30233I Cannot read attribute (-20501).
LTFS11336I The attribute does not exist. Ignore the expected error.
LTFS17235I Writing index of NO_BARCODE to a (Reason: Format, 0 files) 10WT001089.
LTFS17236I Wrote index of NO_BARCODE (Gen = 1, Part = a, Pos = 5, 10WT001089).
LTFS15013I Volume UUID is: 02758e2d-3966-4dad-a887-c30ef5120de4.

LTFS15019I Volume capacity is 17550 GB.
LTFS30252I Logical block protection is disabled.
LTFS15024I Medium formatted successfully.

mount.log

598 LTFS14000I LTFS starting, LTFS version 2.4.5.1 (Prelim), log level 606.
598 LTFS14058I LTFS Format Specification version 2.4.0.
598 LTFS14104I Launched by "ltfs -o verbose=606 -o devname=/dev/sg3 /tape".
598 LTFS14105I This binary is built for Linux (x86_64).
598 LTFS14106I GCC version is 11.3.0.
598 LTFS17087I Kernel version: Linux version 5.15.0-76-generic (buildd@lcy02-amd64-028) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 i386.
598 LTFS17089I Distribution: DISTRIB_ID=Ubuntu.
598 LTFS17089I Distribution: PRETTY_NAME="Ubuntu 22.04.2 LTS".
598 LTFS14064I Sync type is "unmount".
598 LTFS17085I Plugin: Loading "sg" tape backend.
598 LTFS17085I Plugin: Loading "unified" iosched backend.
598 LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection.
598 LTFS30209I Opening a device through sg-ibmtape driver (/dev/sg3).
598 LTFS30250I Opened the SCSI tape device 6.0.0.0 (/dev/sg3).
598 LTFS30207I Vendor ID is IBM     .
598 LTFS30208I Product ID is ULTRIUM-HH9     .
598 LTFS30214I Firmware revision is N9B1.
598 LTFS30215I Drive serial is 10WT001089.
598 LTFS30285I The reserved buffer size of /dev/sg3 is 1048576.
598 LTFS30294I Setting up timeout values from RSOC.
598 LTFS39801D SCSI timeout (op_code 0x5f, timeout = 60).
598 LTFS12023D Reserving device.
598 LTFS30392D Backend reserve (PRO) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5f, timeout = 60).
598 LTFS12028D Unlocking medium.
598 LTFS30392D Backend allow medium removal 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x1e, timeout = 60).
598 LTFS30392D Backend read block limits 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
598 LTFS17160I Maximum device block size is 1048576.
598 LTFS11330I Loading cartridge.
598 LTFS30392D Backend load 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x1b, timeout = 7920).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
598 LTFS30393D Backend modesense: 63 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend test unit ready 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
598 LTFS12026D Locking medium in the drive.
598 LTFS30392D Backend prevent medium removal 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x1e, timeout = 60).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
598 LTFS30392D Backend sg_set_default Resetting LBP.
598 LTFS30393D Backend LBP Enable: 0 .
598 LTFS30393D Backend LBP Method: 2 .
598 LTFS30393D Backend modesense: 10 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend modeselect 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x55, timeout = 60).
598 LTFS30252I Logical block protection is disabled.
598 LTFS30397D Backend logsense: (23, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
598 LTFS30397D Backend capacity part0: (122591, 122614) 10WT001089.
598 LTFS30397D Backend capacity part1: (16798280, 16798291) 10WT001089.
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend read block limits 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS11332I Load successful.
598 LTFS30392D Backend test unit ready 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
598 LTFS30392D Backend test unit ready 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
598 LTFS30397D Backend logsense: (23, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
598 LTFS30397D Backend capacity part0: (122591, 122614) 10WT001089.
598 LTFS30397D Backend capacity part1: (16798280, 16798291) 10WT001089.
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend modeselect 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x55, timeout = 60).
598 LTFS17157I Changing the drive setting to write-anywhere mode.
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS11005I Mounting the volume.
598 LTFS11012D Loading the tape.
598 LTFS30392D Backend test unit ready 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
598 LTFS30392D Backend load 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x1b, timeout = 7920).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
598 LTFS30393D Backend modesense: 63 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend test unit ready 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
598 LTFS30392D Backend sg_set_default Resetting LBP.
598 LTFS30393D Backend LBP Enable: 0 .
598 LTFS30393D Backend LBP Method: 2 .
598 LTFS30393D Backend modesense: 10 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend modeselect 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x55, timeout = 60).
598 LTFS30252I Logical block protection is disabled.
598 LTFS30397D Backend logsense: (23, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
598 LTFS30397D Backend capacity part0: (122591, 122614) 10WT001089.
598 LTFS30397D Backend capacity part1: (16798280, 16798291) 10WT001089.
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend read block limits 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30397D Backend locate: (0, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x92, timeout = 2940).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
598 LTFS11007D Tape is loaded.
598 LTFS30397D Backend logsense: (23, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
598 LTFS30397D Backend capacity part0: (122591, 122614) 10WT001089.
598 LTFS30397D Backend capacity part1: (16798280, 16798291) 10WT001089.
598 LTFS11008D Reading partition labels.
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend read block limits 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
598 LTFS30397D Backend locate: (0, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x92, timeout = 2940).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
598 LTFS30395D Backend read: 4096 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30395D Backend read: 4096 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
598 LTFS30204D READ (0x08) expected error -20004.
598 LTFS30219D Read block: file mark detected.
598 LTFS30395D Backend read: 4096 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30395D Backend read: 4096 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
598 LTFS30204D READ (0x08) expected error -20004.
598 LTFS30219D Read block: file mark detected.
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend read block limits 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
598 LTFS30397D Backend locate: (1, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x92, timeout = 2940).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (1, 0) FM = 0 10WT001089.
598 LTFS30395D Backend read: 4096 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30395D Backend read: 4096 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
598 LTFS30204D READ (0x08) expected error -20004.
598 LTFS30219D Read block: file mark detected.
598 LTFS30395D Backend read: 4096 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30395D Backend read: 4096 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
598 LTFS30204D READ (0x08) expected error -20004.
598 LTFS30219D Read block: file mark detected.
598 LTFS30393D Backend modesense: 15 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend modeselect 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x55, timeout = 60).
598 LTFS30393D Backend modesense: 16 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30392D Backend read block limits 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
598 LTFS11013D Partition labels are valid.
598 LTFS11014D Reading MAM parameters.
598 LTFS30397D Backend readattr: (0, 5667) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS11339D Read volume lock status (0).
598 LTFS30393D Backend modesense: 37 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
598 LTFS30397D Backend logsense: (23, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
598 LTFS30397D Backend logsense: (23, 0) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
598 LTFS30397D Backend readattr: (0, 2060) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (1, 2060) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 9) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS11018D Done reading MAM parameters.
598 LTFS11019D Checking volume consistency.
598 LTFS30397D Backend locate: (0, 5) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x92, timeout = 2940).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 5) FM = 3 10WT001089.
598 LTFS30395D Backend read: 524288 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30201D CDB check condition: sense = 000000, No Additional Sense Information.
598 LTFS30204D READ (0x08) expected error -20000.
598 LTFS30218D Read block: underrun in illegal length. residual = 523294, actual = 994.
598 LTFS30395D Backend read: 524288 bytes 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
598 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
598 LTFS30204D READ (0x08) expected error -20004.
598 LTFS30219D Read block: file mark detected.
598 LTFS30396D Backend space back file marks: 1 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x91, timeout = 2940).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 6) FM = 3 10WT001089.
598 LTFS30396D Backend space forward file marks: 1 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x91, timeout = 2940).
598 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
598 LTFS30398D Backend readpos: (0, 7) FM = 4 10WT001089.
598 LTFS11025D Volume is consistent.
598 LTFS11028D Consistency check finished.
598 LTFS30397D Backend readattr: (0, 2048) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 2049) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 2050) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 2051) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 2053) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 2054) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 2059) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 5667) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS30397D Backend readattr: (0, 2056) 10WT001089.
598 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
598 LTFS17227I Tape attribute: Vendor = IBM     .
598 LTFS17227I Tape attribute: Application Name = LTFS                            .
598 LTFS17227I Tape attribute: Application Version = 2.4.5.1 .
598 LTFS17227I Tape attribute: Medium Label = .
598 LTFS17228I Tape attribute: Text Localization ID = 0x81.
598 LTFS17227I Tape attribute: Barcode =                                 .
598 LTFS17227I Tape attribute: Application Format Version = 2.4.0           .
598 LTFS17228I Tape attribute: Volume Lock Status = 0x00.
598 LTFS17227I Tape attribute: Media Pool name = .
598 LTFS11031I Volume mounted successfully. NO_BARCODE : Gen = 1 / (a, 5) -> (b, 5) / 10WT001089.
598 LTFS14111I Initial setup completed successfully.
598 LTFS14112I Invoke 'mount' command to check the result of final setup.
598 LTFS14113I Specified mount point is listed if succeeded.

mount.2.log

5e0 LTFS14000I LTFS starting, LTFS version 2.4.5.1 (Prelim), log level 606.
5e0 LTFS14058I LTFS Format Specification version 2.4.0.
5e0 LTFS14104I Launched by "ltfs -o verbose=606 -o devname=/dev/sg3 /tape".
5e0 LTFS14105I This binary is built for Linux (x86_64).
5e0 LTFS14106I GCC version is 11.3.0.
5e0 LTFS17087I Kernel version: Linux version 5.15.0-76-generic (buildd@lcy02-amd64-028) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 i386.
5e0 LTFS17089I Distribution: DISTRIB_ID=Ubuntu.
5e0 LTFS17089I Distribution: PRETTY_NAME="Ubuntu 22.04.2 LTS".
5e0 LTFS14064I Sync type is "unmount".
5e0 LTFS17085I Plugin: Loading "sg" tape backend.
5e0 LTFS17085I Plugin: Loading "unified" iosched backend.
5e0 LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection.
5e0 LTFS30209I Opening a device through sg-ibmtape driver (/dev/sg3).
5e0 LTFS30210I Cannot open device: failed to open /dev/sg3 (16).
5e0 LTFS12012E Cannot open device: failed backend open call.
5e0 LTFS10004E Cannot open device '/dev/sg3'.

mount.3.log

5f5 LTFS14000I LTFS starting, LTFS version 2.4.5.1 (Prelim), log level 606.
5f5 LTFS14058I LTFS Format Specification version 2.4.0.
5f5 LTFS14104I Launched by "ltfs -o verbose=606 -o devname=/dev/sg3 /tape".
5f5 LTFS14105I This binary is built for Linux (x86_64).
5f5 LTFS14106I GCC version is 11.3.0.
5f5 LTFS17087I Kernel version: Linux version 5.15.0-76-generic (buildd@lcy02-amd64-028) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 i386.
5f5 LTFS17089I Distribution: DISTRIB_ID=Ubuntu.
5f5 LTFS17089I Distribution: PRETTY_NAME="Ubuntu 22.04.2 LTS".
5f5 LTFS14064I Sync type is "unmount".
5f5 LTFS17085I Plugin: Loading "sg" tape backend.
5f5 LTFS17085I Plugin: Loading "unified" iosched backend.
5f5 LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection.
5f5 LTFS30209I Opening a device through sg-ibmtape driver (/dev/sg3).
5f5 LTFS30250I Opened the SCSI tape device 6.0.0.0 (/dev/sg3).
5f5 LTFS30207I Vendor ID is IBM     .
5f5 LTFS30208I Product ID is ULTRIUM-HH9     .
5f5 LTFS30214I Firmware revision is N9B1.
5f5 LTFS30215I Drive serial is 10WT001089.
5f5 LTFS30285I The reserved buffer size of /dev/sg3 is 1048576.
5f5 LTFS30201D CDB check condition: sense = 062800, Not Ready to Ready Transition, Medium May Have Changed.
5f5 LTFS30205I RSOC (0xa3) returns -20601.
5f5 LTFS30263I RSOC returns Not Ready to Ready Transition, Medium May Have Changed (-20601) /dev/sg3.
5f5 LTFS30262I Forcing drive dump.
5f5 LTFS30393D Backend force dump: 0 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x1d, use default timeout.
5f5 LTFS30253I Saving drive dump to /tmp/ltfs_10WT001089_2023_1101_132734_f.dmp.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30255D Total dump data length is 5018816.
5f5 LTFS30256D Total number of transfers is 10.
5f5 LTFS30257D Transferring dump data.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30393D Backend read buffer: 1 10WT001089.
5f5 LTFS39802W Unknown SCSI OP code 0x3c, use default timeout.
5f5 LTFS30294I Setting up timeout values from vendor and device.
5f5 LTFS39801D SCSI timeout (op_code 0x5f, timeout = 60).
5f5 LTFS12023D Reserving device.
5f5 LTFS30392D Backend reserve (PRO) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5f, timeout = 60).
5f5 LTFS12028D Unlocking medium.
5f5 LTFS30392D Backend allow medium removal 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x1e, timeout = 60).
5f5 LTFS30392D Backend read block limits 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
5f5 LTFS17160I Maximum device block size is 1048576.
5f5 LTFS11330I Loading cartridge.
5f5 LTFS30392D Backend load 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x1b, timeout = 960).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
5f5 LTFS30393D Backend modesense: 63 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend test unit ready 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
5f5 LTFS12026D Locking medium in the drive.
5f5 LTFS30392D Backend prevent medium removal 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x1e, timeout = 60).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
5f5 LTFS30392D Backend sg_set_default Resetting LBP.
5f5 LTFS30393D Backend LBP Enable: 0 .
5f5 LTFS30393D Backend LBP Method: 2 .
5f5 LTFS30393D Backend modesense: 10 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend modeselect 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x55, timeout = 60).
5f5 LTFS30252I Logical block protection is disabled.
5f5 LTFS30397D Backend logsense: (23, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
5f5 LTFS30397D Backend capacity part0: (122593, 122614) 10WT001089.
5f5 LTFS30397D Backend capacity part1: (16795288, 16798291) 10WT001089.
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend read block limits 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS11332I Load successful.
5f5 LTFS30392D Backend test unit ready 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
5f5 LTFS30392D Backend test unit ready 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
5f5 LTFS30397D Backend logsense: (23, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
5f5 LTFS30397D Backend capacity part0: (122593, 122614) 10WT001089.
5f5 LTFS30397D Backend capacity part1: (16795288, 16798291) 10WT001089.
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend modeselect 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x55, timeout = 60).
5f5 LTFS17157I Changing the drive setting to write-anywhere mode.
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS11005I Mounting the volume.
5f5 LTFS11012D Loading the tape.
5f5 LTFS30392D Backend test unit ready 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
5f5 LTFS30392D Backend load 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x1b, timeout = 960).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
5f5 LTFS30393D Backend modesense: 63 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend test unit ready 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x00, timeout = 60).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
5f5 LTFS30392D Backend sg_set_default Resetting LBP.
5f5 LTFS30393D Backend LBP Enable: 0 .
5f5 LTFS30393D Backend LBP Method: 2 .
5f5 LTFS30393D Backend modesense: 10 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend modeselect 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x55, timeout = 60).
5f5 LTFS30252I Logical block protection is disabled.
5f5 LTFS30397D Backend logsense: (23, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
5f5 LTFS30397D Backend capacity part0: (122593, 122614) 10WT001089.
5f5 LTFS30397D Backend capacity part1: (16795288, 16798291) 10WT001089.
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend read block limits 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30397D Backend locate: (0, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x92, timeout = 2940).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
5f5 LTFS11007D Tape is loaded.
5f5 LTFS30397D Backend logsense: (23, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
5f5 LTFS30397D Backend capacity part0: (122593, 122614) 10WT001089.
5f5 LTFS30397D Backend capacity part1: (16795288, 16798291) 10WT001089.
5f5 LTFS11008D Reading partition labels.
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend read block limits 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
5f5 LTFS30397D Backend locate: (0, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x92, timeout = 2940).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 0) FM = 0 10WT001089.
5f5 LTFS30395D Backend read: 4096 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30395D Backend read: 4096 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
5f5 LTFS30204D READ (0x08) expected error -20004.
5f5 LTFS30219D Read block: file mark detected.
5f5 LTFS30395D Backend read: 4096 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30395D Backend read: 4096 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
5f5 LTFS30204D READ (0x08) expected error -20004.
5f5 LTFS30219D Read block: file mark detected.
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend read block limits 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
5f5 LTFS30397D Backend locate: (1, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x92, timeout = 2940).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (1, 0) FM = 0 10WT001089.
5f5 LTFS30395D Backend read: 4096 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30395D Backend read: 4096 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
5f5 LTFS30204D READ (0x08) expected error -20004.
5f5 LTFS30219D Read block: file mark detected.
5f5 LTFS30395D Backend read: 4096 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30395D Backend read: 4096 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
5f5 LTFS30204D READ (0x08) expected error -20004.
5f5 LTFS30219D Read block: file mark detected.
5f5 LTFS30393D Backend modesense: 15 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend modeselect 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x55, timeout = 60).
5f5 LTFS30393D Backend modesense: 16 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30392D Backend read block limits 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x05, timeout = 60).
5f5 LTFS11013D Partition labels are valid.
5f5 LTFS11014D Reading MAM parameters.
5f5 LTFS30397D Backend readattr: (0, 5667) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS11339D Read volume lock status (0).
5f5 LTFS30393D Backend modesense: 37 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x5a, timeout = 60).
5f5 LTFS30397D Backend logsense: (23, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
5f5 LTFS30397D Backend logsense: (23, 0) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x4d, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 2060) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (1, 2060) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 9) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS11018D Done reading MAM parameters.
5f5 LTFS11019D Checking volume consistency.
5f5 LTFS30397D Backend locate: (0, 5) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x92, timeout = 2940).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 5) FM = 3 10WT001089.
5f5 LTFS30395D Backend read: 524288 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30201D CDB check condition: sense = 000000, No Additional Sense Information.
5f5 LTFS30204D READ (0x08) expected error -20000.
5f5 LTFS30218D Read block: underrun in illegal length. residual = 522692, actual = 1596.
5f5 LTFS30395D Backend read: 524288 bytes 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x08, timeout = 2340).
5f5 LTFS30201D CDB check condition: sense = 000001, Filemark Detected.
5f5 LTFS30204D READ (0x08) expected error -20004.
5f5 LTFS30219D Read block: file mark detected.
5f5 LTFS30396D Backend space back file marks: 1 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x91, timeout = 2940).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 6) FM = 3 10WT001089.
5f5 LTFS30396D Backend space forward file marks: 1 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x91, timeout = 2940).
5f5 LTFS39801D SCSI timeout (op_code 0x34, timeout = 60).
5f5 LTFS30398D Backend readpos: (0, 7) FM = 4 10WT001089.
5f5 LTFS11025D Volume is consistent.
5f5 LTFS11028D Consistency check finished.
5f5 LTFS30397D Backend readattr: (0, 2048) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 2049) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 2050) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 2051) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 2053) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 2054) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 2059) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 5667) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS30397D Backend readattr: (0, 2056) 10WT001089.
5f5 LTFS39801D SCSI timeout (op_code 0x8c, timeout = 60).
5f5 LTFS17227I Tape attribute: Vendor = IBM     .
5f5 LTFS17227I Tape attribute: Application Name = LTFS                            .
5f5 LTFS17227I Tape attribute: Application Version = 2.4.5.1 .
5f5 LTFS17227I Tape attribute: Medium Label = .
5f5 LTFS17228I Tape attribute: Text Localization ID = 0x81.
5f5 LTFS17227I Tape attribute: Barcode =                                 .
5f5 LTFS17227I Tape attribute: Application Format Version = 2.4.0           .
5f5 LTFS17228I Tape attribute: Volume Lock Status = 0x00.
5f5 LTFS17227I Tape attribute: Media Pool name = .
5f5 LTFS11031I Volume mounted successfully. NO_BARCODE : Gen = 2 / (a, 5) -> (b, 6592) / 10WT001089.
5f5 LTFS14111I Initial setup completed successfully.
5f5 LTFS14112I Invoke 'mount' command to check the result of final setup.
5f5 LTFS14113I Specified mount point is listed if succeeded.

I believe you compiled the LTFS from source code. Please try to run the test below and send back the log file. It would be helpful to us.

I assume the problem happens even if the file is not 5TB, for example it is 1MB. Is it correct?

  1. Rebuild the LTFS with applying the patch attached
  2. Format a tape
  3. Mount the tape by ltfs command with higher log level -o verbose=606 and write a short file
  4. Unmount the tape
  5. Mount the tape again with higher log level
diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c
index d91f4ad..6501081 100644
--- a/src/tape_drivers/linux/sg/sg_tape.c
+++ b/src/tape_drivers/linux/sg/sg_tape.c
@@ -1729,10 +1729,10 @@ static int _cdb_read(void *device, char *buf, size_t size, bool sili)
                       * In this case, LTFS will trust SCSI sense.
                       */
                      if (diff_len < 0) {
-                         ltfsmsg(LTFS_INFO, 30820I, diff_len, size - diff_len); // "Detect overrun condition"
+                         ltfsmsg(LTFS_INFO, 30217I, diff_len, size - diff_len); // "Detect overrun condition"
                          ret = -EDEV_OVERRUN;
                      } else {
-                         ltfsmsg(LTFS_DEBUG, 30821D, diff_len, size - diff_len); // "Detect underrun condition"
+                         ltfsmsg(LTFS_DEBUG, 30218D, diff_len, size - diff_len); // "Detect underrun condition"
                          length = size - diff_len;
                          ret = DEVICE_GOOD;
                      }
piste-jp commented 1 year ago

Should I proceed to test whether incorrectly issuing the mt eject command before umount would corrupt the data?

The simple answer is Yes, it causes something bad thing. Please do not make anything against tape drive (from /dev/st?).

The log answer is LTFS issues PREVENT_MEDIUM_REMOVAL at the beginning of the mount process, so I think you cannot eject the tape from the drive...

But /dev/st? tries to issue REWIND command at the close() call against file descriptor of /dev/st?. So tape is rewinded silently from LTFS perspective.

melop commented 1 year ago

ok, now I may know what the problem was. If we accidentally switched the order of umount and mt eject commands (executing mt eject before umount), the error occurs. This means that umount was writing some information onto tape, but the mt eject command (although failed), changed the position of the read head. So is there a way prevent this? Users do all sorts of weird things and it seems that a switch of command order screwing up data is really defeating the purpose of having tape backups. For example, can I disallow access to the /dev/st0 device altogether?

Hmm, one possibility comes to my mind.

Do you access to the st device, like just open, get some info and close, with another software include shell script while LTFS is running? (For example, monitor something on tape drive.)

This kind of side channel access might corrupt the LTFS format on tape because st device tries to rewind the tape at close(). Technically, LTFS has any way to detect such kind of silent position change at all (because it is done from same HBA port).

Now LTFS disallow the drive access fro same sg device by the feature on the sg driver but it cannnot disallow the access from st driver.

The scenario is

  1. LTFS mount the tape
  2. User writing a big file (let's assume LTFS has written to record-100)
  3. Now LTFS thinks the position is 100
  4. Another S/W open the drive via st device, and finally close the device
  5. The st device isse the rewind command and position is changed to 0
  6. LTFS doesn't know the position is changed at all and keep issuing WRITE command
  • In other words, LTFS thinks it is writing to position 101 but actually it is writing to position 0
  1. As a result, record at position 0 is 512KB record instead of the 80-bytes label
  2. At the second mount, LTFS reads 512KB record but it shall be 80 byte -> Overrun happens!
piste-jp commented 1 year ago

Two more things,

  1. You can use -o eject option to ltfs command if you want to eject the tape at the end of LTFS
  2. You can use --wipe option to mkltfs for unformat the tape to one partitioned tape (instead of mt erase)
piste-jp commented 1 year ago

So is there a way prevent this?

LTFS is designed to be able to shoot my foot. So it doesn't protect side channel access. But please do not use mt command with LTFS. It is really different layer like filesystem and raw disk device.

If you need to protect side channel access, you need to disable st driver by configuration of modprobe or similar kernel module control mechanism. (I don't know the detail of Ubuntu.... Please don't ask me how to)

melop commented 1 year ago

For now I will disallow access to the /dev/st0 by setting file access attributes so that users cannot use mt to operate the tape drive. ltfs -o eject doesn't seem to be able to eject the tape, so for now I will just ask users to press the physical button on the tape drive to retrieve their tapes.

piste-jp commented 1 year ago

ltfs -o eject doesn't seem to be able to eject the tape

I don't think so. It is NOT a command to eject tape, so no eject happens if you just issue ltfs -o eject /dev/sgX.

It shall be specified into mount command like ltfs -o eject -o devname=/dev/sg3 /tape. The tape shall be unmounted when ltfs finishes the unmount operation triggered by umount or fusermount -u command. (By the way, ltfs just receives a unmount request even if umount command is returned. LTFS will be alive for a while for writing the latest index on the tape, and then finished.)

See the detail options for ltfs command below.

https://github.com/LinearTapeFileSystem/ltfs/wiki/ltfs

melop commented 1 year ago

In this case, should the user first execute umount /tape? Or will ltfs -o eject -o devname=/dev/sg3 /tape handle the unmounting before ejecting the tape?

ltfs -o eject doesn't seem to be able to eject the tape

I don't think so. It is NOT a command to eject tape, so no eject happens if you just issue ltfs -o eject /dev/sgX.

It shall be specified into mount command like ltfs -o eject -o devname=/dev/sg3 /tape. The tape shall be unmounted when ltfs finishes the unmount operation triggered by umount or fusermount -u command. (By the way, ltfs just receives a unmount request even if umount command is returned. LTFS will be alive for a while for writing the latest index on the tape, and then finished.)

See the detail options for ltfs command below.

https://github.com/LinearTapeFileSystem/ltfs/wiki/ltfs

piste-jp commented 1 year ago

Oh, the scenario is

  1. User mounts a tape by ltfs -o eject -o devname=/dev/sg3 /tape
  2. POSIX file system access is enabled via teh mount directory /tape
  3. User triggers the unmount against to the tape by umount /tape (this command is returned soon even if unmount operation of ltfs is not finished)
  4. LTFS tries to write down the latest index on memory to the tape (if it is changed by POSIX file system access)
  5. Step 4 is successfully done, LTFS issues UNLOAD command to the drive, the drive shall eject the tape
piste-jp commented 7 months ago

Closing because there is no update for long time.