LinearTapeFileSystem / ltfs

Reference implementation of the LTFS format Spec for stand alone tape drive
BSD 3-Clause "New" or "Revised" License
256 stars 77 forks source link

Unable to Load Cartridge #295

Closed bishbashbackup closed 3 years ago

bishbashbackup commented 3 years ago

Hello,

I've just compiled and installed this on a workstation running Ubuntu 18.04 LTS, which is connected to the following tape drives: Quantum LTO-6 Half-Height Drive B Quantum LTO-6 Half-Height Drive C

It seemed to install without any issues, but it doesn't seem to list the tape drives:

maceadmin@amacewrk05:\~$ ltfs -o device_list 3118 LTFS14000I LTFS starting, LTFS version 2.5.0.0 (Prelim), log level 2. 3118 LTFS14058I LTFS Format Specification version 2.4.0. 3118 LTFS14104I Launched by "ltfs -o device_list". 3118 LTFS14105I This binary is built for Linux (x86_64). 3118 LTFS14106I GCC version is 7.5.0. 3118 LTFS17087I Kernel version: Linux version 5.4.0-70-generic (buildd@lgw01-amd64-039) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #78\~18.04.1-Ubuntu SMP Sat Mar 20 14:10:07 UTC 2021 i386. 3118 LTFS17089I Distribution: NAME="Ubuntu". 3118 LTFS17089I Distribution: DISTRIB_ID=Ubuntu. 3118 LTFS17085I Plugin: Loading "sg" tape backend. Tape Device list:.

However, I can list the tape drives using lshw:

maceadmin@amacewrk05:~$ sudo lshw -class tape -tape:0
description: SCSI Tape product: ULTRIUM 6 vendor: QUANTUM physical id: 0.2.0 bus info: scsi@6:0.2.0 logical name: /dev/nst0 version: 4142 serial: HU14171HYL capabilities: removable configuration: ansiversion=6
-tape:1 description: SCSI Tape product: ULTRIUM-HH6 vendor: QUANTUM physical id: 0.3.0 bus info: scsi@6:0.3.0 logical name: /dev/nst1 version: J451 serial: 91WT082810 capabilities: removable configuration: ansiversion=6

If I try to load a tape in the model B drive, I get this:

maceadmin@amacewrk05:\~$ sudo ltfs -o devname=HU14171HYL /media/lto1 3274 LTFS14000I LTFS starting, LTFS version 2.5.0.0 (Prelim), log level 2. 3274 LTFS14058I LTFS Format Specification version 2.4.0. 3274 LTFS14104I Launched by "ltfs -o devname=HU14171HYL /media/lto1". 3274 LTFS14105I This binary is built for Linux (x86_64). 3274 LTFS14106I GCC version is 7.5.0. 3274 LTFS17087I Kernel version: Linux version 5.4.0-70-generic (buildd@lgw01-amd64-039) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #78\~18.04.1-Ubuntu SMP Sat Mar 20 14:10:07 UTC 2021 i386. 3274 LTFS17089I Distribution: NAME="Ubuntu". 3274 LTFS17089I Distribution: DISTRIB_ID=Ubuntu. 3274 LTFS14063I Sync type is "time", Sync time is 300 sec. 3274 LTFS17085I Plugin: Loading "sg" tape backend. 3274 LTFS17085I Plugin: Loading "unified" iosched backend. 3274 LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection. 3274 LTFS30209I Opening a device through sg-ibmtape driver (HU14171HYL). 3274 LTFS30288I Opening a tape device for drive serial HU14171HYL. 3274 LTFS30250I Opened the SCSI tape device 6.0.2.0 (/dev/sg6). 3274 LTFS30207I Vendor ID is QUANTUM . 3274 LTFS30208I Product ID is ULTRIUM 6 . 3274 LTFS30214I Firmware revision is 4142. 3274 LTFS30215I Drive serial is HU14171HYL. 3274 LTFS30290I Changer /dev/sg6 isn't reserved from any nodes. 3274 LTFS30285I The reserved buffer size of HU14171HYL is 1048576. 3274 LTFS30294I Setting up timeout values from RSOC. 3274 LTFS17160I Maximum device block size is 1048576. 3274 LTFS11330I Loading cartridge. 3274 LTFS30232I Cannot get remaining capacity: failed to parse the log page. 3274 LTFS12030E Cannot get capacity data: backend call failed (-21700). 3274 LTFS11999E Cannot load the medium: failed to get capacity data (-21700). 3274 LTFS11331E Failed to load the cartridge (ltfs_load_tape). 3274 LTFS30232I Cannot get remaining capacity: failed to parse the log page. 3274 LTFS12030E Cannot get capacity data: backend call failed (-21700). 3274 LTFS11999E Cannot load the medium: failed to get capacity data (-21700). 3274 LTFS14075E Cannot set up tape drive. 3274 LTFS30205I MODESELECT (0x55) returns -20500. 3274 LTFS30263I MODESELECT returns Invalid Field in Parameter List (-20500) /dev/sg6.

If I try to load a tape in the model C drive, I get this:

maceadmin@amacewrk05:\~$ sudo ltfs -o devname=91WT082810 /media/lto2 329a LTFS14000I LTFS starting, LTFS version 2.5.0.0 (Prelim), log level 2. 329a LTFS14058I LTFS Format Specification version 2.4.0. 329a LTFS14104I Launched by "ltfs -o devname=91WT082810 /media/lto2". 329a LTFS14105I This binary is built for Linux (x86_64). 329a LTFS14106I GCC version is 7.5.0. 329a LTFS17087I Kernel version: Linux version 5.4.0-70-generic (buildd@lgw01-amd64-039) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #78\~18.04.1-Ubuntu SMP Sat Mar 20 14:10:07 UTC 2021 i386. 329a LTFS17089I Distribution: NAME="Ubuntu". 329a LTFS17089I Distribution: DISTRIB_ID=Ubuntu. 329a LTFS14063I Sync type is "time", Sync time is 300 sec. 329a LTFS17085I Plugin: Loading "sg" tape backend. 329a LTFS17085I Plugin: Loading "unified" iosched backend. 329a LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection. 329a LTFS30209I Opening a device through sg-ibmtape driver (91WT082810). 329a LTFS30288I Opening a tape device for drive serial 91WT082810. 329a LTFS30250I Opened the SCSI tape device 6.0.3.0 (/dev/sg7). 329a LTFS30207I Vendor ID is QUANTUM . 329a LTFS30208I Product ID is ULTRIUM-HH6 . 329a LTFS30214I Firmware revision is J451. 329a LTFS30215I Drive serial is 91WT082810. 329a LTFS30205I TEST_UNIT_READY (0x00) returns -20601. 329a LTFS30290I Changer /dev/sg7 isn't reserved from any nodes. 329a LTFS30285I The reserved buffer size of 91WT082810 is 1048576. 329a LTFS30294I Setting up timeout values from RSOC. 329a LTFS17160I Maximum device block size is 1048576. 329a LTFS11330I Loading cartridge. 329a LTFS11332I Load successful. 329a LTFS17157I Changing the drive setting to write-anywhere mode. 329a LTFS11005I Mounting the volume. 329a LTFS30205I READ_ATTR (0x8c) returns -20501. 329a LTFS30263I READ_ATTR returns Invalid Field in CDB (-20501) /dev/sg7. 329a LTFS30233I Cannot read attribute (-20501). 329a LTFS11336I The attribute does not exist. Ignore the expected error. 329a LTFS30216W Length mismatch is detected. (Act = 524288, resid = 0, resid_sense = 484063). 329a LTFS30216W Length mismatch is detected. (Act = 524288, resid = 0, resid_sense = 484063). 329a LTFS12049E Cannot read: backend call failed (-21716). 329a LTFS17039E XML parser: failed to read a block from the medium (-21716). 329a LTFS17037E XML parser: failed to read from XML stream. 329a LTFS17273E XML parser: Detected an I/O error on reading an index from the tape (-21716). 329a LTFS17016E Cannot parse index direct from medium (-21716). 329a LTFS11194W Cannot read index: failed to read and parse XML data (-21716). 329a LTFS11024E Cannot mount volume: read index failed on the index partition. 329a LTFS14013E Cannot mount the volume. 329a LTFS30252I Logical block protection is disabled.

I'm not sure what's going wrong. So any advice would be greatly appreciated.

TulioLazarini commented 3 years ago

Hi @alexhabgood , have you tried to run ltfs with sudo? Mine requires sudo powers to run properly. Give it a try and let's see what happens. Cheers.

TulioLazarini commented 3 years ago

Sorry, I mean running sudo ltfs -o device_list, to get the proper sg devices, which is the default backend.

bishbashbackup commented 3 years ago

Hi @TulioLazarini

That seems to be recognising the model C drive now, still no model B though:

maceadmin@amacewrk05:\~$ sudo ltfs -o device_list [sudo] password for maceadmin: 344f LTFS14000I LTFS starting, LTFS version 2.5.0.0 (Prelim), log level 2. 344f LTFS14058I LTFS Format Specification version 2.4.0. 344f LTFS14104I Launched by "ltfs -o device_list". 344f LTFS14105I This binary is built for Linux (x86_64). 344f LTFS14106I GCC version is 7.5.0. 344f LTFS17087I Kernel version: Linux version 5.4.0-70-generic (buildd@lgw01-amd64-039) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #78~18.04.1-Ubuntu SMP Sat Mar 20 14:10:07 UTC 2021 i386. 344f LTFS17089I Distribution: NAME="Ubuntu". 344f LTFS17089I Distribution: DISTRIB_ID=Ubuntu. 344f LTFS17085I Plugin: Loading "sg" tape backend. Tape Device list:. Device Name = /dev/sg7 (6.0.5.0), Vendor ID = QUANTUM , Product ID = ULTRIUM-HH6 , Serial Number = 91WT082810, Product Name =[ULTRIUM-HH6].

Also, still not able to mount a tape using the /dev/sg7:

maceadmin@amacewrk05:\~$ sudo ltfs -o devname=/dev/sg7 /media/lto1 3453 LTFS14000I LTFS starting, LTFS version 2.5.0.0 (Prelim), log level 2. 3453 LTFS14058I LTFS Format Specification version 2.4.0. 3453 LTFS14104I Launched by "ltfs -o devname=/dev/sg7 /media/lto1". 3453 LTFS14105I This binary is built for Linux (x86_64). 3453 LTFS14106I GCC version is 7.5.0. 3453 LTFS17087I Kernel version: Linux version 5.4.0-70-generic (buildd@lgw01-amd64-039) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #78~18.04.1-Ubuntu SMP Sat Mar 20 14:10:07 UTC 2021 i386. 3453 LTFS17089I Distribution: NAME="Ubuntu". 3453 LTFS17089I Distribution: DISTRIB_ID=Ubuntu. 3453 LTFS14063I Sync type is "time", Sync time is 300 sec. 3453 LTFS17085I Plugin: Loading "sg" tape backend. 3453 LTFS17085I Plugin: Loading "unified" iosched backend. 3453 LTFS14095I Set the tape device write-anywhere mode to avoid cartridge ejection. 3453 LTFS30209I Opening a device through sg-ibmtape driver (/dev/sg7). 3453 LTFS30250I Opened the SCSI tape device 6.0.5.0 (/dev/sg7). 3453 LTFS30207I Vendor ID is QUANTUM . 3453 LTFS30208I Product ID is ULTRIUM-HH6 . 3453 LTFS30214I Firmware revision is J451. 3453 LTFS30215I Drive serial is 91WT082810. 3453 LTFS30285I The reserved buffer size of /dev/sg7 is 1048576. 3453 LTFS30205I RSOC (0xa3) returns -20601. 3453 LTFS30263I RSOC returns Not Ready to Ready Transition, Medium May Have Changed (-20601) /dev/sg7. 3453 LTFS30294I Setting up timeout values from vendor and device. 3453 LTFS17160I Maximum device block size is 1048576. 3453 LTFS11330I Loading cartridge. 3453 LTFS11332I Load successful. 3453 LTFS17157I Changing the drive setting to write-anywhere mode. 3453 LTFS11005I Mounting the volume. 3453 LTFS30205I READ_ATTR (0x8c) returns -20501. 3453 LTFS30263I READ_ATTR returns Invalid Field in CDB (-20501) /dev/sg7. 3453 LTFS30233I Cannot read attribute (-20501). 3453 LTFS11336I The attribute does not exist. Ignore the expected error. 3453 LTFS30216W Length mismatch is detected. (Act = 524288, resid = 0, resid_sense = 242970). 3453 LTFS30216W Length mismatch is detected. (Act = 524288, resid = 0, resid_sense = 242970). 3453 LTFS12049E Cannot read: backend call failed (-21716). 3453 LTFS17039E XML parser: failed to read a block from the medium (-21716). 3453 LTFS17037E XML parser: failed to read from XML stream. 3453 LTFS17273E XML parser: Detected an I/O error on reading an index from the tape (-21716). 3453 LTFS17016E Cannot parse index direct from medium (-21716). 3453 LTFS11194W Cannot read index: failed to read and parse XML data (-21716). 3453 LTFS11024E Cannot mount volume: read index failed on the index partition. 3453 LTFS14013E Cannot mount the volume. 3453 LTFS30252I Logical block protection is disabled.

piste2750 commented 3 years ago

For the model C drive, I think you can try the --enable-buggy-ifs option in the configure script.

https://github.com/LinearTapeFileSystem/ltfs/wiki/HBA-info#note-for-buggy-hbas

bishbashbackup commented 3 years ago

Hi @piste2750,

Yep that seems to have got the model C drive able to load tapes. The HBA being used is an ATTO ExpressSAS H680. Not sure I've fully understood this explanation:

LTFS doesn't support the HBAs which doesn't handle the transfer length of SCSI data by default. The reason is because the safety of the data but LTFS provides a option to relax this limitation.

Does this mean that I'm at risk of dodgy file transfers if I continue to use this HBA? I only ask because I've got a different model of HBA which I could replace it with, if that would be preferable?

The Model B drive has still got the same issue it had before.

piste2750 commented 3 years ago

Basically, HBA shall return residual length returned from tape drive to the caller to have actual transfer length correctly. But the driver of this HBA doesn't do this.

--enable-buggy-ifs option makes the code to fetch the actual transfer length detected by HBA. I don't know it is actually at risk or not, but HBA drops some information from the tape drive. I don't recommend to use such kind of HBA.

For the model B drive, it looks some log sense page is not implemented. I believe it would be supported easily if I can have the SCSI specs of both model B and model C.

3274 LTFS30232I Cannot get remaining capacity: failed to parse the log page.
3274 LTFS12030E Cannot get capacity data: backend call failed (-21700).
3274 LTFS11999E Cannot load the medium: failed to get capacity data (-21700).
3274 LTFS11331E Failed to load the cartridge (ltfs_load_tape).
3274 LTFS30232I Cannot get remaining capacity: failed to parse the log page.
3274 LTFS12030E Cannot get capacity data: backend call failed (-21700).
3274 LTFS11999E Cannot load the medium: failed to get capacity data (-21700).
3274 LTFS14075E Cannot set up tape drive.
TulioLazarini commented 3 years ago

Does this mean that I'm at risk of dodgy file transfers if I continue to use this HBA? I only ask because I've got a different model of HBA which I could replace it with, if that would be preferable?

I also have a Quantum internal HH drive (LTO-8), and using a LSI SAS 9207-8i controller with it, flashed in IT mode. Works like a charm with Bacula, and started to run tests with LTFS recently - so far, so good.

piste2750 commented 3 years ago

I made a quick hack for the model B. Please feel free to try. (But I didn't confirm any build on this patch...)

diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c
index f583079..1169e44 100644
--- a/src/tape_drivers/linux/sg/sg_tape.c
+++ b/src/tape_drivers/linux/sg/sg_tape.c
@@ -513,6 +513,12 @@ static int _raw_open(struct sg_data *priv)
                return -EDEV_DEVICE_UNSUPPORTABLE; /* Unsupported device */
        }

+       /* Special care about Quantum model B */
+       if (priv->vendor == VENDOR_QUANTUM) {
+               if (id_data.product_id[7] == ' ')
+                       priv->vendor = VENDOR_QUANTUM_B;
+       }
+
        if (priv->drive_serial[0]) {
                /* if serial number is already set, compare it */
                if (strcmp(priv->drive_serial, id_data.unit_serial)) {
@@ -2969,7 +2975,8 @@ int sg_remaining_capacity(void *device, struct tc_remaining_cap *cap)
        memset(buffer, 0, LOGSENSEPAGE);

        if ((IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) ||
-               (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))) {
+               (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))
+               (priv->vendor == VENDOR_QUANTUM_B && IS_LTO(priv->drive_type))) {

                /* Use LogPage 0x31 */
                ret = sg_logsense(device, (uint8_t)LOG_TAPECAPACITY, (uint8_t)0, (void *)buffer, LOGSENSEPAGE);
diff --git a/src/tape_drivers/tape_drivers.h b/src/tape_drivers/tape_drivers.h
index a9d7bf5..6e11771 100644
--- a/src/tape_drivers/tape_drivers.h
+++ b/src/tape_drivers/tape_drivers.h
@@ -178,6 +178,7 @@ enum {
        VENDOR_IBM,
        VENDOR_HP,
        VENDOR_QUANTUM,
+       VENDOR_QUANTUM_B,
 };
bishbashbackup commented 3 years ago

Thanks for the patch!

However, it failed when doing the make:

sg_tape.c:2978:58: error: called object is not a function or function pointer
   (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))
   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Makefile:511: recipe for target 'libtape_sg_la-sg_tape.lo' failed
make[3]: *** [libtape_sg_la-sg_tape.lo] Error 1

I'm assuming it was just missing an "OR"? quantum_b_patch.txt

Or does that whole line need deleting?

piste2750 commented 3 years ago

Sorry for that.

You are right. it is just missing OR ||. I made a tiny and easy mistake because I used a poor editor...

bishbashbackup commented 3 years ago

Just tested, and it's working for me. Thanks so much!

So, would be great if we could get that committed. :D

piste2750 commented 3 years ago

Good to hear that.

Sure, I will create a PR. But let me have a time.

I would like to integrate vendor code, like VENDOR_QUANTUM_B, into the supported drive lists.

bishbashbackup commented 3 years ago

Super. Will close the issue. Thanks again!