watfordjc
commented
2 years ago Data Redesign
The LTO-Encryption-SPTI application at the moment is mostly a load of constants and structs and manual parsing. Part of the reason for that is that the data is received unformatted.
Having decided to P/Invoke CreateFile() directly in LTO-Encryption-Manager rather than creating a wrapper function in a C library, I am now considering the feasibility of doing things mostly in C# rather than using a C library. The SPTI sample is C-based, and LTO-Encryption-SPTI is written in C, but almost everything is converting to structs and parsing. Sometimes the parsing is on fields of a structure that can't be converted to a struct.
In order to enable encryption, the following are needed via P/Invoke and SPTI at a minimum:
- [x] A
HANDLEto the drive. - [ ] The WWN/LUN for the drive (Device Identifiers page).
- [ ] Whether or not the drive's encryption is supported (Supported Security Protocol, Data Encryption Management Capabilities, Data Encryption Capabilities, and Supported Key Formats pages).
- [ ] Current encryption status (Data Encryption Status page).
- [ ] Drive's public key for key wrapping (Device Server Key Wrapping Public Key page).
- [ ] Tape barcode, whether the tape is formatted LTFS, whether the tape is encrypted (MAM attributes).
- [ ] First block encryption status KAD (Next Block Encryption Status KAD).
The first item, a HANDLE to the drive, is obtained by getting the drive path via CIM or an alternative method, and then (if necessary) appending a hash/octothorpe and the GUID for a tape drive (GUID_DEVINTERFACE_TAPE): #{53F5630B-B6BF-11D0-94F2-00A0C91EFB8B}, and using that full path with CreateFile().
Step one of obtaining the WWN/LUN (or anything else) requires calling DeviceIoControl() 4 times to check we can communicate with the drive, although the only thing I'm checking is that SrbType is of the newer/extended SCSI block variety STORAGE_REQUEST_BLOCK:
- Obtain a
STORAGE_DESCRIPTOR_HEADER(parsed for the size of aSTORAGE_ADAPTER_DESCRIPTOR). - Obtain a
STORAGE_ADAPTER_DESCRIPTOR(parsed for theAlignmentMaskandSrbTypevalues). - Obtain a
STORAGE_DESCRIPTOR_HEADER(parsed for the size of aSTORAGE_DEVICE_DESCRIPTOR). - Obtain a
STORAGE_DEVICE_DESCRIPTOR(parsed for theBusTypeandSerialNumber).
Next, the process of sending/getting data to/from the drive is similar for most pages. SPTI wraps the SRB and sends it to the drive. Let's look at the C code:
if (srbType == SRB_TYPE_STORAGE_REQUEST_BLOCK)
{
int pageCode;
/*
* CDB: Inquiry, Device Identifiers VPD page
*/
length = ResetSrbIn(psptwb_ex, SCSIOP_INQUIRY);
if (length == 0) { goto Cleanup; }
psptwb_ex->spt.Cdb[1] = CDB_INQUIRY_EVPD;
psptwb_ex->spt.Cdb[2] = VPD_DEVICE_IDENTIFIERS;
status = SendSrb(fileHandle, psptwb_ex, length, &returned);
if (CheckStatus(fileHandle, psptwb_ex, status, returned, length))
{
pageCode = psptwb_ex->ucDataBuf[1];
if (pageCode == VPD_DEVICE_IDENTIFIERS)
{
ParseDeviceIdentifiers((PVPD_IDENTIFICATION_PAGE)psptwb_ex->ucDataBuf, &logicalUnitIdentifierLength, &logicalUnitIdentifier);
}
}
...
}I'm going to need one of those psptwb_ex variables. That would be a PSCSI_PASS_THROUGH_WITH_BUFFERS_EX. I'm going to need to convert some C structs to C#. After spending way to long trying to marshal between C# and C and back, and ending up with a 9 byte offset issue, I have decided I am going to do it all in C# with CsWin32 and (where not covered) Win32 P/Invoke.
Let's compare what I've currently got in terms of C# code for the Device Identifiers VPD page:
public static void GetTapeDriveIdentifiers(TapeDrive tapeDrive)
{
IntPtr psptwb_ex = Marshal.AllocHGlobal(Marshal.SizeOf<SCSI_PASS_THROUGH_WITH_BUFFERS_EX>());
SCSI_PASS_THROUGH_WITH_BUFFERS_EX sptwb_ex = new();
Marshal.StructureToPtr(sptwb_ex, psptwb_ex, true);
uint length = ResetSrbIn(psptwb_ex, Constants.SCSIOP_INQUIRY);
sptwb_ex = Marshal.PtrToStructure<SCSI_PASS_THROUGH_WITH_BUFFERS_EX>(psptwb_ex);
Windows.Win32.InlineArrayIndexerExtensions.ItemRef(ref sptwb_ex.spt.Cdb, 1) = Constants.CDB_INQUIRY_EVPD;
Windows.Win32.InlineArrayIndexerExtensions.ItemRef(ref sptwb_ex.spt.Cdb, 2) = Constants.VPD_DEVICE_IDENTIFIERS;
Marshal.StructureToPtr(sptwb_ex, psptwb_ex, true);
uint returnedData = 0;
Windows.Win32.System.IO.OVERLAPPED overlapped;
bool ok;
unsafe
{
ok = Windows.Win32.PInvoke.DeviceIoControl(tapeDrive.Handle,
Windows.Win32.PInvoke.IOCTL_SCSI_PASS_THROUGH_EX,
(void*)psptwb_ex,
(uint)Marshal.SizeOf(sptwb_ex),
(void*)psptwb_ex,
length,
&returnedData,
&overlapped);
}
if (ok)
{
sptwb_ex = Marshal.PtrToStructure<SCSI_PASS_THROUGH_WITH_BUFFERS_EX>(psptwb_ex);
Trace.WriteLine(Convert.ToHexString(sptwb_ex.ucDataBuf, 0, (int)sptwb_ex.spt.DataInTransferLength));
}
Marshal.FreeHGlobal(psptwb_ex);
}ResetSrbIn() pretty much does what its name suggests: it resets an SRB for usage in the in/read direction. Having found the InlineArrayIndexerExtensions method (it doesn't even have any Google results) after hunting through IntelliSense for a way to access the 2nd byte of a 1 byte variable sized array (UCHAR[ANYSIZE_ARRAY] in Win32 C), I migrated ResetSrbIn() to C#.
There is one issue with migrating to C#: many constants and structs, such as the VPD page struct, do not appear to be available via CsWin32. C# also doesn't really do bit fields in structs. Piping the Trace.WriteLine(Convert.ToHexString(...)) output through xxd and hexdump, though, I can see the LUN (and HP_____Ultrium 6-SCSI__[SerialNumber], where _ is a space) in the bytes.
I need to parse the pages though, and with CsWin32 appearing to not have the page structs, I am going to have to implement all of them myself, including working out how to handle bit fields. This is one of the problems with manual parsing: having to reimplement the same thing again.
I am considering defining the pages using ASN.1+ECN, although there might not be a library or tool that can convert those definitions to C#.
The Compaq/HP OID 1.3.6.1.4.1.232.5.5.4.1 (iso.identified-organization.dod.internet.private.enterprise.compaq.cpqScsi.cpqSasComponent.cpqSasTapeDrv.cpqSasTapeDrvTable.cpqSasTapeDrvEntry) is a table entry for a SAS tape drive used with tape libraries that do SNMP and it has some MIB objects defined such as 1.3.6.1.4.1.232.5.5.4.1.1.8 (cpqSasTapeDrvSerialNumber):
cpqSasTapeDrvSerialNumber OBJECT-TYPE
SYNTAX DisplayString (SIZE (0..40))
ACCESS read-only
STATUS mandatory
DESCRIPTION
"SAS Tape Drive Serial Number.
This is the serial number assigned to the tape drive."
::= { cpqSasTapeDrvEntry 8 }Such objects/modules may have a use if SNMP support were desired, however getting the pages from the drive and parsing them so that such an object can be populated is something for a tape library to implement. As far as I can see, those tables in the LTO manuals are the only definitions the LTO publish.
Let's take the VPD struct from scsi.h (with comments removed), and a non-ECN translation in bullet list form:
typedef struct _VPD_IDENTIFICATION_PAGE {
UCHAR DeviceType : 5;
UCHAR DeviceTypeQualifier : 3;
UCHAR PageCode;
UCHAR Reserved;
UCHAR PageLength;
UCHAR Descriptors[0];
#endif
} VPD_IDENTIFICATION_PAGE, *PVPD_IDENTIFICATION_PAGE;- The first byte contains two options: a 5 bit peripheral device type in the lower bits, and a 3 bit peripheral qualifier in the upper bits. The values of these fields are from an enum/list (e.g.
DeviceType = 0x05means CD Device). - The page code, defining what the rest of the data is. For
VPD_IDENTIFICATION_PAGEthat would be0x83. - A reserved/padding byte.
- The page length (total length minus this byte and the 3 bytes above).
- The page (variable length sequence of
Descriptors).
Using the ASN.1 example RLC-RADIO-HANDOVER-COMPLETE (from ETSI TS 101 761-2 V1.3.1, AKA the Hiperlan specification) as a template (the specification has ASN.1 throughout and transfer syntax tables in Annex A), the ASN.1 for VPD-Identification-Page arguments would be something like this:
Device-Identification-Page-Arg ::= SEQUENCE {
device-type Device-Type
device-type-qualifier Device-Type-Qualifier
page-code Page-Code
page-length Page-Length
device-identification-descriptor-list Device-Identification-Descriptor-List
}Adding in the reserved/padding, the initial encoding structure would be something like this:
#Device-Identification-Page-Arg-struct ::= #CONCATENATION {
device-type #Device-Type
device-type-qualifier #Device-Type-Qualifier
page-code #Page-Code
aux-reserved-1 #PAD
page-length #Page-Length
device-identification-descriptor-list #Device-Identification-Descriptor-List
}[TODO...]
I can't seem to get SPIN working. Keep getting Medium not present errors.
Feature Branch
Current feature branch for this issue: not created yet.
Progress
Background
This issue was originally at watfordjc/LTO-Encryption-SPTI#5
LTO-Encryption-SPTI is currently a linear Win32 C application.
LTO-Encryption-Manager has now reached the point where it needs to use SPTI.
The goal of this issue is to implement the most basic functions as library functions, starting with opening a file descriptor for the tape drive, using a Win32 API to obtain the serial number and WWN/LUN, and then closing the file descriptor.
The Data Backups and Archiving Project contains this loosely related note: