support >6 character tape serials

[dericed]

Opening this ticket to continue the conversation which occurred within: https://github.com/amiaopensource/ltopers/issues/164, https://github.com/amiaopensource/ltopers/pull/165, and https://twitter.com/AVpres/status/964199679771717632.

I did find once citation in a tolisgroup.com page which referred an 8 character tape serial, see this article.

However the CLI for mkltfs from IBM, Quantum, Tandberg, list a 6 character restriction on tape-serial such as

  -s, --tape-serial=<id>    Tape serial number (6 alphanumeric ASCII characters)

The SNIA LTFS Format Specification also lists the "volume serial number" as having a length of 6. Elsewhere in the specification is defines ltfs.volumeSerial as "Serial number for the LTFS Volume specified at format time", which I'm presuming means that "volume serial number" of the specification means the same as "tape serial" within the ltfs/mkltfs cli.

I may write to tolisgroup to follow up on their article to find a reference to the "Library version of the LTFS tools" that they mention. Ideally, if there are ltfs/mkltfs utilities using an alternate tape serial of a different length than 6, than I'd like to find a way to query the ltfs/mkltfs help docs to determine the acceptable range, length, and regex of tape serial and apply the correct one, defaulting to the LTFS specification of 6 if unknown. To do so requires seeing the ltfs/mkltfs of a version that does not use the standard 6 character tape serial to figure out how to try to query it to determine that length.

If you have any information or citations about non-6-char tape serials working with ltfs/mkltfs, please share.

[dericed]

I got an email back from an engineer at an LTO producing company that said that ltfs library editions can use a media type extension within the tape serial which lets the library know (in a setting with potentially multiple generations of media) which drive to use for the tape. I've searched for a download for an "ltfs library edition" but the document for those which exist either indicate downloads that are offline or indicate that the ltfs le software is on a CD that accompanies the library. I'd like to find the ltfs -h output of library editions of ltfs and mkltfs, so that the regex of the tape serial can be inferred from the edition of the utilities installed.

[retokromer]

Glad that «an engineer at an LTO producing company» confirmed what I said. I will post – probably tomorrow evening – some information on the three implementations we did realise for customers.

[retokromer]

Since 2003, AV Preservation by reto.ch has installed or supervised the installation of LTO-based solutions for literally hundreds of customers from the archival community. As much as possible, we have always used open-source solutions, for evident reasons. Three of those installations do use the 8-char cartridge serial instead of the 6-char, i.e. much less than 1%. I have to add that I do not like LTO, but this is indeed, in my personal opinion, the least worst solution that exists in real life.

During the last years we have used whenever possible LTOpers, because it was a very nice choice in our opinion. This is the reason why I intensively contributed in maintaining LTOpers. End of December 2016 I have added the 8-char serial to the 6-char in the intention to make LTOpers more widely compatible. I could not find a way to automate the choice and nobody has argued against this format extension one year ago.

Sadly, the current version of LTOpers is no longer working on these three installations I know. And, since Monday this week, I am trying to find the informations that @dericed wishes. This is what I know today and what I can share on a public platform like GitHub:

1. One installation is based on an LTO/LTFS all-in-all solution from the USA. The manufacturer did not answer my technical questions and sent me to their European representative. The European representative did not answer my technical questions either and sent my to the reseller in Germany, which covers Switzerland as well. The reseller is… just a reseller: I could buy for EUR 10,018.80 plus VAT the software only! Through the local reseller, my technical questions went back to the European representative, who promised to contact the headquarters in the USA. That is one situation at the moment.

2. Another installation do include an ltfs -h which has simply be copy-and-pasted from an implementation made by… another manufacturer! Tandberg and HPE seem to be more popular than their… actual sellings! This does not help at all to differentiate between 6-char and 8-char in an easy way.

3. For the moment, I could not access to the third installation I know. However, as I will be in Paris for the FIAF Audio-Visual Training and Outreach Summit on the 7^th of March, I bought another train ticket from Lausanne to Paris and I booked an additional night in Paris, and I will go to the archive personally to see if something more useful can be found and shared publicly.

Hope this helps!

[retokromer]

Two additional pieces of information:

1. There are many LTFS implementers! Ultrium – aka HPE, IBM and Quantum – do currently list 35 of them on their website. We have tested and used only a dozen of them at maximum. As said above, «newer» implementations seems often to be based on copy-and-pasting older existing ones, sometimes even from other manufacturers.

2. There is also the «bastard» format LTO-8 Type M, in short M8, to consider, which uses LTO-7 cartridges with a kind of LTO-8 formatting. For the moment we advise to not use this in an archival context, because the compatibility in not clear enough to us. It uses the bar-code label xxxxxxM8 and, at least in one case, I personally suspect that the same is used as cartridge serial as well – which would not longer be a cartridge type, but become a «density type» in this case.

[retokromer]

I could play a little with a library installed in Paris. This implementation is completely crazy! It says

-s, --tape-serial=<id>    Tape serial number (6 alphanumeric ASCII characters)

in mkltfs -h but when using -s TEST01 gives the error message

LTFS15029E Tape serial must be 6 characters

and it works very fine with -s TEST01L7… Visibly it’s not intended to be used outside their black box, even it’s working when programmed by «wrong» Bash scripts or via the Terminal.

[dericed]

Perhaps the vendor is not using the RFC2119 understanding of the term "must". If there's no sane way for the script to determine the appropriate regex (since the help page of some instances with alternate regex is wrong), then perhaps a flag to indicate that it is a black-box version and to disable the regex choice is best. In non-black-box environments the "Tape serial must be 6 characters" statement appears to be true.

[retokromer]

perhaps a flag to indicate that it is a black-box version and to disable the regex choice is best

Or, possibly, a flag to use the alternate regex, which checks the consistency in that world, would be even better.

[retokromer]

I confirm that a bar-code label of type xxxxxxM8 may be used as cartridge serial in non-standard systems. This case was not considered in the regex from 2016 and might be added.

In the meantime… I am still waiting for the answer from the USA vendor.

[retokromer]

@dericed Does https://github.com/amiaopensource/ltopers/pull/177 resolve this for you?

[dericed]

works for me