Which tar format is best?

[rainersigwald]

Question semi-unrelated to this change: @rainersigwald any reason why GNU format was used?

My assumption is that PAX was not used so that file size could be reduced, since it always created an extended attributes entry to precede each actual entry. That's fine, but just wanted to mention that PAX is the most flexible format, so if you encounter any issues, you can always try that format.

_Originally posted by @carlossanlop in https://github.com/dotnet/sdk-container-builds/pull/153#discussion_r960847413_

[rainersigwald]

It's because I was just guessing here! @carlossanlop is there a way to tell, given a .tar file, what format it uses? I'd like to match whatever docker does, but I couldn't figure that out when I tried--and I forgot to file an issue for follow up.

[carlossanlop]

is there a way to tell, given a .tar file, what format it uses?

An archive can have entries of different formats intermixed. So the format belongs to an entry itself, not to the whole archive.

but I couldn't figure that out when I tried

For reading: After collecting the next entry of an archive with GetNextEntry or GetNextEntryAsync, you can consult what format the entry by reading the .Format property in the returned TarEntry.

For writing: The TarWriter class has a constructor variant that takes a TarEntryFormat argument to specify the default format to use when writing entries using the WriteEntry(string fileName, string entryName) or WriteEntryAsync(string fileName, string entryName, CancellationToken cancellationToken) methods.

It's because I was just guessing here!

The GNU format is used as default for some tar tools, but it has the following limitations:

• It's not POSIX compatible. The other formats are.
• Does not allow setting your own metadata.
• Uses the Name and LinkName legacy fields from the header, which store strings in ASCII. Only if such strings are larger than 100 bytes, they get stored in an additional metadata entry without restrictions.

I prefer PAX because:

• Even though it can increase the size of an archive, because it adds a metadata entry that precedes every actual file entry, the metadata entry is good because it contains unrestricted information about the file. For example:
  • The file path and the link path are stored in UTF8 and can have any length.
  • If the file data section length, after getting converted to an ASCII string, is larger than 12 bytes, then it gets stored without length restrictions in the extended attributes as well.
  • Same for GName, UName, modification time.
  • You can add your own custom extended attributes if desired, as long as they don't have name collisions with the known fields.

And since I mentioned that you can intermix formats, you could add logic to decide if you want to save some entries in GNU if they don't have any size restrictions, and the rest that surpass the limits can be stored in PAX.

Hope that helps.