This has the danger of just being another standard. However the benefit of NDEF standard is how compact it is (since it is a binary format).

However, there may be merit to adapting the NDEF standard, but as a general independent format for any current or future 2d barcodes encoding. As for why? Because it would allow for consistent interpretation of binary content (e.g. is this tag an encryption key or a phone number).

A key point to note is that NDEF does not describe anything about the nature of it's container (e.g. split tag, size of tag). The way NFC tackles this is via the use of Capability Container as described in section 6.1 of NFC Forum document "Type 2 Tag Operation Specification" http://apps4android.org/nfc-specifications/NFCForum-TS-Type-2-Tag_1.1.pdf .

Capability Container of a NFC Forum Type 2 Tag

In a type 2 nfc tag, the capability container is structured like this

Byte 0 : Magic Number "E1h" (NDEF data is present)
Byte 1 : Version Number of NFC Type 2 standard
Byte 2 : Size of NDEF data in Bytes = value * 8
Byte 3 : Read/Write Condition
- Bit 7 to 4 : General Read Access = 0x0h, Reserved = 0x1h to 0x7h, proprietary = 8h to Eh
- Bit 3 to 0 : General Write Access = 0x0h, Reserved = 0x1h to 0x7h, proprietary = 8h to Eh

Barcode NDEF container

A barcode to an application is seen as an array of bytes. Where the 2d barcode NDEF content will be structured as:

[Capability Container][NDEF Content]

So if the NDEF Content is going to be used in the context of a barcode or set of barcodes then these are some considerations:

Requirements:

Must co-exist with existing QR code usage as "ASCII text container" holding urls, short text notes, phone numbers, virtual contact cards.
Must be split-table over multiple QR codes (but not using structured append of QR as that is not portable between barcode format and is lacking support with QR reader/generators)
Compression support is recommended if tag is to be used as a storage medium of larger content (e.g. NDEF holding a mime type of a html page).

Assumption/Assertion of all 2d barcodes:

It does its own error checking of the content
It is a read only format (Unlike NFC tags). So do not need to know size of qr tag.
Most qr content in the field is ascii encoded. So any magic number should avoid printable characters.
The content received by the reader is always as a byte array. Where a byte means 8bit.

Structure Consideration of the capability container

So if the encoding of [NDEF CONTENT] is already defined in NFC Data Exchange Format (NDEF) Technical Specification NFCForum-TS-NDEF_1.0 then we only need to define the wrapper around it, which in this context is the barcode's Capability Container.

Due to the need to minimise size of barcode (for ease of scanning), the capability container should be as small as possible. If possible keeping it to a single byte would be the best, but growing as needed. So maybe the capability container is of variable length.

Inter-operating with existing URI barcodes

Since this needs to inter-operate with existing "ascii" URIs QRs in the field, this will need to avoid these symbols:

\n is a newline
\0 null char is used as an ascii end of string
Any printable ASCII code

So what this mean is that ascii packed into an 8bit char has unprintable characters only in 0001xxxx or 1xxxxxxx.

So to distinguish between ascii encoded URI QR barcodes verses this NDEF binary encoding, we need a magic number or bit to distingush between the two. 0001xxxx will minimise collision with other binary encoding that are not ascii, but you only get 4 spare bits for flags. 1xxxxxxx will give you 7 potential flag bits.

Since I have not seen many other pure binary encoding in QR code. I think it should be safe to use a single magic bit. But feel free to inform me otherwise.

Capability Container Structure

The first byte should be treated as a recognition byte. It holds the container version and compression type. Thus the magic byte could be used in a simple switch case fashion.

Byte 0 : Magic Byte

7 : 1 - Magic bit : Lets reader know it is likely to be an NDEF container. Since most printable ascii character is in 0xxxxxxx . Also ensures not null char.
6 : CompressionID bit
5 : CompressionID bit
4 : CompressionID bit
3 : Version Code?
2 : Version Code?
1 : Version Code?
0 : Version Code?

Version Code can be updated max of 16 times with this layout (Avoid if possible):

0: Represents the first released standard (In progress...)
1 to 15: Reserved

CompressionID:

0: No Compression Used
1: DEFLATE

Byte 1 : Flag Bits?

7 : Structured Append? (Has additional bytes to define checksum, total code, position etc...)
6 :
5 :
4 :
3 :
2 :
1 :
0 :
If Structured Append flag in first byte then next byte is:
- If 00aaabbb then is a short form struct append (max of 6 tag). a=pos, b=total
- If 01xxxxxx : reserved ( 7bit pos and 7bit total? for total of 128 appended tags? )
- If 10xxxxxx : reserved ( Maybe similar to above, but with checksum? or extra meta data like filename)
- If 11xxxxxx : reserved
Tags?
Filename?
Mime? (With some prefix compression perhaps)

Variable length unsigned integer...

0xxxxxxx Last entry N
1xxxxxxx N+1 (0 to 127)

00000000 = 0
01111111 = 127
11111111 01111111 = ( 127 +1 )*( 127 +1 ) - 1
11111111 11111111 01111111 = ( 127 +1 )( 127 +1 )( 127 +1 ) -1

essentially its based on max unsigned value = 2^(number of bits) -1 and how it can be split as ( 2^(7+7) ) -1 can be seen as ( 2^(7) * 2^(7) ) -1

How would an android program interpret such code?

It would first send the barcode to a decoder (from a camera app). It would then read the capability container, to work out how to handle/decompress the NDEF content. Once the NDEF content is extracted, the android app would just simple broadcast a 'fake nfc tag' intent to let the android OS handle the content as if it was an NFC tag.

This means the author of the program does not have to code in for every use case, and instead could just let the android OS handle the data instead based on the existing NFC NDEF standard.

Reference:

http://www.ascii-code.com/
NTAG210_212: http://www.nxp.com/documents/data_sheet/NTAG210_212.pdf
NFC Forum document "Type 2 Tag Operation Specification" http://apps4android.org/nfc-specifications/NFCForum-TS-Type-2-Tag_1.1.pdf
http://www.rapidtables.com/code/text/ascii-table.htm
http://www.eet-china.com/ARTICLES/2006AUG/PDF/NFCForum-TS-NDEF.pdf?SOURCES=DOWNLOAD

Portable Barcode Draft Proposal

The objective is to embed data everywhere in any kind of 2d barcodes. E.g. Size of TV, or clothing size, etc....

Just only need the barcode decoder to be able to read out binary data.

Tag Append Header

This simply focus on appending data as needed in sequence.

1st Byte (Magic)

7 : 1
6 : 0
5 : Major Version Code
4 : Major Version Code
3 : Major Version Code
2 : Major Version Code
1 : Major Version Code
0 : Major Version Code

2nd Byte (Append Sequence)

This allows for joining up to 16 tags (much like Structed Append in QR code). If each large qr tag can be 1kbytes, then that gets us 16kbytes in 16 tags.

7 : Sequence Position
6 : Sequence Position
5 : Sequence Position
4 : Sequence Position
3 : Sequence Total
2 : Sequence Total
1 : Sequence Total
0 : Sequence Total

3rd Byte (Optional. Use if Sequence Total is none zero)

Used to avoid scanning other sequences. Also simple check of scanning correctness. Parity is obtained from full content after all tag sequences is scanned.

7 : Sequence Parity
6 : Sequence Parity
5 : Sequence Parity
4 : Sequence Parity
3 : Sequence Parity
2 : Sequence Parity
1 : Sequence Parity
0 : Sequence Parity

Data Container

Once all tags is scanned, the content is joined together and has this structure.

Content Structure Settings

5 : Type Class 1
4 : Type Class 0
3 : (Size of Field 1) Type Length
2 : (Size of Field 0) Type Length
1 : (Size of Field 1) Metadata (CBOR)
0 : (Size of Field 0) Metadata (CBOR)
7 : (Size of Field 1) Payload Length
6 : (Size of Field 0) Payload Length
Type Class
- 0b00 : Registered ID (URI)
- 0b01 : Registered ID (Media)
- 0b10 : Plaintext ID (URI)
- 0b11 : Plaintext ID (Media)
Registed ID is regulated by the standard comittee
If type is unknown then type length is set to zero. Type Class is then ignored
If payload field length is 0 (omitted), then obtain length via context. (e.g. qr code)
For registed IDs the value of 0 could be
- (URI) : UUID
- (Media) : NDEF (since it is used in NFC tags)

Total Overhead

For a single tag only holding a known type (e.g. UUID, or url) , the smallest the header can get is

2(tag header)+1(content struct header)+2(known type) = 5 bytes

2018-02-06 edit : CBOR style tagdrop semiportable container

Okay, I talked to the JAB guys, and I think their approach of just relying on file signature is sort of good enough. We can reasonably assume that anyone trying to read our barcode is using our tagdrop app... thus creating some file sig like tagdrop: or td: might be good enough...

After thinking though, I think I have a different idea how to tackle this concept... CBOR has been ratified and has significant growth as an internet standard. And is very suitable for this.

Also I noticed CoAP has blockwise transfer with principles than can apply here. E.g. being able to predefine the buffer size ahead of time.

Block Level

Each code in the sequence will show the block size (in power of 2) and the total blocks needed. This will allow for ahead of time buffer allocation. Payload exact size is define within the payload. This minimises overhead of each code. The file ID is simply to avoid collision. But we can define file ID zero, to be the directory list perhaps.

With this we can keep overhead to just 4 bytes per barcode (excluding the tagdrop signiture to be determined).

Max block of 256, is a reasonable upper limit of human patience... or storage in microfilms array.

[?:TagDrop Signature][[PayloadID, PayloadSize, BlockSize (2^n), BlockTotal, BlockPos][Binary Chunk]

Payload

Once all the blocks has been assembled, you can interpret this as a CBOR structure shown below. Error checking omitted as barcode typically has error checking built in already.

{0: Binary/CBOR payload, 1:mediatype, 2:filename, x: etc...}

or as an array

[Binary/CBOR payload, mediatype, filename]

Over time, tagdrop can add more fields etc... but the idea is that using CBOR, unknown fields can be easily ignored.

mofosyne / tagdrop

Capability Container like binary header in QR code? #16