pack

[jordens]

Four big things and some smaller tweaks in one PR (sorry @ryan-summers)

• The Packed keys implementation for very compact node keys
• Revamped the Keys/Iterator<Item=Key> stuff
• Added postcard support, which leads to very small key-value access with Packed keys
• Revamped the Iterator architecture a bit to reduce code duplication, especially in the high-complexity area

Changes:

• add packed keys
• packed tweaks
• rewrite packed
• split key, packed, rework packed
• introduce generic iterator
• port PathIter
• port IndexIter to generic iter
• add PackedIter
• make generic iter non-generic over treekey
• simplify iter creation
• Keys::next: don't be generic
• packed corner cases
• packed: aligned, docs
• rm len(), simplify iter, docs/changelog
• expand packed
• tweaks
• deref for Packed
• tweak packed
• changelog
• refactor JsonCoreSlash impl
• fix miniconf_mqtt dependency
• postcard: add blanket implementation
• add comment about ExactSizeIterator

[jordens]

but I'm not fully understanding the use case here for Packed. Why are we exposing this packed bit representation to users? It seems extremely opaque to be able to figure out the bit representation without knowing the keys in advance and having code calculate it all for you.

We are exposing this to the user in the same way we are exposing the respective serde wire format. Yes. These are both opaque in some way (like the serde wire format) but that's the point. There are tools to parse and generate them. TreeKey offers one-to-one mappings between any Keys: /+str paths, indices, and now also Packed "opaque small numbers". Imagine device telemetry. You are very bandwidth limited. The telemetry struct is a TreeKey. The different nodes emit at very different rates. You want to send (key: u8, value: [u8]): Send (key.into_lsb() as u8, telemetry.get_postcard_by_key(key)). On the host side you have to (once, e.g. at device firmware compile time), obtain the packed-path mapping. Then you can decode packed key into path and deserialize the value. Or our serial-settings with sequential-storage in flash. This now allows you to shrink the key size a lot. Faster comparison/hashing/lookup, less flash usage. Downside is only that the Packed encoding is less stable than /+str keys against field order changes and field addition/removal (but the encoding only breaks once the number of fields at a given depth crosses a power-of-two boundary). But Packed more stable against field renaming.

[jordens]

I really like the update to Keys since it makes more sense than Indices

I don't think there ever was Indices. You mean Iterator<Item: Keys>?

[ryan-summers]

I don't think there ever was Indices. You mean Iterator<Item: Keys>?

I was referring primarily to using usize slices to index into things. It makes more sense to generically iterate across the tree using Keys, which is then implemented directly for usize slices if that's how the user wants to index.

[jordens]

I was referring primarily to using usize slices to index into things. It makes more sense to generically iterate across the tree using Keys, which is then implemented directly for usize slices if that's how the user wants to index.

That was also possible before (any Iterator<Item: Key> would work). Just that Keys has a slightly different API than Iterator since it passes len as well which is needed for Packed.