Question - A response coming back as hex?

[mvniekerk]

Hi all, thank you for the crate. I need to decode a 64bit value sent as a hex response, are there any way to annotate / derive this in atat?

[mvniekerk]

At the moment I'm doing a heapless::String<32> and be done with it but this feels wrong

[jamwaffles]

I encountered the same issue and wrote a custom visitor for serde to use. Apologies for the macro - I needed to decode multiple types :). The key is to use u64::from_str_radix. The rest of the code is just serde integration for a bit of ergonomics.

struct HexLiteralVisitor<T> {
    _ty: PhantomData<T>,
}

macro_rules! impl_hex_literal_visitor {
    ($int_type:ty) => {
        impl<'de> Visitor<'de> for HexLiteralVisitor<$int_type> {
            type Value = $int_type;

            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                formatter.write_str("an unsigned integer in hexadecimal notation")
            }

            fn visit_bytes<E>(self, v: &[u8]) -> Result<Self::Value, E>
            where
                E: de::Error,
            {
                let s = std::str::from_utf8(v)
                    .map_err(serde::de::Error::custom)?
                    .replace("0x", "");

                let res = <$int_type>::from_str_radix(&s, 16).map_err(serde::de::Error::custom)?;

                Ok(res)
            }
        }
    };
}

impl_hex_literal_visitor!(u8);
impl_hex_literal_visitor!(u128);
impl_hex_literal_visitor!(u64);

I had to wrap the u64 in a newtype so I could use the visitor defined above:

pub struct HexThing(pub u64);

impl<'de> Deserialize<'de> for HexThing {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: serde::Deserializer<'de>,
    {
        let out = deserializer.deserialize_bytes(HexLiteralVisitor::<u64> { _ty: PhantomData })?;

        Self::from_bits(out).ok_or_else(|| de::Error::custom("Invalid variant"))
    }
}

Then your response struct would use HexThing instead of a raw u64.

I've copied this code from a working crate but haven't tested the examples posted here so YMMV but I'm happy to help if you hit any errors.

[mvniekerk]

Hi @jamwaffles - Thank you for the response, I'll try it out! First out of the gate is the std::str::from_utf8 - I'm doing this on embedded so this will need tweaking. I'll report back here then when I get something working.

[mvniekerk]

I've got something that compiles at least:

use core::marker::PhantomData;
use serde::de::Visitor;
use serde::*;
use core::fmt;

struct HexLiteralVisitor<T> {
    _ty: PhantomData<T>,
}

pub struct HexStr<T> (pub T);

macro_rules! impl_hex_literal_visitor {
    ($($int_type:ty)*) => {$(
        impl<'de> Visitor<'de> for HexLiteralVisitor<$int_type> {
            type Value = $int_type;

            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                formatter.write_str("an unsigned integer in hexadecimal notation")
            }

            fn visit_bytes<E>(self, v: &[u8]) -> Result<Self::Value, E>
            where
                E: de::Error,
            {

                let mut s = core::str::from_utf8(v)
                    .map_err(serde::de::Error::custom)?;
                if s.starts_with("0x") || s.starts_with("0X") {
                    s = &s[2..];
                }

                let res = <$int_type>::from_str_radix(&s, 16).map_err(serde::de::Error::custom)?;

                Ok(res)
            }
        }

    impl<'de> Deserialize<'de> for HexStr<$int_type> {
        fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
            where
                D: serde::Deserializer<'de>,
        {
            let val = deserializer.deserialize_bytes(HexLiteralVisitor::<$int_type> { _ty: PhantomData })?;
            Ok(HexStr(val))
        }
    }
    )*}
}

impl_hex_literal_visitor! { u8 u16 u32 }
impl_hex_literal_visitor! { u64 u128 }

[mvniekerk]

I'm also seeing this in the atat code:

// NOTE(deserialize_*signed) we avoid parsing into u64 and then casting to a smaller integer, which
// is what upstream does, to avoid pulling in 64-bit compiler intrinsics, which waste a few KBs of
// Flash, when targeting non 64-bit architectures

[mvniekerk]

I've had a look at the ser/de code on atat. I don't see an elegant way to add hex value parsing. I need a u128 (AppKey for LoRaWAN) to be parsed from hex. Anyway, keeping it as str and living happily on.

[jamwaffles]

First out of the gate is the std::str::from_utf8 - I'm doing this on embedded so this will need tweaking.

Right, yes, this is from an embedded Linux system :sweat_smile:

Well done jumping through the no_std hoops, but it's a shame it's still not that useful due to size :(. Could you instead store the value as a [u8; 8]?

[mvniekerk]

After some fidling, some code that does away with the from_str_radix:

use atat::nom::InputIter;
use core::fmt;
use core::marker::PhantomData;
use core::ops::{Deref, Shl};
use serde::de::Visitor;
use serde::*;

struct HexLiteralVisitor<T> {
    _ty: PhantomData<T>,
}

pub struct HexStr<T>(pub T);

macro_rules! impl_hex_literal_visitor {
    ($($int_type:ty)*) => {$(
        impl<'de> Visitor<'de> for HexLiteralVisitor<$int_type> {
            type Value = $int_type;

            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                formatter.write_str("an unsigned integer in hexadecimal notation")
            }

            fn visit_bytes<E>(self, v: &[u8]) -> Result<Self::Value, E>
            where
                E: de::Error,
            {
                let mut s = core::str::from_utf8(v)
                    .map_err(serde::de::Error::custom)?;
                if s.starts_with("0x") || s.starts_with("0X") {
                    s = &s[2..];
                }

                let mut ret: $int_type = 0;

                for c in s.iter_elements() {
                    let v = match c {
                        '0'..='9' => (c as $int_type) - ('0' as $int_type),
                        'A'..='F' => 0xa + ((c as $int_type) - ('A' as $int_type)),
                        'a'..='f' => 0xa + ((c as $int_type) - ('a' as $int_type)),
                        _ => 0
                    };

                    ret = ret
                        .shl(4)
                        .ok_or(serde::de::Error::custom("Invalid number"))?
                        .checked_add(v)
                        .ok_or(serde::de::Error::custom("Invalid number"))?;
                }

                Ok(ret)
            }
        }

        impl<'de> Deserialize<'de> for HexStr<$int_type> {
            fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
                where
                    D: serde::Deserializer<'de>,
            {
                let val = deserializer.deserialize_bytes(HexLiteralVisitor::<$int_type> { _ty: PhantomData })?;
                Ok(HexStr(val))
            }
        }

        impl Deref for HexStr<$int_type> {
            type Target = $int_type;

            fn deref(&self) -> &Self::Target {
                &self.0
            }
        }
    )*}
}

impl_hex_literal_visitor! { u8 u16 u32 u64 u128 }

[mvniekerk]

Thinking of making a PR for the ATAT crate... 🤔

[mvniekerk]

That was the de part, now for the ser part...

[MathiasKoch]

PR would be extremely welcomed! 🎉

Though i fail to see the issue with str_radix?

[mvniekerk]

From this:

// NOTE(deserialize_*signed) we avoid parsing into u64 and then casting to a smaller integer, which
// is what upstream does, to avoid pulling in 64-bit compiler intrinsics, which waste a few KBs of
// Flash, when targeting non 64-bit architectures

So... according to that I did not do the from_str_radix

[mvniekerk]

https://github.com/BlackbirdHQ/atat/pull/135

Fresh PR.