search

sharksforarms / deku

Declarative binary reading and writing: bit-level, symmetric, serialization/deserialization
Apache License 2.0
1.05k stars 54 forks source link

Document the "logging" feature (was "Debugging faciility") #362

Closed vext01 closed 7 months ago

vext01 commented 9 months ago

Hi,

I've written a couple of packet/bytecode decoders using Deku. It's great, but there's one thing I think could really help: some way to get runtime decoding information to help with debugging.

If you are decoding a large chunk of data in one go, then if parsing fails, the information you are often isn't really enough to figure out what's happening.

It would be useful to get some kind of trace out of Deku so that it is easier to hone in on the cause of an error.

Perhaps there could be some kind of env_logger support that causes deku to print what field it is decoding and the value it got?

Apologies if this already exists, but I don't see it.

Cheers

vext01 commented 9 months ago

I've just found the logging feature.

It somewhat helps, but it would be better if it could print the result of parsing each sub-object, instead of what will be parsed (and a bit bag of bits).

wcampbell0x2a commented 9 months ago

Sorry that isn't documented, I hope to document that feature better in the future.

One thing to note is that logging is improved by this MR: https://github.com/sharksforarms/deku/pull/352.

I'll make sure to add the value that was decoded. I've wanted that also when debugging my decoders.

vext01 commented 9 months ago

Thanks so much, and thanks again for Deku. I think it's neat!

vext01 commented 9 months ago

(Off-topic: I also note that you are doing benchmarking in that PR -- Good! Because I had wondered how quick deku was to parse compared to a manually written parser. It might be good to say a few words about performance somewhere in your docs/README, and if you have performance numbers, possibly show those too? The things I have been parsing haven't required bitwise granularity (I always read whole bytes), and I wonder if I'm paying for that in terms of performance?)

wcampbell0x2a commented 9 months ago

(Off-topic: I also note that you are doing benchmarking in that PR -- Good! Because I had wondered how quick deku was to parse compared to a manually written parser. It might be good to say a few words about performance somewhere in your docs/README, and if you have performance numbers, possibly show those too? The things I have been parsing haven't required bitwise granularity (I always read whole bytes), and I wonder if I'm paying for that in terms of performance?)

Currently right now, you suffer a small amount of performance due to the use of BitVec( a really small amount in the grand scheme of most program runtimes). But when all the MR's are merged we will stay the same as just writing your own ( I have some benchmarks that I haven't uploaded that will track that).

vext01 commented 7 months ago

I've just tried the improved logging from https://github.com/sharksforarms/deku/pull/352

Much better! I hope if gets merged very soon!

vext01 commented 7 months ago

I also realised that logging isn't documented. I will raise a PR soonish.