Closed rogpeppe closed 3 years ago
We want to be able to decode some legacy points that contain non-ASCII data, so provide that functionality with a SetLegacy method.
SetLegacy
Also fix the non-legacy parser so that it rejects non-printable ASCII in tokens.
Performance doesn't seem to be greatly affected:
name old time/op new time/op delta DecodeEntriesSkipping/long-lines-8 26.2ms ± 1% 26.1ms ± 1% ~ (p=0.095 n=5+5) DecodeEntriesSkipping/long-lines-with-escapes-8 30.1ms ± 0% 26.3ms ± 0% -12.61% (p=0.008 n=5+5) DecodeEntriesSkipping/single-short-line-8 426ns ± 1% 436ns ± 1% +2.44% (p=0.032 n=5+4) DecodeEntriesSkipping/single-short-line-with-escapes-8 435ns ± 1% 439ns ± 1% ~ (p=0.056 n=5+5) DecodeEntriesSkipping/many-short-lines-8 283ms ± 2% 283ms ± 0% ~ (p=0.690 n=5+5) DecodeEntriesSkipping/field-key-escape-not-escapable-8 383ns ± 0% 398ns ± 2% +3.81% (p=0.008 n=5+5) DecodeEntriesSkipping/tag-value-triple-escape-space-8 461ns ± 1% 472ns ± 2% +2.37% (p=0.008 n=5+5) DecodeEntriesSkipping/procstat-8 5.58µs ± 0% 5.58µs ± 1% ~ (p=0.500 n=5+5) DecodeEntriesWithoutSkipping/long-lines-8 26.0ms ± 0% 25.9ms ± 1% ~ (p=0.095 n=5+5) DecodeEntriesWithoutSkipping/long-lines-with-escapes-8 101ms ± 1% 101ms ± 0% ~ (p=0.841 n=5+5) DecodeEntriesWithoutSkipping/single-short-line-8 405ns ± 0% 420ns ± 1% +3.70% (p=0.016 n=4+5) DecodeEntriesWithoutSkipping/single-short-line-with-escapes-8 429ns ± 1% 442ns ± 1% +2.95% (p=0.008 n=5+5) DecodeEntriesWithoutSkipping/many-short-lines-8 267ms ± 1% 279ms ± 0% +4.66% (p=0.008 n=5+5) DecodeEntriesWithoutSkipping/field-key-escape-not-escapable-8 368ns ± 1% 383ns ± 1% +3.98% (p=0.008 n=5+5) DecodeEntriesWithoutSkipping/tag-value-triple-escape-space-8 461ns ± 0% 489ns ± 9% +5.98% (p=0.008 n=5+5) DecodeEntriesWithoutSkipping/procstat-8 8.22µs ± 0% 8.28µs ± 0% +0.72% (p=0.008 n=5+5) Encode/strict/100-points-8 97.2µs ±54% 102.1µs ±56% ~ (p=0.841 n=5+5) Encode/strict/1-point-8 765ns ± 2% 761ns ± 0% ~ (p=0.302 n=5+4) Encode/lax/100-points-8 57.2µs ± 1% 57.0µs ± 3% ~ (p=0.886 n=4+4) Encode/lax/1-point-8 567ns ± 7% 581ns ± 3% ~ (p=0.151 n=5+5) name old speed new speed delta DecodeEntriesSkipping/long-lines-8 1.00GB/s ± 1% 1.00GB/s ± 1% ~ (p=0.095 n=5+5) DecodeEntriesSkipping/long-lines-with-escapes-8 872MB/s ± 0% 998MB/s ± 0% +14.43% (p=0.008 n=5+5) DecodeEntriesSkipping/single-short-line-8 68.1MB/s ± 1% 66.5MB/s ± 1% -2.39% (p=0.032 n=5+4) DecodeEntriesSkipping/single-short-line-with-escapes-8 73.6MB/s ± 1% 72.9MB/s ± 1% ~ (p=0.056 n=5+5) DecodeEntriesSkipping/many-short-lines-8 92.8MB/s ± 2% 92.5MB/s ± 0% ~ (p=0.690 n=5+5) DecodeEntriesSkipping/field-key-escape-not-escapable-8 86.1MB/s ± 0% 82.9MB/s ± 2% -3.65% (p=0.008 n=5+5) DecodeEntriesSkipping/tag-value-triple-escape-space-8 108MB/s ± 1% 106MB/s ± 2% -2.31% (p=0.008 n=5+5) DecodeEntriesSkipping/procstat-8 238MB/s ± 0% 238MB/s ± 1% ~ (p=0.548 n=5+5) DecodeEntriesWithoutSkipping/long-lines-8 1.01GB/s ± 0% 1.01GB/s ± 1% ~ (p=0.095 n=5+5) DecodeEntriesWithoutSkipping/long-lines-with-escapes-8 259MB/s ± 1% 259MB/s ± 0% ~ (p=0.841 n=5+5) DecodeEntriesWithoutSkipping/single-short-line-8 71.5MB/s ± 0% 69.0MB/s ± 1% -3.56% (p=0.016 n=4+5) DecodeEntriesWithoutSkipping/single-short-line-with-escapes-8 74.6MB/s ± 1% 72.5MB/s ± 1% -2.86% (p=0.008 n=5+5) DecodeEntriesWithoutSkipping/many-short-lines-8 98.2MB/s ± 1% 93.8MB/s ± 0% -4.45% (p=0.008 n=5+5) DecodeEntriesWithoutSkipping/field-key-escape-not-escapable-8 89.7MB/s ± 1% 86.2MB/s ± 1% -3.83% (p=0.008 n=5+5) DecodeEntriesWithoutSkipping/tag-value-triple-escape-space-8 108MB/s ± 0% 102MB/s ± 9% -5.43% (p=0.008 n=5+5) DecodeEntriesWithoutSkipping/procstat-8 161MB/s ± 0% 160MB/s ± 0% -0.71% (p=0.008 n=5+5) Encode/strict/100-points-8 214MB/s ±40% 208MB/s ±42% ~ (p=0.841 n=5+5) Encode/strict/1-point-8 252MB/s ± 2% 253MB/s ± 0% ~ (p=0.413 n=5+4) Encode/lax/100-points-8 337MB/s ± 1% 339MB/s ± 3% ~ (p=0.886 n=4+4) Encode/lax/1-point-8 341MB/s ± 7% 333MB/s ± 3% ~ (p=0.151 n=5+5) name old alloc/op new alloc/op delta DecodeEntriesSkipping/long-lines-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesSkipping/long-lines-with-escapes-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesSkipping/single-short-line-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesSkipping/single-short-line-with-escapes-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesSkipping/many-short-lines-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesSkipping/field-key-escape-not-escapable-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesSkipping/tag-value-triple-escape-space-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesSkipping/procstat-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/long-lines-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/long-lines-with-escapes-8 19.5kB ± 0% 19.5kB ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/single-short-line-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/single-short-line-with-escapes-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/many-short-lines-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/field-key-escape-not-escapable-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/tag-value-triple-escape-space-8 512B ± 0% 512B ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/procstat-8 512B ± 0% 512B ± 0% ~ (all equal) Encode/strict/100-points-8 110kB ±12% 112kB ±12% ~ (p=0.333 n=5+5) Encode/strict/1-point-8 1.18kB ± 1% 1.19kB ± 0% +1.65% (p=0.008 n=5+5) Encode/lax/100-points-8 111kB ± 0% 112kB ± 2% +0.99% (p=0.016 n=4+5) Encode/lax/1-point-8 1.11kB ± 1% 1.14kB ± 1% +2.39% (p=0.008 n=5+5) name old allocs/op new allocs/op delta DecodeEntriesSkipping/long-lines-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesSkipping/long-lines-with-escapes-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesSkipping/single-short-line-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesSkipping/single-short-line-with-escapes-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesSkipping/many-short-lines-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesSkipping/field-key-escape-not-escapable-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesSkipping/tag-value-triple-escape-space-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesSkipping/procstat-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/long-lines-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/long-lines-with-escapes-8 8.00 ± 0% 8.00 ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/single-short-line-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/single-short-line-with-escapes-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/many-short-lines-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/field-key-escape-not-escapable-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/tag-value-triple-escape-space-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) DecodeEntriesWithoutSkipping/procstat-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) Encode/strict/100-points-8 0.00 0.00 ~ (all equal) Encode/strict/1-point-8 0.00 0.00 ~ (all equal) Encode/lax/100-points-8 0.00 0.00 ~ (all equal) Encode/lax/1-point-8 0.00 0.00 ~ (all equal)
abandoning for now
rogpeppe
commented
3 years ago rogpeppe
commented
3 years ago
We want to be able to decode some legacy points that contain non-ASCII data, so provide that functionality with a
SetLegacymethod.Also fix the non-legacy parser so that it rejects non-printable ASCII in tokens.
Performance doesn't seem to be greatly affected: