pydantic / jiter

Fast iterable JSON parser.
https://crates.io/crates/jiter
MIT License
182 stars 11 forks source link

Build fails: StringChunk declared as crate-private, can't leak? #113

Closed charles-dyfis-net closed 3 months ago

charles-dyfis-net commented 3 months ago

The below is seen when building against rustc and cargo 1.73.0, as shipped in the 23.11 release of nixpkgs, on MacOS/aarch64:

python3.12-jiter> error[E0446]: crate-private type `StringChunk` in public interface
python3.12-jiter>    --> crates/jiter/src/simd_aarch64.rs:195:1
python3.12-jiter>     |
python3.12-jiter> 195 | / pub fn decode_string_chunk(
python3.12-jiter> 196 | |     data: &[u8],
python3.12-jiter> 197 | |     mut index: usize,
python3.12-jiter> 198 | |     mut ascii_only: bool,
python3.12-jiter> 199 | |     allow_partial: bool,
python3.12-jiter> 200 | | ) -> JsonResult<(StringChunk, bool, usize)> {
python3.12-jiter>     | |___________________________________________^ can't leak crate-private type
python3.12-jiter>     |
python3.12-jiter>    ::: crates/jiter/src/string_decoder.rs:166:1
python3.12-jiter>     |
python3.12-jiter> 166 |   pub(crate) enum StringChunk {
python3.12-jiter>     |   --------------------------- `StringChunk` declared as crate-private
python3.12-jiter> error[E0446]: crate-private type `StringChunk` in public interface
python3.12-jiter>    --> crates/jiter/src/string_decoder.rs:148:1
python3.12-jiter>     |
python3.12-jiter> 148 | / pub fn decode_chunk(
python3.12-jiter> 149 | |     data: &[u8],
python3.12-jiter> 150 | |     index: usize,
python3.12-jiter> 151 | |     ascii_only: bool,
python3.12-jiter> 152 | |     allow_partial: bool,
python3.12-jiter> 153 | | ) -> JsonResult<(StringChunk, bool, usize)> {
python3.12-jiter>     | |___________________________________________^ can't leak crate-private type
python3.12-jiter> ...
python3.12-jiter> 166 |   pub(crate) enum StringChunk {
python3.12-jiter>     |   --------------------------- `StringChunk` declared as crate-private
python3.12-jiter> For more information about this error, try `rustc --explain E0446`.
python3.12-jiter> error: could not compile `jiter` (lib) due to 2 previous errors
python3.12-jiter> warning: build failed, waiting for other jobs to finish...
python3.12-jiter> 💥 maturin failed
charles-dyfis-net commented 3 months ago

Looks like this may be as simple as:

diff --git a/crates/jiter/src/simd_aarch64.rs b/crates/jiter/src/simd_aarch64.rs
index b4d08ff..59adb60 100644
--- a/crates/jiter/src/simd_aarch64.rs
+++ b/crates/jiter/src/simd_aarch64.rs
@@ -192,7 +192,7 @@ const CONTROL_16: SimdVecu8_16 = simd_const!([32u8; 16]);
 const ASCII_MAX_16: SimdVecu8_16 = simd_const!([127u8; 16]);

 #[inline(always)]
-pub fn decode_string_chunk(
+pub(crate) fn decode_string_chunk(
     data: &[u8],
     mut index: usize,
     mut ascii_only: bool,
diff --git a/crates/jiter/src/string_decoder.rs b/crates/jiter/src/string_decoder.rs
index c2d9318..e196c60 100644
--- a/crates/jiter/src/string_decoder.rs
+++ b/crates/jiter/src/string_decoder.rs
@@ -145,7 +145,7 @@ fn decode_to_tape<'t, 'j>(
 }

 #[inline(always)]
-pub fn decode_chunk(
+pub(crate) fn decode_chunk(
     data: &[u8],
     index: usize,
     ascii_only: bool,

...at least, that works for me.