search

FasterXML / jackson-core

Core part of Jackson that defines Streaming API as well as basic shared abstractions
Apache License 2.0
2.25k stars 773 forks source link

Fix #1310: Count JSON tokens #1296

Closed pjfanning closed 2 months ago

pjfanning commented 3 months ago

Idea is to be able to apply a StreamReadConstraint that limits total JSON tokens that you can have in an input doc.

See https://github.com/FasterXML/jackson/discussions/231#discussioncomment-9603279

cowtowncoder commented 2 months ago

Created #1310 as the issue covering this PR.

I think implementation makes sense, as-is. So that's fine. But merging to master/3.0 may be challenging, so what I think I will do is create another PR for a subset of this one, to handle parts that are likely to cause most problems. That can be merged in 2.18, then master, and then sync pr.

cowtowncoder commented 2 months ago

@pjfanning Ok I think this is good: I merged update-token parts via separate PR into 2.18, this PR & master and remaining diffs are bit smaller and hopefully merge with few conflicts. I assume this could change from draft to regular one now? While perf tests might be nice, I feel confident that when disabled (no limit defined), overhead should be negligible.

cowtowncoder commented 2 months ago

@pjfanning Is it ok to merge this? (only asking since it's still a Draft)

pjfanning commented 2 months ago

@cowtowncoder I've marked this as ready for review