Stranger6667 / jsonschema-rs

JSON Schema validation library
https://docs.rs/jsonschema
MIT License
482 stars 90 forks source link

feat: configure patterns regex engine #487

Open shumkov opened 1 month ago

shumkov commented 1 month ago

Fancy Regex supports backtracking which is required for some cases but as a downside is vulnerable to ReDoS attacks. This becomes a decisive factor when an application operates with user-defined schemas. Regex, in turn, doesn't support look-around and backreferences but guarantees linear time matching that mitigates the attack.

This PR enables the configuration of the regex engine for pattern-based keywords: Regex or FancyRegex (by default).

use jsonschema::{CompilationOptions, RegexEngine, RegexOptions};
let mut options = CompilationOptions::default();
// Set Regex as a default engine for pattern keyword
options.with_patterns_regex_engine(RegexEngine::Regex(RegexOptions {
  size_limit: Some(5 * (1 << 20)),
  ..Default::default()
}));

The formats still use Fancy Regex. I didn't find a simple way to keep patterns static and configurable at the same time. Probably, the right approach is to add an option to use fast formats such as ajv-formats but this is out of the scope of this PR.

shumkov commented 3 weeks ago

Hello @Stranger6667! Will it be interesting for you guys or it's just my specific use case?

Stranger6667 commented 3 weeks ago

This looks cool! Sorry for the delay, I’ll check it in detail in the next couple of days and will let you know

codecov[bot] commented 3 weeks ago

Codecov Report

Attention: Patch coverage is 67.05882% with 28 lines in your changes missing coverage. Please review.

Project coverage is 89.69%. Comparing base (8adae12) to head (7b00a24).

Files Patch % Lines
jsonschema/src/regex.rs 51.16% 21 Missing :warning:
jsonschema/src/compilation/options.rs 63.63% 4 Missing :warning:
jsonschema/src/keywords/pattern.rs 70.00% 3 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #487 +/- ## ========================================== - Coverage 89.90% 89.69% -0.21% ========================================== Files 58 59 +1 Lines 9942 10007 +65 ========================================== + Hits 8938 8976 +38 - Misses 1004 1031 +27 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codspeed-hq[bot] commented 3 weeks ago

CodSpeed Performance Report

Merging #487 will degrade performances by 20.9%

Comparing dashpay:configure_regexp (7b00a24) with master (8adae12)

Summary

⚡ 15 improvements ❌ 10 regressions ✅ 295 untouched benchmarks

:warning: _Please fix the performance issues or acknowledge them on CodSpeed._

Benchmarks breakdown

Benchmark master dashpay:configure_regexp Change
fast jsonschema/is_valid/invalid 829.2 ns 945.8 ns -12.33%
jsonpointer[empty] 372.5 ns 429.7 ns -13.32%
additional_items_object 123foo[jsonschema/is_valid/valid] 1.3 µs 1.2 µs +12.55%
all_of 1[jsonschema/is_valid/invalid] 1,004.4 ns 887.8 ns +13.14%
any_of_multiple_types foo[jsonschema/is_valid/valid] 941.1 ns 853.6 ns +10.25%
any_of_multiple_types null[jsonschema/is_valid/invalid] 941.9 ns 854.4 ns +10.24%
contains 1[jsonschema/is_valid/invalid] 643.1 ns 759.7 ns -15.36%
contains 5[jsonschema/is_valid/valid] 613.1 ns 700.6 ns -12.49%
exclusive_maximum 2[jsonschema/is_valid/valid] 331.1 ns 418.6 ns -20.9%
exclusive_maximum 3[jsonschema/is_valid/invalid] 331.1 ns 418.6 ns -20.9%
exclusive_minimum 3[jsonschema/is_valid/invalid] 301.7 ns 360 ns -16.2%
exclusive_minimum 4[jsonschema/is_valid/valid] 301.7 ns 360 ns -16.2%
format_email foo[jsonschema/is_valid/invalid] 614.7 ns 527.2 ns +16.6%
format_email test@test.com[jsonschema/is_valid/valid] 624.7 ns 537.2 ns +16.29%
format_iri http//ƒøø.ßår/?∂éœ=πîx#πîüx[jsonschema/is_valid/valid] 23.6 µs 21.3 µs +10.58%
items 123[jsonschema/validate/valid] 4.3 µs 5.2 µs -18.71%
items 12x[jsonschema/validate/invalid] 8.1 µs 9.1 µs -10.33%
max_length aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa[jsonschema/is_valid/invalid] 848.1 ns 760.6 ns +11.5%
max_length foo[jsonschema/is_valid/valid] 665.6 ns 578.1 ns +15.14%
maximum 3[jsonschema/is_valid/valid] 360 ns 301.7 ns +19.34%
... ... ... ... ...


:information_source: _Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks._