search

davidmoten / rxjava-extras

Utilities for use with rxjava
Apache License 2.0
269 stars 27 forks source link

Add split with limit operator (to protect against long input without delimiters) #34

Closed davidmoten closed 4 years ago

davidmoten commented 4 years ago

Add Transformers.split(maxItemLength, pattern, maxPatternLength) operator.

This is a useful operator in the circumstance that the input is not neccessarily controlled and there exists the possibility of a really long stream of strings arriving without a delimiter. This may cause the normal split operator to throw an OutOfMemoryException as it accumulates a really long string waiting for the delimiter.

Unit test coverage of new operator is complete but for some isUnsubscribed() checks. These are painful to write tests for because buffering is occurring in the state-machine operator so I've left them out.

See javadoc of new operator for more detail.

In theory there is extra allocation overhead using this operator but benchmarks show no significant difference.

codecov-io commented 4 years ago

Codecov Report

Merging #34 into master will decrease coverage by 0.06%. The diff coverage is 81.96%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master      #34      +/-   ##
============================================
- Coverage     68.74%   68.68%   -0.07%     
- Complexity      417      427      +10     
============================================
  Files            69       69              
  Lines          3558     3602      +44     
  Branches        390      402      +12     
============================================
+ Hits           2446     2474      +28     
- Misses          988      996       +8     
- Partials        124      132       +8
Impacted Files Coverage Δ Complexity Δ
.../rx/internal/operators/TransformerStringSplit.java 80.95% <81.35%> (+5.95%) 10.00 <10.00> (+9.00)
...in/java/com/github/davidmoten/rx/Transformers.java 75.72% <100.00%> (+0.28%) 45.00 <2.00> (+2.00)
...tors/QueueWithResourcesNonBlockingUnsubscribe.java 66.66% <0.00%> (-12.13%) 11.00% <0.00%> (ø%)
...va/com/github/davidmoten/rx/util/MapWithIndex.java 81.81% <0.00%> (-4.55%) 3.00% <0.00%> (ø%)
...en/rx/internal/operators/OperatorBufferToFile.java 82.17% <0.00%> (-2.33%) 4.00% <0.00%> (ø%)
...oten/rx/internal/operators/FileBasedSPSCQueue.java 72.99% <0.00%> (-2.19%) 12.00% <0.00%> (ø%)
...dmoten/rx/internal/operators/RollingSPSCQueue.java 50.00% <0.00%> (-1.82%) 18.00% <0.00%> (-1.00%)
...ub/davidmoten/rx/buffertofile/DataSerializers.java 53.84% <0.00%> (-1.06%) 3.00% <0.00%> (ø%)
...om/github/davidmoten/rx/testing/TestingHelper.java 77.77% <0.00%> (-0.08%) 27.00% <0.00%> (ø%)
src/main/java/com/github/davidmoten/rx/Jaxws.java 0.00% <0.00%> (ø) 0.00% <0.00%> (ø%)
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a91d2ba...9912682. Read the comment docs.