Closed gatear closed 3 months ago
Introduction of Integer Interval Class The PR contains a new class to represent an interval of integers. This would essentially aid in specifying the range of rows corresponding to batch SQL statements.
Stream Generation Method for SQL Statements New method has been introduced to generate a flow of SQL statements from a schema coupled with a limit. Support has been added to generate these commands in batch mode or individually.
Modification in SQLTransformer's Apply Method
The apply()
method in SqlTransformer
has been updated to now accept a long
rowId parameter. This modification lets it handle larger amount of data.
Updated Start and End Stream Methods
The getStartStream()
and getEndStream()
methods have been updated to throw an error when an unsupported operation is performed. The error message accompanying this is informative, allowing for better understanding of the reason for the exception.
Added Testing Method for SQL Statement Stream
A new testing method testSqlStream()
has been added which scrutinizes the generateStream()
method for generating SQL statements. The test involves generating SQL statements based on a schema and a limit, and then compares the result with the expected output. This enhances the reliability of the newly added method.
@snuyanzin it's ready for review. This work is another critical item for my use-case with Spark.
After this change I'll follow-up on this https://github.com/datafaker-net/datafaker/pull/1261#issuecomment-2172615830 and show some useful examples for Spark.
Thanks 🙏
@gatear This PR looks good to me, thank you for that, I think this should land in 2.3.0
Anyone else feedback?
It looks okay to me, but I honestly haven't worked with or on the Transformers at all.
lgtm, i left a minor comment
Attention: Patch coverage is 91.66667%
with 2 lines
in your changes missing coverage. Please review.
Project coverage is 91.92%. Comparing base (
b37c566
) to head (19c0147
). Report is 170 commits behind head on main.
Files | Patch % | Lines |
---|---|---|
.../datafaker/transformations/sql/SqlTransformer.java | 91.66% | 2 Missing :warning: |
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Generate a stream of SQL statements, support both singular and batch statements.