Define the type of the change your pull request will bring to the repo
[ ] Change Request
[ ] Bug Fix
[x] New Feature
[ ] Documentation
Breaking Changes
Does your pull request introduce breaking changes?
[x] Yes, this pull request will introduce breaking changes
[ ] No, no breaking changes will be introduced
Description
This PR brings the ability to work with nested fields for sources.
Like it gives us an ability to consume data from various APIs and query structures like
Also, Mongodb snapshot streaming improvements.
Added snapshot event type so we can use sink batch inserts for mongodb or postgresql to perform initial load faster
Testing
Describe how you have tested the changes or how we should test the changes:
[x] I have added new tests for my changes
[x] I have successfully run and passed all tests with my changes locally
[ ] I have not added any testing
Checklist
Please confirm the following:
[x] I have made sure all new and existing tests passed
[ ] I have updated any necessary documentation
[x] I have ensured all commits are neat and tidy
Reviewers
Please add any appropriate reviewers for your PR.
Additional context
This PR also brings some small performance issues, like redundant JSON marshaling and unmarshaling for certain data Sources. I'm going to fix that later
Tests performed
During the testing, I checked the following combinations:
Change Type
Define the type of the change your pull request will bring to the repo
Breaking Changes
Does your pull request introduce breaking changes?
Description
This PR brings the ability to work with nested fields for sources. Like it gives us an ability to consume data from various APIs and query structures like
Also, Mongodb snapshot streaming improvements. Added
snapshot
event type so we can use sink batch inserts for mongodb or postgresql to perform initial load fasterTesting
Describe how you have tested the changes or how we should test the changes:
Checklist
Please confirm the following:
Reviewers
Please add any appropriate reviewers for your PR.
Additional context
This PR also brings some small performance issues, like redundant JSON marshaling and unmarshaling for certain data Sources. I'm going to fix that later
Tests performed
During the testing, I checked the following combinations: