Fixes #
Support bitmap type of starrocks in spark context.
Problem Summary(Required) :
This minor patch introduce starrocks bitmap type into spark as native array type, while limited but sill supports read from/write into in both direction.
Before this patch, it is possible and in some case more reliable for user to choose its mapping and walk around to interact with starrocks, like doing the trick manually like this implementation.
Things to note that, as the BE do not natively support non-primitive arrow type such as Array in interaction, this patch use string as a intermedia before handing to user, which could produce huge memory impact if the resulting bitmap have a high rank.
Checklist:
[x] I have added test cases for my bug fix or my new feature
[x] This pr will affect users' behaviors
[x] This pr needs user documentation (for new or modified features or behaviors)
[ ] I have added documentation for my new feature or new function
What type of PR is this:
Which issues of this PR fixes :
Fixes # Support bitmap type of starrocks in spark context.
Problem Summary(Required) :
This minor patch introduce starrocks bitmap type into spark as native array type, while limited but sill supports read from/write into in both direction.
Before this patch, it is possible and in some case more reliable for user to choose its mapping and walk around to interact with starrocks, like doing the trick manually like this implementation.
Things to note that, as the BE do not natively support non-primitive arrow type such as Array in interaction, this patch use string as a intermedia before handing to user, which could produce huge memory impact if the resulting bitmap have a high rank.
Checklist: