Enhance FileSystemWriter: Add Support for Parquet and Delta Lake, Advanced Partitioning, and Comprehensive Tests

srdc / tofhir

Mapping toolset to migrate/transform existing datasets to HL7 FHIR

Apache License 2.0

9 stars 1 forks source link

Closed dogukan10 closed 2 months ago

dogukan10 commented 2 months ago

Enhance FileSystemWriter: Update the FileSystemWriter to support writing to Parquet and Delta Lake formats.
Implement Partitioning: Enable partitioning by resource type as well as additional columns in the DataFrame.
Add FileSystemWriter Tests: Implement comprehensive test cases to validate the functionality of FileSystemWriter.