Currently, the lino pull command supports exporting data in JSONL format. While JSONL is a flexible and widely-used format, it may not always be suitable for all use cases. To address the needs of a broader range of users and applications, we propose enhancing the lino pull command to support multiple export formats.
New Formats to Support:
SQL: Generate SQL scripts that can be executed or saved as files to initialize a test database. This is particularly useful for testing applications that do not support JSONL, such as Spring Boot applications.
Avro/Parquet: Optimize CPU consumption and file compression by using binary serialization formats like Avro or Parquet.
Proposed Changes:
Add a --format flag to the lino pull command, allowing users to specify the export format.
Implement export options for SQL and Avro/Parquet formats.
Ensure compatibility with existing flags and options in the lino pull command.
Update the documentation to reflect the new export formats and usage examples.
Impact:
Users will have more flexibility in choosing the format that best suits their application's needs.
Improve performance and storage efficiency for users who require compressed binary formats.
Acceptance Criteria:
The lino pull command successfully exports data in the specified format (SQL, Avro, Parquet).
Documentation and usage examples are updated.
Comprehensive tests are added for each export format.
youen
commented
2 weeks ago
Currently, the
lino pullcommand supports exporting data in JSONL format. While JSONL is a flexible and widely-used format, it may not always be suitable for all use cases. To address the needs of a broader range of users and applications, we propose enhancing thelino pullcommand to support multiple export formats.New Formats to Support:
Proposed Changes:
--formatflag to thelino pullcommand, allowing users to specify the export format.lino pullcommand.Impact:
Acceptance Criteria:
lino pullcommand successfully exports data in the specified format (SQL, Avro, Parquet).