What is the documentation lacking? Please describe.
Please add a clear description of the output of pre-processing; specifically, describe all files, their format, schema, and encoding requirements that are output by pre-processing.
Describe the improvement you'd like
Add this description in the comments of the general_parser function
Additional context
The above enhancement will enable writing custom (scalable) pre-processors that can emit Marius input files and won't require one starting from a raw CSV file.
What is the documentation lacking? Please describe. Please add a clear description of the output of pre-processing; specifically, describe all files, their format, schema, and encoding requirements that are output by pre-processing.
Describe the improvement you'd like Add this description in the comments of the general_parser function
Additional context The above enhancement will enable writing custom (scalable) pre-processors that can emit Marius input files and won't require one starting from a raw CSV file.