Open igorbrigadir opened 1 year ago
The command line behavior is fine, but when using DataFrameConverter as a library in python code, setting output_columns effectively does nothing,
DataFrameConverter
output_columns
https://github.com/DocNow/twarc-csv/blob/main/dataframe_converter.py#L530
because that gets applied in
https://github.com/DocNow/twarc-csv/blob/main/csv_writer.py#L70
instead. Internally, DataFrameConverter always works with the full set of columns. Not sure if this is desirable or counter intuitive enough to change, it probably should be outputting the restricted set to be less awkward.
The command line behavior is fine, but when using
DataFrameConverteras a library in python code, settingoutput_columnseffectively does nothing,https://github.com/DocNow/twarc-csv/blob/main/dataframe_converter.py#L530
because that gets applied in
https://github.com/DocNow/twarc-csv/blob/main/csv_writer.py#L70
instead. Internally,
DataFrameConverteralways works with the full set of columns. Not sure if this is desirable or counter intuitive enough to change, it probably should be outputting the restricted set to be less awkward.