JoshClose / CsvHelper

Library to help reading and writing CSV files
http://joshclose.github.io/CsvHelper/
Other
4.79k stars 1.07k forks source link

Simple way to add header rows for columns #2036

Open booktrakker opened 2 years ago

booktrakker commented 2 years ago

I came to realize after you answered my question about how to handle multiple headers has an elegant solution I think would be a real boon. I suspect this would be rather easy to implement so I do hope you find it worth implementing.

Basically you would add support for a new Attribute - something like [Header("headertext")] - that we would add to each property in much the same way you had me use an index. This Attribute would put the required header in the row before the header that is generated by the property name, and if done right, would allow multiple such attributes to be placed before each property to account for however many rows are needed.

An initial pass would get the max number of such rows so that any property with fewer rows can be populated with blank entries, all other columns would have the Attributes in the rows before the Property Name header row.

This would make it extremely easy to maintain without having to index the rows, which can become a maintenance issue.

In my Amazon example the class that defines the structure would have an entry something like this:

[Header("Product Type")] public string feed_product_type { get; set; }

This would result in the Column Header looking like this (where "ABIS_BOOK" is the actual value of that field for the first record exported):

Product Type feed_product_type "ABIS_BOOK"

JoshClose commented 2 years ago

There are already [Name(string name)] and [Name(params string[])] attributes to do this. You should still supply and Index because the order of the properties returned from Type.GetProperties() may not be the same as the order they appear in your class. If you have a header you may not care about the order.

I'm not going to add any features around multiple headers because it's not a valid CSV file. It's pretty easy to do yourself though, and you can put the functionality in an extension method or static method for reuse.

void Main()
{
    var records = new List<Foo>
    {
        new Foo { feed_product_type = "ABIS_BOOK" },
    };
    var config = new CsvConfiguration(CultureInfo.InvariantCulture)
    {
    };
    using (var writer = new StringWriter())
    using (var csv = new CsvWriter(writer, config))
    {
        csv.WriteHeader<Foo>();
        csv.NextRecord();

        foreach (var property in typeof(Foo).GetProperties())
        {
            csv.WriteField(property.Name);
        }
        csv.NextRecord();

        csv.WriteRecords(records);
        writer.Dump();
    }
}

private class Foo
{
    [Name("Product Type")]
    public string feed_product_type { get; set; }
}

image

booktrakker commented 2 years ago

Hi Josh,

Thank you so much for your help! I did what I thought you said to do, it did not work, so I was going to email you and I decided to take a closer look at what you put below and…

Of course I had omitted key lines, and once I replicated what you have below it worked exactly as I needed it to work.

I am truly grateful for your rapid response and excellent help, not to mention the library itself.

We struggle with the Amazon requirements due to our tiny size, and they have no clue, so to get help quickly is a real boon. I have waited more than 3 weeks for Amazon at times, so it was quite refreshing!

Regards,

Jim Gutterman

booktrakker commented 2 years ago

Hi Josh,

I implemented what you described below without indexing the properties and as you no doubt expected, headers did not always line up. I attached my Class file so you can inspect that to make sure I did it right, and two examples of what should be identical files from two different systems running the same code.

I implemented the Indexes as I understand them, and it did not line up the headers when run on one target machine while working perfectly on another.

Oddly, it always does when I run it in the Development environment but not when run as installed. I attached what should be the same file; AmazonUploadBL_1 (1).txt is from one machine and AmazonUploadBL_1 (2).txt – the valid one – is from another machine.

I am at a loss as to why this happens, but I need these headers to line up properly, so please help me understand what is going on here.

Here is the method I use to write the file:

    public static string WriteBookLoaderFile(BookLoaderList list, ErrorReporter errReport)

    {

        try

        {

            var csvConfig = new CsvConfiguration(CultureInfo.CurrentCulture)

            {

                HasHeaderRecord = true,

                Delimiter = "\t",

                Encoding = Encoding.UTF8,

                ShouldQuote = args => false

            };

            using (var writer = new StringWriter())

            using (var csv = new CsvWriter(writer, csvConfig))

            {

                csv.WriteField(cVersionHeader);

                csv.NextRecord();

                csv.WriteHeader<BookLoader>();

                csv.NextRecord();

                foreach (var property in typeof(BookLoader).GetProperties())

                {

                    if (property.Name != "Item")

                    {

                        csv.WriteField(property.Name);

                    }

                }

                csv.NextRecord();

                csv.WriteRecords(list.BookList);

                csv.Flush();

                return writer.ToString();

            }

        }

        catch (Exception ex)

        {

            errReport.LogErr(ex);

        }

        return string.Empty;

    }

}

Jim Gutterman

Director of Development

BookTrakker.com

@.> @.

From: Josh Close @.> Sent: Monday, September 19, 2022 11:19 AM To: JoshClose/CsvHelper @.> Cc: booktrakker @.>; Author @.> Subject: Re: [JoshClose/CsvHelper] Simple way to add header rows for columns (Issue #2036)

There are already [Name(string name)] and [Name(params string[])] attributes to do this. You should still supply and Index because the order of the properties returned from Type.GetProperties() may not be the same as the order they appear in your class. If you have a header you may not care about the order.

I'm not going to add any features around multiple headers because it's not a valid CSV file. It's pretty easy to do yourself though, and you can put the functionality in an extension method or static method for reuse.

void Main() { var records = new List { new Foo { feed_product_type = "ABIS_BOOK" }, }; var config = new CsvConfiguration(CultureInfo.InvariantCulture) { }; using (var writer = new StringWriter()) using (var csv = new CsvWriter(writer, config)) { csv.WriteHeader(); csv.NextRecord();

    foreach (var property in typeof(Foo).GetProperties())
    {
        csv.WriteField(property.Name);
    }
    csv.NextRecord();

    csv.WriteRecords(records);
    writer.Dump();
}

}

private class Foo { [Name("Product Type")] public string feed_product_type { get; set; } }

https://user-images.githubusercontent.com/114971/191086483-b93f8a2d-8c22-4a53-9688-f131a6323396.png

— Reply to this email directly, https://github.com/JoshClose/CsvHelper/issues/2036#issuecomment-1251379400 view it on GitHub, or https://github.com/notifications/unsubscribe-auth/ANTYX7ITF4YTISJJBNQESBDV7CVB5ANCNFSM6AAAAAAQQI4LTY unsubscribe. You are receiving this because you authored the thread. https://github.com/notifications/beacon/ANTYX7I6R5B4ZU6PP5BDAMTV7CVB5A5CNFSM6AAAAAAQQI4LT2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSKS2EMQ.gif Message ID: < @.> @.>

TemplateType=bookloader Version=2022.0922 Category=bookloader TemplateSignature=Qk9PS1NfMTk3M19BTkRfTEFURVI= settings=attributeRow=3&contentLanguageTag=en_US&dataRow=4&feedType=113&headerLanguageTag=en_US&isProcessingSummary=false&labelRow=2&primaryMarketplaceId=amzn1.mp.o.ATVPDKIKX0DER&templateIdentifier=7a8e0736-b13a-406c-b73c-f2debbdb9e43&timestamp=2022-09-23T06% Use ENGLISH to fill this template.The top 3 rows are for Amazon.com use only. Do not modify or delete the top 3 rows. Images Variation Basic Discovery Product Enrichment Fulfillment Compliance Offer
Product Type Seller SKU Product ID Product ID Type Product Name Manufacturer Author Author Author Binding Publication date Language Your Price Quantity Main Image URL Main Offer Image Other Image URL1 Offer Image Other Image URL2 Other Image URL3 Other Image URL4 Other Image URL5 Other Image URL6 Other Image URL7 Other Image URL8 Relationship Type Package Level package_contains_quantity package_contains_identifier brand_name Update Delete Manufacturer Part Number Product Description Edition Number Language Language Language Language Language Category (item-type) Series Title Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features volume pages BISAC Subject Code Format Format Format Format Format Search Terms Subject Color Map BISAC Subject Code Subject Keyword Color Foreword Adapter Afterword Preface reader Series Editor Colorist Series Number Drawings Photographer Compiler Artist Translator Editor Creator Composer Painter Contributor Introduction Series Volume Illustrator legal_compliance_certification_certifying_authority_name legal_compliance_certification_geographic_jurisdiction Offer Image1 Offer Image2 Offer Image3 Offer Image4 Offer Image5 Fulfillment Center ID Package Height Package Width Package Length Package Dimensions Unit Of Measure Package Weight Package Weight Unit Of Measure Item Weight Lithium content (grams) Lithium Battery Packaging Is this product a battery or does it utilize batteries? Batteries are Included Battery type/size Battery type/size Battery type/size Number of Lithium-ion Cells Number of Lithium Metal Cells Watt hours per battery item_weight_unit_of_measure Number of batteries Number of batteries Number of batteries Fabric Type Fabric Type Fabric Type Import Designation Battery composition Battery weight (grams) battery_weight_unit_of_measure lithium_battery_energy_content_unit_of_measure lithium_battery_weight_unit_of_measure Applicable Dangerous Goods Regulations Applicable Dangerous Goods Regulations Applicable Dangerous Goods Regulations Applicable Dangerous Goods Regulations Applicable Dangerous Goods Regulations UN number Safety Data Sheet (SDS) URL Volume item_volume_unit_of_measure GHS Classification Subcategory GHS Classification Subcategory GHS Classification Subcategory legal_compliance_certification_expiration_date Regulatory Organization Name Compliance Certification Status Cpsia Warning Cpsia Warning Cpsia Warning Cpsia Warning Flash point (°C)? legal_compliance_certification_date_of_issue CPSIA Warning Description legal_compliance_certification_metadata Legal Compliance Certification Categorization/GHS pictograms (select all that apply) Categorization/GHS pictograms (select all that apply) Categorization/GHS pictograms (select all that apply) California Proposition 65 Warning Type California Proposition 65 Chemical Names California Proposition 65 Chemical Names California Proposition 65 Chemical Names California Proposition 65 Chemical Names California Proposition 65 Chemical Names Pesticide Marking Pesticide Marking Pesticide Marking Pesticide Registration Status Pesticide Registration Status Pesticide Registration Status Pesticide Certification Number Pesticide Certification Number Pesticide Certification Number Currency Signed By Restock Date Standard Plus Handling Time Is Gift Wrap Available Dust Jacket Condition Max Order Quantity Shipping-Template Scheduled Delivery SKU List Product Tax Code Offering Can Be Gift Messaged Item Condition Offer Condition Note Expedited Shipping Will Ship Internationally Sale Price Sale Start Date Sale End Date Minimum Advertised Price Offer End Date Offer Start Date feed_product_type item_sku external_product_id external_product_id_type item_name manufacturer author1 author2 author3 binding publication_date language_published standard_price quantity main_image_url main_offer_image other_image_url1 offer_image other_image_url2 other_image_url3 other_image_url4 other_image_url5 other_image_url6 other_image_url7 other_image_url8 relationship_type package_level package_contains_quantity package_contains_identifier brand_name update_delete part_number product_description edition language_value1 language_value2 language_value3 language_value4 language_value5 item_type collection_name bullet_point1 bullet_point2 bullet_point3 bullet_point4 bullet_point5 bullet_point6 bullet_point7 bullet_point8 bullet_point9 bullet_point10 volume_number pages thema_classification_code format1 format2 format3 format4 format5 generic_keywords unknown_subject color_map bisac_subject_description subject_keywords color_name foreword adapter afterword preface reader series_editor colorist series_number drawings photographer compiler artist translator editor creator composer painter contributor introduction volume_base illustrator legal_compliance_certification_certifying_authority_name legal_compliance_certification_geographic_jurisdiction offer_image1 offer_image2 offer_image3 offer_image4 offer_image5 fulfillment_center_id package_height package_width package_length package_dimensions_unit_of_measure package_weight package_weight_unit_of_measure item_weight lithium_battery_weight lithium_battery_packaging batteries_required are_batteries_included battery_type1 battery_type2 battery_type3 number_of_lithium_ion_cells number_of_lithium_metal_cells lithium_battery_energy_content item_weight_unit_of_measure number_of_batteries1 number_of_batteries2 number_of_batteries3 fabric_type1 fabric_type2 fabric_type3 import_designation battery_cell_composition battery_weight battery_weight_unit_of_measure lithium_battery_energy_content_unit_of_measure lithium_battery_weight_unit_of_measure supplier_declared_dg_hz_regulation1 supplier_declared_dg_hz_regulation2 supplier_declared_dg_hz_regulation3 supplier_declared_dg_hz_regulation4 supplier_declared_dg_hz_regulation5 hazmat_united_nations_regulatory_id safety_data_sheet_url item_volume item_volume_unit_of_measure ghs_classification_subcategory1 ghs_classification_subcategory2 ghs_classification_subcategory3 legal_compliance_certification_expiration_date legal_compliance_certification_regulatory_organization_name legal_compliance_certification_status cpsia_cautionary_statement1 cpsia_cautionary_statement2 cpsia_cautionary_statement3 cpsia_cautionary_statement4 flash_point legal_compliance_certification_date_of_issue cpsia_cautionary_description legal_compliance_certification_metadata legal_compliance_certification_value ghs_classification_class1 ghs_classification_class2 ghs_classification_class3 california_proposition_65_compliance_type california_proposition_65_chemical_names1 california_proposition_65_chemical_names2 california_proposition_65_chemical_names3 california_proposition_65_chemical_names4 california_proposition_65_chemical_names5 pesticide_marking_type1 pesticide_marking_type2 pesticide_marking_type3 pesticide_marking_registration_status1 pesticide_marking_registration_status2 pesticide_marking_registration_status3 pesticide_marking_certification_number1 pesticide_marking_certification_number2 pesticide_marking_certification_number3 currency signed_by restock_date standard_plus fulfillment_latency offering_can_be_giftwrapped dust_jacket_description max_order_quantity merchant_shipping_group_name delivery_schedule_group_id product_tax_code offering_can_be_gift_messaged condition_type condition_note expedited_shipping will_ship_internationally sale_price sale_from_date sale_end_date map_price offering_end_date offering_start_date bookloader 73298 0899091156 ISBN THE NEW YEAR'S OWL Encounters with Animals, People and the Land They Share Yankee Books Shetterly, Susan Hand hardcover 1986 English 26.00 1 Update https://img.btimages.net/tap/73298.jpg 0.90 0 FALSE FALSE LB 0 Other 3 very good Migrated Template Collectible - Very Good Signed by Author & Illustrator; Yankee Books; 1986; 1st Edition; 8vo 8" - 9" tall; Hardcover; Very Good in Very Good- dust jacket; Signed on title page by author and illustrator. Gift inscription on endpaper. Small edge tear to the DJ, top front.; 149 Pages; Illustrated by Robert Shetterly, Jr.

TemplateType=bookloader Version=2022.0922 Category=bookloader TemplateSignature=Qk9PS1NfMTk3M19BTkRfTEFURVI= settings=attributeRow=3&contentLanguageTag=en_US&dataRow=4&feedType=113&headerLanguageTag=en_US&isProcessingSummary=false&labelRow=2&primaryMarketplaceId=amzn1.mp.o.ATVPDKIKX0DER&templateIdentifier=7a8e0736-b13a-406c-b73c-f2debbdb9e43&timestamp=2022-09-23T06% Use ENGLISH to fill this template.The top 3 rows are for Amazon.com use only. Do not modify or delete the top 3 rows. Images Variation Basic Discovery Product Enrichment Fulfillment Compliance Offer
Product Type Seller SKU Product ID Product ID Type Product Name Manufacturer Author Author Author Binding Publication date Language Your Price Quantity Main Image URL Main Offer Image Other Image URL1 Offer Image Other Image URL2 Other Image URL3 Other Image URL4 Other Image URL5 Other Image URL6 Other Image URL7 Other Image URL8 Relationship Type Package Level package_contains_quantity package_contains_identifier brand_name Update Delete Manufacturer Part Number Product Description Edition Number Language Language Language Language Language Category (item-type) Series Title Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features Key Product Features volume pages BISAC Subject Code Format Format Format Format Format Search Terms Subject Color Map BISAC Subject Code Subject Keyword Color Foreword Adapter Afterword Preface reader Series Editor Colorist Series Number Drawings Photographer Compiler Artist Translator Editor Creator Composer Painter Contributor Introduction Series Volume Illustrator legal_compliance_certification_certifying_authority_name legal_compliance_certification_geographic_jurisdiction Offer Image1 Offer Image2 Offer Image3 Offer Image4 Offer Image5 Fulfillment Center ID Package Height Package Width Package Length Package Dimensions Unit Of Measure Package Weight Package Weight Unit Of Measure Item Weight Lithium content (grams) Lithium Battery Packaging Is this product a battery or does it utilize batteries? Batteries are Included Battery type/size Battery type/size Battery type/size Number of Lithium-ion Cells Number of Lithium Metal Cells Watt hours per battery item_weight_unit_of_measure Number of batteries Number of batteries Number of batteries Fabric Type Fabric Type Fabric Type Import Designation Battery composition Battery weight (grams) battery_weight_unit_of_measure lithium_battery_energy_content_unit_of_measure lithium_battery_weight_unit_of_measure Applicable Dangerous Goods Regulations Applicable Dangerous Goods Regulations Applicable Dangerous Goods Regulations Applicable Dangerous Goods Regulations Applicable Dangerous Goods Regulations UN number Safety Data Sheet (SDS) URL Volume item_volume_unit_of_measure GHS Classification Subcategory GHS Classification Subcategory GHS Classification Subcategory legal_compliance_certification_expiration_date Regulatory Organization Name Compliance Certification Status Cpsia Warning Cpsia Warning Cpsia Warning Cpsia Warning Flash point (°C)? legal_compliance_certification_date_of_issue CPSIA Warning Description legal_compliance_certification_metadata Legal Compliance Certification Categorization/GHS pictograms (select all that apply) Categorization/GHS pictograms (select all that apply) Categorization/GHS pictograms (select all that apply) California Proposition 65 Warning Type California Proposition 65 Chemical Names California Proposition 65 Chemical Names California Proposition 65 Chemical Names California Proposition 65 Chemical Names California Proposition 65 Chemical Names Pesticide Marking Pesticide Marking Pesticide Marking Pesticide Registration Status Pesticide Registration Status Pesticide Registration Status Pesticide Certification Number Pesticide Certification Number Pesticide Certification Number Currency Signed By Restock Date Standard Plus Handling Time Is Gift Wrap Available Dust Jacket Condition Max Order Quantity Shipping-Template Scheduled Delivery SKU List Product Tax Code Offering Can Be Gift Messaged Item Condition Offer Condition Note Expedited Shipping Will Ship Internationally Sale Price Sale Start Date Sale End Date Minimum Advertised Price Offer End Date Offer Start Date offer_image1 offer_image2 feed_product_type item_sku external_product_id external_product_id_type item_name manufacturer author1 author2 author3 binding publication_date language_published standard_price quantity main_image_url main_offer_image other_image_url1 offer_image other_image_url2 other_image_url3 other_image_url4 other_image_url5 other_image_url6 other_image_url7 other_image_url8 relationship_type package_level package_contains_quantity package_contains_identifier brand_name update_delete part_number product_description edition language_value1 language_value2 language_value3 language_value4 language_value5 item_type collection_name bullet_point1 bullet_point2 bullet_point3 bullet_point4 bullet_point5 bullet_point6 bullet_point7 bullet_point8 bullet_point9 bullet_point10 volume_number pages thema_classification_code format1 format2 format3 format4 format5 generic_keywords unknown_subject color_map bisac_subject_description subject_keywords color_name foreword adapter afterword preface reader series_editor colorist series_number drawings photographer compiler artist translator editor creator composer painter contributor introduction volume_base illustrator legal_compliance_certification_certifying_authority_name legal_compliance_certification_geographic_jurisdiction offer_image3 offer_image4 offer_image5 fulfillment_center_id package_height package_width package_length package_dimensions_unit_of_measure package_weight package_weight_unit_of_measure item_weight lithium_battery_weight lithium_battery_packaging batteries_required are_batteries_included battery_type1 battery_type2 battery_type3 number_of_lithium_ion_cells number_of_lithium_metal_cells lithium_battery_energy_content item_weight_unit_of_measure number_of_batteries1 number_of_batteries2 number_of_batteries3 fabric_type1 fabric_type2 fabric_type3 import_designation battery_cell_composition battery_weight battery_weight_unit_of_measure lithium_battery_energy_content_unit_of_measure lithium_battery_weight_unit_of_measure supplier_declared_dg_hz_regulation1 supplier_declared_dg_hz_regulation2 supplier_declared_dg_hz_regulation3 supplier_declared_dg_hz_regulation4 supplier_declared_dg_hz_regulation5 hazmat_united_nations_regulatory_id safety_data_sheet_url item_volume item_volume_unit_of_measure ghs_classification_subcategory1 ghs_classification_subcategory2 ghs_classification_subcategory3 legal_compliance_certification_expiration_date legal_compliance_certification_regulatory_organization_name legal_compliance_certification_status cpsia_cautionary_statement1 cpsia_cautionary_statement2 cpsia_cautionary_statement3 cpsia_cautionary_statement4 flash_point legal_compliance_certification_date_of_issue cpsia_cautionary_description legal_compliance_certification_metadata legal_compliance_certification_value ghs_classification_class1 ghs_classification_class2 ghs_classification_class3 california_proposition_65_compliance_type california_proposition_65_chemical_names1 california_proposition_65_chemical_names2 california_proposition_65_chemical_names3 california_proposition_65_chemical_names4 california_proposition_65_chemical_names5 pesticide_marking_type1 pesticide_marking_type2 pesticide_marking_type3 pesticide_marking_registration_status1 pesticide_marking_registration_status2 pesticide_marking_registration_status3 pesticide_marking_certification_number1 pesticide_marking_certification_number2 pesticide_marking_certification_number3 currency signed_by restock_date standard_plus fulfillment_latency offering_can_be_giftwrapped dust_jacket_description max_order_quantity merchant_shipping_group_name delivery_schedule_group_id product_tax_code offering_can_be_gift_messaged condition_type condition_note expedited_shipping will_ship_internationally sale_price sale_from_date sale_end_date map_price offering_end_date offering_start_date bookloader 73298 0899091156 ISBN THE NEW YEAR'S OWL Encounters with Animals, People and the Land They Share Yankee Books Shetterly, Susan Hand hardcover 1986 English 26.00 1 Update https://img.btimages.net/tap/73298.jpg https://img.btimages.net/tap/73298btp.jpg 0.90 0 FALSE FALSE LB 0 Other 3 very good Migrated Template Collectible - Very Good Signed by Author & Illustrator; Yankee Books; 1986; 1st Edition; 8vo 8" - 9" tall; Hardcover; Very Good in Very Good- dust jacket; Signed on title page by author and illustrator. Gift inscription on endpaper. Small edge tear to the DJ, top front.; 149 Pages; Illustrated by Robert Shetterly, Jr.

JoshClose commented 2 years ago

When doing it yourself it's not being sorted by the indexes. Here is an example where it's sorting them also.

void Main()
{
    var records = new List<Foo>
    {
        new Foo { column_one = "one", column_two = "two" },
    };
    var config = new CsvConfiguration(CultureInfo.InvariantCulture)
    {
    };
    using (var writer = new StringWriter())
    using (var csv = new CsvWriter(writer, config))
    {
        csv.WriteHeader<Foo>();
        csv.NextRecord();

        var names =
            from property in typeof(Foo).GetProperties()
            let index = property.GetCustomAttribute<IndexAttribute>()
            orderby index.Index ascending
            select property.Name;

        foreach (var name in names)
        {
            csv.WriteField(name);
        }
        csv.NextRecord();

        csv.WriteRecords(records);
        writer.Dump();
    }
}

private class Foo
{
    [Index(1)]
    [Name("Column 2")]
    public string column_two { get; set; }

    [Index(0)]
    [Name("Column 1")]
    public string column_one { get; set; }
}

image