Cinchoo / ChoETL

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
MIT License
797 stars 134 forks source link

Merging nested arrays to csv when arrays are empty #304

Open praveena1728 opened 11 months ago

praveena1728 commented 11 months ago

I have data in json similar to the structure below:

[
{
        "incident": {
            "id": "120950002",
            "name": "Mosquito Fire"
        },
        "additionalClaimant": [
            {
                "claimantFirstName": "Olga",
                "claimantLastName": "Yachnik",

            },
           {
                "claimantFirstName": "Meera",
                "claimantLastName": "Lavina",

            },
        ]
},
{
        "incident": {
            "id": "120950002",
            "name": "Mosquito Fire"
        },
        "additionalClaimant": [],
},
]

The json contains an array called "AdditionalClaimant" which sometimes has value but also can be an empty array. I need to convert this to csv using ChoETL where the claimantFirstName values in the array should merge to one column ("Olga,Meera"). and this is my code

    //Use ChoETL to convert data to CSV
    using (var r = ChoJSONReader.LoadText(attorneyClaimData)
        .WithField("Fire", jsonPath: "$.incident.name", isArray: false)
        .WithField("FirstName", jsonPath: "$.additionalClaimant[*].claimantFirstName", defaultValue: "null")
    )
    {
        using (var w = new ChoCSVWriter("AttorneyClaims.csv")
            .WithFirstLineHeader()
            .UseNestedKeyFormat(false)
            .Configure(c => c.ArrayValueSeparator = ';')
            )
        {
            w.Write(r);
        }
    }

The problem arises when the AdditionalClaimant array is empty. I hit the error as below "message": "Failed to write 'System.Object[]' value for 'FirstName' field." Please can you help around how to navigate around this?

Cinchoo commented 10 months ago

Here is how you can convert your sample json to csv

        static void Issue304()
        {
            string json = @"[
  {
    ""incident"": {
      ""id"": ""120950002"",
      ""name"": ""Mosquito Fire""
    },
    ""additionalClaimant"": [
      {
        ""claimantFirstName"": ""Olga"",
        ""claimantLastName"": ""Yachnik""

      },
      {
        ""claimantFirstName"": ""Meera"",
        ""claimantLastName"": ""Lavina""

      }
    ]
  },
  {
    ""incident"": {
      ""id"": ""120950002"",
      ""name"": ""Mosquito Fire""
    },
    ""additionalClaimant"": []
  }
]";

            using (var r = ChoJSONReader.LoadText(json)
                .Configure(c => c.DefaultArrayHandling = false)
                .Configure(c => c.FlattenNode = true)
                .Configure(c => c.UseNestedKeyFormat = true)
                .Configure(c => c.FlattenByNodeName = "additionalClaimant")
                .Configure(c => c.NestedColumnSeparator = '.')
                )
            {
                using (var w = new ChoCSVWriter(Console.Out)
                    .WithFirstLineHeader()
                    )
                {
                    w.Write(r);
                }

            }
        }