Empty data results in error: " Cannot convert undefined or null to object"

[sharon182]

I'm trying to parse empty data (empty json array) using Node Transform, but in the transoformation phase I'm getting a Cannot convert undefined or null to object error.

Is there any support of parsing empty data and get a CSV file with just the header columns ?

Details :

• json2csv 6.1.3
• Node Transform module
• Node 18

[juanjoDiaz]

Hi @sharon182 ,

Have to tried the latest version (v7.0.3)?

Can you provide a sample code to reproduce the issue?

There are tests specifically about handling empty arrays and I haven't been able to reproduce the error with the latest version.

[sharon182]

Hi, I just updated to the latest version and still have the same issue. Here is a sample code of my stream parser :

Few Notes

1. headersMap is just a map of json fields to a formatted column name

2. flatRespById(records) - is just a method that takes a search response and flat the nested objects by some logic and returns an array of objects that need to be parsed to CSV. The issue is when the flatRespById method returns an empty array and then the error attached to this message is being sent.

   createCSVStreamParser(): Transform<SearchResponse, CSVRecords[]> {
   const headersMap = this.headersMap;
   const formatHeaders = () => (recordKey: string) => headersMap.get(recordKey) || recordKey;
   const opts: ParserOptions = {
     formatters: {
       header: formatHeaders(),
     },
     transforms: [
       (records: SearchResponse) => {
         return this.utility.flatRespById(records);
       },
       (record: any) => {
         Object.keys(record).forEach((key) => {
           if (record[key] === 'null') {
             record[key] = '';
           }
         });
         return record;
       },
     ],
   };
   
   const parser = new Transform(opts, {});
   return parser;
   }

[sharon182]

@juanjoDiaz Ok, so i managed to solve this issue by removing the header formatter and set the headers formatting in side the fields option as follows:

  createCSVStreamParser(): Transform<SearchResponse, CSVRecords[]> {
    const headersMap = this.headersMap;
    const formatHeaders = () => (recordKey: string) => headersMap.get(recordKey) || recordKey;
    const opts: ParserOptions = {
      fields: buildCustomFields(this.headersMap), //buildCustomFields is a custom method that maps object fields to CSV columns 
      transforms: [
        (records: SearchResponse) => {
          return this.utility.flatRespById(records);
        },
        (record: any) => {
          Object.keys(record).forEach((key) => {
            if (record[key] === 'null') {
              record[key] = '';
            }
          });
          return record;
        },
      ],
    };

    const parser = new Transform(opts, {});
    return parser;
  }

[juanjoDiaz]

Good that you found a workaround.

The issue is happening here: https://github.com/juanjoDiaz/json2csv/blob/3f075bd158665e4121964e10e03fd2366e0b5e45/packages/plainjs/src/StreamParser.ts#L187-L190 The first element of your array is null so the stream parser cannot auto-detect the fields from that first element. Even though null is a valid JSON object, I would argue that null values are wrong values.