Add Redesigned Action: Read

[jhorbulyk]

Config Fields

• Emit Behavior (Dropdown with Fetch All & Emit Individually, required): Indicates if rows should be emitted in a single message with an array or one at a time.

Input Metadata

• Attachment URL (string, required): A URL to an attachment which should be a CSV file
• The following CSV parse options:
  • First row contains headers (boolean, required): Corresponds to the header option in papaparse
  • Delimiter (string, optional, default to autodetect):

Output Metadata

• When Emit Behavior is Fetch All, the output should be an object with a single property results which is an array where each item in the array is an object representing a row in the CSV file.
• When Emit Behavoir is Emit Individually, each message body should be an object representing a row of the CSV file.

Implementation Notes

• We should deprecate the existing read action in favor of this action.
• It is suggested that we migrate to papaparse when building this new action.

[A3a3e1]

1. 1st step - Investigate Read and Write actions rework. 1.1. Check whether https://www.papaparse.com/ lib satisfies the requirements

[if0s]

Hi @jhorbulyk, during development new Read action there were several questions:

1. By default, after parsing all value is strings, we can use dynamicTyping to try convert them to appropriate data type, but functionality of this method is not really big, for example numbers with space "1 234.5" or comma "1,234" will not be set to number type, only with dot - "1.234" or simple "1234", what we should do:

   • stay all values as strings
   • enable dynamicTyping to let papaparse decide which kind of data to use
   • add dynamicTyping to Config Fields
   • something else...

2. If header set to false this library create array of string instead of objects, something like this, ['asd', 'fgh', 'jkl']

   • should we convert it to object? It may be like this:

     {
     "field1": "asd",
     "field2": "fgh",
     "field3": "jkl"
     }

   • assing all data to one object:

     {
     "data": [
     "asd",
     "fgh",
     "jkl"
     ]
     }

   • set header as always true
   • something else...

[jhorbulyk]

1. I guess dynamicTyping can be added as another CSV parse mechanism in the input metadata. If people need more complex parsing than what is built-in to papaparse, they can write there own string parsing expression with JSONata.
2. I would convert the result to an object where the keys are column0, column1, column2, etc. (Either 0-based or 1-based column counting)