box / boxcli

A command line interface for interacting with the Box API.
https://developer.box.com
Apache License 2.0
222 stars 59 forks source link

Support UTF-8 (BOM) CSV as the input file for `--bulk-file-path` option #491

Closed kojimaru14 closed 1 year ago

kojimaru14 commented 1 year ago

When I use a UTF-8 (BOM) CSV file as the file for --bulk-file-path option, the command fails with the below error.

For e.g.

C:\Users\username\Downloads>box folders:create --bulk-file-path=utf8_with_bom.csv
[========================================] 100% | 1/1

1 entry failed!
----------
Entry 1 (
    description=null
    parentID=214821588160
) failed with error:
Missing 1 required arg:
name  Name of new folder
See more help with --help

Removing the BOM helps resolve the above error.

Could we add a support for UTF-8 (BOM) CSV?

kojimaru14 commented 1 year ago

Based on my quick check, it seems like we need to add bom: true as the parameter to csvParse... like this?

let parsedData = await csvParse(fileContents, {
    delimiter: ',',
    bom: true,   // To automatically detect BOM
    cast(value, context) {
        if (value.length === 0) {
            // Regard unquoted empty values as null
            return context.quoting ? '' : null;
        }
        return value;
    },
);

https://github.com/box/boxcli/blob/ea6605b39396a75e398cb027d873e13cf14ca53a/src/box-command.js#L496-L505

According to https://csv.js.org/parse/options/bom/, by default, bom: false. It seems like we need to add bom: true to automatically detect BOM?

mhagmajer commented 1 year ago

Thanks @kojimaru14 for the valid pointer. I've created a PR to update the csv library version and turn on the bom setting to automatically detect BOM.