mafintosh / csv-parser

Streaming csv parser inspired by binary-csv that aims to be faster than everyone else
MIT License
1.41k stars 134 forks source link

process exits on apple silicon #208

Open 3commascapital opened 2 years ago

3commascapital commented 2 years ago

Expected Behavior

to parse the csv (tsv in this case)

Actual Behavior

exits process. seems like with code 0

How Do We Reproduce?

download a bitcoin utxo day of data from the bottom of this page: https://gz.blockchair.com/bitcoin/outputs/ and try to unzip it. i am mapping the keys to camel cased, and converting values to useful ones such as numbers, dates, etc (separator is "\t")

specifically, i am using neat-csv to parse,

import * as fs from 'fs'
import BigNumber from 'bignumber.js'

export interface BlockchairOutput {
  blockId: number;
  transactionHash: string;
  index: number;
  time: Date;
  value: BigNumber;
  valueUsd: BigNumber;
  recipient: string;
  type: string;
  scriptHex: string;
  isFromCoinbase: boolean;
  isSpendable: number;
}

  const file = fs.createReadStream(filepath)
    .pipe(
      zlib.createGunzip()
    )
  const data = await neatCsv<BlockchairOutput>(file as unknown as fs.ReadStream, {
    separator: '\t',
    mapHeaders: ({ header }) => _.camelCase(header),
    mapValues: ({ header, value }) => {
      switch(header) {
        case 'blockId':
          return +value
        case 'index':
          return +value
        case 'value':
          return new BigNumber(value)
        case 'time':
          return new Date(value + 'Z')
        case 'valueUsd':
          return new BigNumber(value)
        case 'isFromCoinbase':
          return +value === 1
        default:
          return value
      }
    }
  })