manojkarthick / pqrs

Command line tool for inspecting Parquet files
Apache License 2.0
296 stars 29 forks source link

build: bump deps clap/parquet/arrow #41

Closed SteveLauC closed 1 year ago

SteveLauC commented 1 year ago

What are included in this PR

  1. Bump parquet/arrow to 38.0.0
  2. Bump clap to 4.2.7
  3. In parquet, ArrowReader and ParquetFileArrowReader are deprecated and replaced with ArrowReaderBuilder and ParquetRecordBatchReaderBuilder
  4. In clap, multiple_values = true is replaced with num_args = 1..
  5. Use #[arg(short, long)] instead of #[clap(short, long)]
  6. The behavior of record_batch_to_json_row() has been changed and thus our integration test for pqrs cat --json needs to be changed too(CAT_JSON_OUTPUT needs to be updated)

    # old output
    $ pqrs cat data/cities.parquet --json
    
    #########################
    File: data/cities.parquet
    #########################
    
    {"continent":"Europe","country":{"name":"France","city":["Paris","Nice","Marseilles","Cannes"]}}
    {"continent":"Europe","country":{"name":"Greece","city":["Athens","Piraeus","Hania","Heraklion","Rethymnon","Fira"]}}
    {"continent":"North America","country":{"name":"Canada","city":["Toronto","Vancouver","St. John's","Saint John","Montreal","Halifax","Winnipeg","Calgary","Saskatoon","Ottawa","Yellowknife"]}}
    
    # new output
    $ ./target/debug/pqrs cat data/cities.parquet --json
    
    #########################
    File: data/cities.parquet
    #########################
    
    {"continent":"Europe","country":{"city":["Paris","Nice","Marseilles","Cannes"],"name":"France"}}
    {"continent":"Europe","country":{"city":["Athens","Piraeus","Hania","Heraklion","Rethymnon","Fira"],"name":"Greece"}}
    {"continent":"North America","country":{"city":["Toronto","Vancouver","St. John's","Saint John","Montreal","Halifax","Winnipeg","Calgary","Saskatoon","Ottawa","Yellowknife"],"name":"Canada"}}
    
  7. code format with cargo fmt
SteveLauC commented 1 year ago

Since this is the successor of #32, let's close it. Closes #32

manojkarthick commented 1 year ago

This looks great! Thanks a lot making these updates @SteveLauC - really appreciate it!