"ls" error when running upload_mimic4_v1_0.sh on terminal

[iamjingxian]

Prerequisites

• [X] Put an X between the brackets on this line if you have done all of the following:
  • Checked the online documentation: https://mimic.mit.edu/
  • Checked that your issue isn't already addressed: https://github.com/MIT-LCP/mimic-code/issues?utf8=%E2%9C%93&q=

Description

Description of the issue, including:

• what you have tried
• references to similar issues
• queries demonstrating your question (if applicable)

Prerequisites

• [X] Put an X between the brackets on this line if you have done all of the following:
  • Checked the online documentation: https://mimic.mit.edu/
  • Checked that your issue isn't already addressed: https://github.com/MIT-LCP/mimic-code/issues?utf8=%E2%9C%93&q=

"ls" error when running upload_mimic4_v1_0.sh on terminal

When running the upload_mimic4_v1_0.sh script that I got from step 6 of https://github.com/MIT-LCP/mimic-iv/tree/master/buildmimic/bigquery

I got the following error "CommandException: "ls" command does not support "file://" URLs. Did you mean to use a gs:// URL?"

I believe I made the required changes indicated in the instructions: (1) the name of the bucket where the compressed files are stored on Cloud Storage - changed to "mornin-vanco" (2) the name of the dataset to create and upload the tables - "mimic" and (3) the path to the JSON file (schema) on your local machine.

See the modified code below

#!/bin/bash
# Initialize parameters
bucket=“mornin-vanco"  # we chose this bucket earlier when uploading data
dataset_prefix=“mimic”
schema_local_folder="/Users/russli/physionet.org/files/mimic-iv-master/buildmimic/bigquery/schemas” 

# Get the list of files in the bucket

for module in core hosp icu;
do

    FILES=$(gsutil ls gs://$bucket/v1.0/$module/*.csv.gz)

    for file in $FILES
    do

    # Extract the table name from the file path (ex: gs://mimic4_v1_0/ADMISSIONS.csv.gz)
    base=${file##*/}            # remove path
    filename=${base%.*}         # remove .gz
    tablename=${filename%.*}    # remove .csv

    # Create table and populate it with data from the bucket
    echo bq load --allow_quoted_newlines --skip_leading_rows=1 --source_format=CSV --replace ${dataset_prefix}_${module}.$tablename gs://$bucket/v1.0/$module/$tablename.csv.gz $schema_local_folder/$module/$tablename.json

    # Check for error
    if [ $? -eq 0 ];then
        echo "OK....$tablename"
    else
        echo "FAIL..$tablename"
    fi

    done
done
exit 0

Screenshot of error in terminal

[iamjingxian]

Hi All, the issue was resolved with the use of the correct quotation mark ", instead of “

no idea was is all it took!