How to: system, cli, bash

K0nkere commented 1 year ago

Выполнить команду из питона

os.system("wget ... -O ... -P")

K0nkere commented 1 year ago

Посмотреть что запущено на порту 9696

sudo lsof -t -i:9696

Убить процесс, запущенный на порту 9696

sudo kill $(sudo lsof -t -i:9696)

K0nkere commented 1 year ago

Скрипт загрузки данных в цикле с архивацией

set -e # stop script on zero code 
URL_PREFIX=https://github.com/DataTalksClub/nyc-tlc-data/releases/download
# setting taxi_type and year as arguments on script launch
TAXI_TYPE=$1 #"green"
YEAR=$2 #2020

for MONTH in {1..12}; do
  FMONTH=$(printf "%02d" ${MONTH})
  FULL_URL="${URL_PREFIX}/${TAXI_TYPE}/${TAXI_TYPE}_tripdata_${YEAR}-${FMONTH}.csv.gz"

  LOCAL_PREFIX="data/raw/${TAXI_TYPE}/${YEAR}/${FMONTH}"
  LOCAL_FILENAME="${TAXI_TYPE}_tripdata_${YEAR}-${FMONTH}.csv.gz"
  LOCAL_PATH="${LOCAL_PREFIX}/${LOCAL_FILENAME}"

  echo "Downloading to ${LOCAL_PATH}"
  mkdir -p ${LOCAL_PREFIX} # -p tag is to create subfolders from parent folder
  wget ${FULL_URL} -O ${LOCAL_PATH}

  # echo "compressing ${LOCAL_PATH}" # compressing with gzip if original was clear .csv
  # gzip ${LOCAL_PATH}
done

Посмотреть несколько первых строк в архиве

zcat <file_path> | head -n 10

K0nkere / DL_Dice-detection-project

How to: system, cli, bash #10