program "wget -qO- https://bit.ly/2ghU6eZ" failed

[JimiHFord]

I am trying to import zip code data into a table by following this github gist.

I am using https://github.com/go-pg/migrations inside of a go application - here is the migration file: 4_add_zip_codes_table.go

package migrate

import (
    "fmt"

    "github.com/go-pg/migrations"
)

// Inspired by https://gist.github.com/pramsey/85b5791f43cfae33ea59d67ff7c67c6b

const postgisExtension = `
CREATE EXTENSION IF NOT EXISTS postgis
`

const zipcodeTable = `
CREATE TABLE zcta (
    zip TEXT,
    longitude REAL,
    latitude REAL,
    geom GEOMETRY(Point, 4326)
)`

// Read data direct from URL using curl
const populateZipcodes = `
COPY zcta (zip, latitude, longitude) 
FROM PROGRAM 'wget -qO- https://bit.ly/2ghU6eZ'
WITH (
  FORMAT csv,
  HEADER true
)`

// Add geometry values to geometry column
const populateGeom = `
UPDATE zcta 
  SET geom = ST_SetSRID(ST_MakePoint(longitude, latitude), 4326)
`

// Index for zip code
const zipcodeIndex = `
CREATE INDEX zcta_zip_idx 
  ON zcta (zip)
`

// Cluster to re-write the table after update
const cluster = `
CLUSTER zcta USING zcta_zip_idx
`

// Spatial index for geometry
const spatialIndex = `
CREATE INDEX zcta_geom_idx 
  ON zcta USING GIST (geom)
`

func init() {
    up := []string{
        postgisExtension,
        zipcodeTable,
        populateZipcodes,
        populateGeom,
        zipcodeIndex,
        cluster,
        spatialIndex,
    }

    down := []string{
        `DROP TABLE zcta`,
    }

    migrations.Register(func(db migrations.DB) error {
        fmt.Println("create zip code table")
        for _, q := range up {
            _, err := db.Exec(q)
            if err != nil {
                return err
            }
        }
        return nil
    }, func(db migrations.DB) error {
        fmt.Println("drop zip code table")
        for _, q := range down {
            _, err := db.Exec(q)
            if err != nil {
                return err
            }
        }
        return nil
    })
}

Notice, the original gist suggests using curl, but I am pretty sure curl is not installed in the postgis/postgis image (nor is it installed in the postgres image). So I changed this line to use wget instead since it appears as though this postgis/postgis image should have wget installed already.

When I run go run main.go migrate up, I get the following output:

Using config file: /home/jimi/.some-api.yaml
create zip code table
2021/07/03 22:29:36 ERROR #38000 program "wget -qO- https://bit.ly/2ghU6eZ" failed
exit status 1

Does anyone know what I might be doing wrong when it comes to the

COPY zcta (zip, latitude, longitude) 
FROM PROGRAM 'wget -qO- https://bit.ly/2ghU6eZ'
WITH (
  FORMAT csv,
  HEADER true
)

part?

[ImreSamu]

"wget -qO- https://bit.ly/2ghU6eZ" failed

IMHO: the wget command is not installed in the server part ( server docker image )

$ docker run -it --rm postgis/postgis:13-3.1 bash -c 'wget -qO- https://bit.ly/2ghU6eZ'
bash: wget: command not found

• you can extend the debian images with the wget command.
• or you can use other methods to importing csv files.
• or you can use the postgis/postgis:13-3.1-alpine image ( wget included )

example:

 docker run -it --rm postgis/postgis:13-3.1-alpine bash -c 'wget -qO- https://bit.ly/2ghU6eZ | head'
ZIP,LAT,LNG
00601,18.180555, -66.749961
00602,18.361945, -67.175597
00603,18.455183, -67.119887
00606,18.158345, -66.932911
00610,18.295366, -67.125135
00612,18.402253, -66.711397
00616,18.420412, -66.671979
00617,18.445147, -66.559696
00622,17.991245, -67.153993

[ImreSamu]

if you want to extend the debian image

• see example: https://github.com/postgis/docker-postgis/tree/master/examples/image-with-postgis-clis , just add the curl, wget, ...

[JimiHFord]

Ah! You're a genius :tada: thank you!!

I think I was looking in the alpine section when I saw wget being used. That would explain it. Thanks so much!