search

datatogether / sentry

Parallelized web crawler written in Golang
GNU Affero General Public License v3.0
14 stars 6 forks source link

Good README needed #7

Open KrzysztofMadejski opened 7 years ago

KrzysztofMadejski commented 7 years ago
KrzysztofMadejski commented 7 years ago

Installation

Docker installation

docker compose up

Manual installation

  1. Install Go language

  2. Download and build repository

    export GOPATH=$(go env GOPATH)
mkdir -pv $GOPATH
cd $GOPATH

git clone https://github.com/archivers-space/sentry
cd sentry
go install
  3. Configure Postgres server and then set connection URL 
    export POSTGRES_DB_URL=postgres://[USERNAME_HERE]:[PASSWORD_HERE]@localhost:[PORT]/[DB_NAME]
  4. Run sentry 
    $GOPATH/bin/sentry

How to configure S3 buckets?

titaniumbones commented 7 years ago

@KrzysztofMadejski do you want to submit a quick PR now with some of this? We start sprinting in 2 hours and maybe @b5 and I cna jump on the branch you create. This is what we want the sprint to accomplish!!

KrzysztofMadejski commented 7 years ago

Should it be part of the readme or a separate INSTALLATION.md file?

titaniumbones commented 7 years ago

we're putting install instructions in README for now, so start with that and if it gets to long we cna move it out into its own doc. Thank you!!

On 10/27/2017 07:25 AM, Krzysiek Madejski wrote:

Should it be part of the readme or a separate INSTALLATION.md file?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/datatogether/sentry/issues/7#issuecomment-339945760, or mute the thread https://github.com/notifications/unsubscribe-auth/AAWPNGQy5o-jlhHZqzQH4tj2E_c6IvS8ks5swb2sgaJpZM4N1WM3.

KrzysztofMadejski commented 7 years ago

@titaniumbones I'm missing the information how to configure scraper so it actually scrapes anything.

titaniumbones commented 7 years ago

ok cool we're hoping to get to this about 2-3 hours from now.

@b5 let's try to do this today.

dcwalk commented 7 years ago

PR #48 has improved usage instructions