mediacloud / story-indexer

The core pipeline used to ingest online news stories in the Media Cloud archive.
https://mediacloud.org
Apache License 2.0
2 stars 5 forks source link

Refactor Elastic conf to not require S3 credentials for development deployment #290

Closed thepsalmist closed 4 months ago

thepsalmist commented 6 months ago

Currently, the elastic-conf service throws an error when any of the args is missing, this includes ~ELASTICSEARCH_SNAPSHOT_REPO, which is an S3 repository. As mentioned by @philbudne it isn't a necessity to have S3 credentials for a dev deployment. To do; refactor the elastic-conf to capture the above Question: Should we have a dev.sh file in private config repo to maintain any dev deployment secrets?

kilemensi commented 6 months ago

If we want to continue running/testing SLM in dev, using a filesystem repository rather than S3 is probably the better approach; should eliminate the need of S3 credentials in dev.

philbudne commented 6 months ago

Question: Should we have a dev.sh file in private config repo to maintain any dev deployment secrets?

My preference: no. People should be able to test/develop code without access to any secrets.

thepsalmist commented 4 months ago

Resolved in #295