Open jlaura opened 4 years ago
@jlaura not a naive question at all. A while back this was an express API (easier to run locally) but that's deprecated now that we've moved to API Gateway. Because we aren't using SAM or serverless, there aren't great options to run this locally out of the box. Likely the "easiest" (but not that easy) way to run the API locally is by following the deploy instructions here until the deploy step and then running kes compile
instead of kes deploy
. This will give you a cloudformation template which you can use with localstack. There's a chance @matthewhanson is working on updated deployment steps which simplify this a bit as well
Hi @jlaura , you might want to check out the new STAC-API: https://github.com/stac-utils/stac-api It's a completely refactored version of sat-api, and is compliant with the latest version of STAC, 0.9.0. I have not deployed it locally, but it uses serverless so should be easier to set up.
@matthewhanson Thanks for the link. I have been able to make some more progress getting this running locally.
I wanted to post some info in case it is of use for anyone else trying to set this up locally.
Minio(S3) / Apache are running to store / serve the STAC compliant metadata, COGs, and thumbnail jpegs using the following docker-compose. Note that I am also using traefik locally, but this can be removed / omitted.
``` version: '3' services: apache: image: 'bitnami/apache:latest' ports: - '9080:8080' - '9443:8443' volumes: - ../single_image/:/app labels: - "traefik.enable=true" - "traefik.frontend.priority=600" - "traefik.frontend.rule=Method:GET" - "traefik.frontend.rule=PathPrefixStrip:/stac" - "traefik.backend=apache" - "traefik.docker.network=traefik-public" networks: - default - traefik-public minio: image: 'minio/minio:latest' ports: - 9000:9000 volumes: - ./minio_data/:/data labels: - "traefik.enable=true" - "traefik.frontend.priority=600" - "traefik.frontend.rule=Method:GET" - "traefik.frontend.rule=PathPrefixStrip:/minio" - "traefik.backend=minio" - "traefik.docker.network=traefik-public" networks: - default - traefik-public command: server /data environment: MINIO_ACCESS_KEY: minio MINIO_SECRET_KEY: minio123 networks: traefik-public: external : true ```
We then need an elasticsearch backend to get the metadata pushed into. I am running this in yet another docker-compose:
``` 1,1 All version: '3' services: elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:6.8.7 environment: - cluster.name=docker-cluster - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 ports: - "9300:9200" kibana: image: docker.elastic.co/kibana/kibana:6.3.2 ports: - "9601:5601" ```
I am currently working on getting the ingest and api lambdas exposed as well in this setup. Luckily, lambci offers containers that I think will work.
``` version: '3' services: lambda_ingest: image: lambci/lambda:nodejs12.x environment: DOCKER_LAMBDA_STAY_OPEN : 1 ES_HOST : http://smalls:9300/ ports: - "9001:9001" networks: - default volumes: - ./lambdas/ingest/dist:/var/task:ro,delegated - ./libs:/opt command: index.handler lambda_api: image: lambci/lambda:nodejs12.x environment: DOCKER_LAMBDA_STAY_OPEN : 1 ES_HOST : http://smalls:9300/ SATAPI_URL: http://smalls:9002/ ports: - "9002:9001" networks: - default volumes: - ./lambdas/api/dist:/var/task:ro,delegated command: index.handler ```
Using curl, I can do an ingest without a problem: curl -d '{"url":"http://127.0.0.1/stac:9000/themis/test/collection.json"}' http://127.0.0.1:9001/2015-03-31/functions/myfunction/invocations
and can see in the ES logs that things are being written. It is also possible to see the return on the call to the api
function: curl http://127.0.0.1:9002/2015-03-31/functions/myfunction/invocations
.
Next step is to test sat-search. I am running into a few issues here that I wonder if others might have insight:
Let me know if this info is helpful or if you all prefer users to focus on doing an AWS deploy.
@drewbo @jlaura A bit late to the party but I am putting the finishing touches on a SAT-API stack running on a local openfaas /kubernetes cluster with a local ES/KB deployment: sat-api-openfaas. My notes aren't finished yet but it does appear to work per spec with the Sentinel-5P dataset hosted on AWS. Please let me know if you have any questions -- thanks!
Apologies for the likely naive question. I have a local, test STAC-API compliant instance running on in a docker cluster (minio for the holding STAC compliant JSON files, Apache, etc.) and then have STAC-Browser serving these files out in the nice web instance.
I am interested in testing sat-api without pushing to AWS. So I have also set up an ES cluster and pushed the stac metadata into that. How do I go about deploying the sat-api locally?
My current attempts to use a simple docker-compose file with the API are failing and I am just guessing at the environment variables and the ports that I need to populate/expose.