The next generation repository system for DLSS
Have questions? Check out the FAQ.
Some agreements laid out by the development team are listed in the Development Practices document.
Install go (grab binary from here or use brew install go
on Mac OSX).
Setup your Go workspace (where your Go code, binaries, etc. are kept together. See some helpful documentation here on the Go concept of workspaces: https://github.com/alco/gostart#1-go-tool-is-only-compatible-with-code-that-resides-in-a-workspace.):
$ mkdir -p ~/go
$ export GOPATH=~/go
$ export PATH=~/go/bin:$PATH
$ cd ~/go
Your Go code repositories will reside within ~/go/src/...
in the $GOPATH
. Name these paths to avoid library clash, for example TACO Go code could be in ~/go/src/github.com/sul-dlss-labs/taco
. This should be where your Github repository resides too.
In order to download the project code to ~/go/src/github.com/sul-dlss-labs/taco
, from any directory in your `$GOPATH
, run:
$ go get github.com/sul-dlss-labs/taco
Handle Go project dependencies with the Go dep
package:
brew install dep
then brew upgrade dep
(if on Mac OSX).Gopkg.toml
and Gopkg.lock
have not yet been populated, you need to add an inferred list of your dependencies by running dep init
.dep ensure
.dep ensure -add github.com/pkg/errors
. This should add the dependency and put the new dependency in your Gopkg.*
files.Setup Configuration Variables
HONEYBADGER_API_KEY=aaaaaaaa
$ AWS_REGION=localstack AWS_ACCESS_KEY_ID=999999 AWS_SECRET_ACCESS_KEY=1231 go run main.go
Note: we explain the AWS keys usage below.
Start external services using Localstack then run:
$ docker build -t taco .
$ docker run -e AWS_REGION=localstack -e AWS_ACCESS_KEY_ID=999999 -e AWS_SECRET_ACCESS_KEY=1231 -p 8080:8080 taco
This will create a binary in your path that you can then run the application with.
$ go build -o tacod main.go
$ ./tacod
Now start the API server (passing in API keys; these can be fake and are only required for localstack):
$ AWS_REGION=localstack AWS_ACCESS_KEY_ID=999999 AWS_SECRET_ACCESS_KEY=1231 ./tacod
Then you can interact with it using curl
:
$ curl -H "Content-Type: application/json" \
-H "On-Behalf-Of: lmcrae@stanford.edu" \
-d@examples/request.json http://localhost:8080/v1/resource
it will return a response like:
{"id":"fe1f66a9-5285-4b28-8240-0482c8fff6c7"}
Then you can use the returned identifier to retrieve the original:
$ curl -H "Content-Type: application/json" \
-H "On-Behalf-Of: lmcrae@stanford.edu" \
http://localhost:8080/v1/resource/fe1f66a9-5285-4b28-8240-0482c8fff6c7
You can then update the resource with a PATCH request:
Note: in the example file, the externalIdentifier
and identification.identifier
must be set to the value returned from the POST, and
the tacoIdentifier
must be set to the value retrieved in the GET
Without changing the version number:
$ curl -X PATCH -H "Content-Type: application/json" \
-d@examples/update_request.json \
-H "On-Behalf-Of: lmcrae@stanford.edu" \
http://localhost:8080/v1/resource/fe1f66a9-5285-4b28-8240-0482c8fff6c7
With changing the version number:
$ curl -X PATCH -H "Content-Type: application/json" \
-d@examples/update_request_new_version.json \
-H "On-Behalf-Of: lmcrae@stanford.edu" \
http://localhost:8080/v1/resource/fe1f66a9-5285-4b28-8240-0482c8fff6c7
Now, we're going to get a file we can use to test the uploader:
$ curl -Lo test.pdf \
https://github.com/samvera/hyrax/raw/master/spec/fixtures/hyrax/hyrax_test4.pdf
Than you can upload that file into TACO by doing:
$ curl -H "On-Behalf-Of: lmcrae@stanford.edu" \
-F "upload=@test.pdf;type=application/pdf" http://localhost:8080/v1/file
{"id":"ef8e96fc-b29a-452e-b311-ca695256e7ca"}
Then you can retrieve the file by accessing it at that identifier:
$ curl -H "On-Behalf-Of: lmcrae@stanford.edu" \
-L http://localhost:8080/v1/file/ef8e96fc-b29a-452e-b311-ca695256e7ca
You can remove it by deleting the metadata resource:
$ curl -X DELETE -H "On-Behalf-Of: lmcrae@stanford.edu" \
http://localhost:8080/v1/resource/ef8e96fc-b29a-452e-b311-ca695256e7ca
Use the following steps to build and run taco with its dependencies. The TACO service will be available at http://\[::\]:8080
$ docker-compose build
$ docker-compose up -d
$ docker-compose run resources
$ docker-compose run taco
To run the integration and end to end tests on the containers run:
$ docker-compose build
$ docker-compose up -d
$ docker-compose run resources
$ docker-compose run tester
Docker Compose can also be used to spin up just the localstack services and taco in separate terminal windows to be used together. To do so, open a tab in your terminal and run:
$ docker-compose build
$ docker-compose up -d
$ docker-compose run resources
$ docker-compose run localstack
Then, in a separate window, start the TACO container. The TACO service will be available at http://\[::\]:8080
$ docker-compose run taco
Logs for any service can be tail'd or followed using the docker-compose logs
command:
$ docker-compose logs -f taco
To stop and remove containers, networks, images, and volumes created by docker-compose up
run:
$ docker-compose down
Testing and test driven development is outlined in the Testing Guidelines & Framework documentation.
We use go-swagger
to generate the API code within the generated/
directory.
We connect the specification-generated API code to our own handlers defined with handlers/
. Handlers are where we add our own logic for processing requests.
Our handlers and the generated API code is connected within main.go
, which is the file to start the API.
The API code is generated from swagger.yml
using go-swagger
library. As this is not used in the existing codebase anywhere currently, you'll need to install the go-swagger
library before running these commands (commands for those using Mac OSX):
$ brew tap go-swagger/go-swagger
$ brew install go-swagger
$ brew upgrade go-swagger
This should give you the swagger
binary command in your $GOPATH and allow you to manage versions better (TBD write this up). The version of your go-swagger binary is 0.13.0 (run swagger version
to check this).
If instead of brew, you decide to install go-swagger from source (i.e. go get -u github.com/go-swagger/go-swagger/cmd/swagger
), you will be running the library at its Github dev
branch. You will need to change into that library (cd $GOPATH/src/github.com/go-swagger/go-swagger
) and checkout from Github the latest release (git checkout tags/0.13.0
). Then you will need to run go install github.com/go-swagger/go-swagger/cmd/swagger
to generate the go-swagger binary in your $GOPATH/bin
.
There appears to be no best way to handle specification-based re-generation of the generated/
API code, so the following steps are recommended:
$ git rm -rf generated/
$ mkdir generated
$ swagger generate server -t generated --exclude-main --principal authorization.Agent
go generate
This will serialize all the json in the maps/
directory into validators/maps.go
Additionally it grabs the defaults from maps/DepositFile.json
and
creates a template for File resources in handlers/file_template.go
.
Anything outside of generated/
is our own code and should not be touched by a regeneration of the API code from the Swagger specification.
To see the SWAGGER generated documentation, go to https://sul-dlss-labs.github.io/taco/. This is automatically generated off of the Swagger spec in this repository on the master
branch.
If you want to generate this documentation locally, you can run the following:
$ swagger serve swagger.yml
This should prompt you to your web browser for the HTML generated docs.