This tool is a work in progress collaboration between MIT Civic Media and Bocoup to build an open source tool for crowd sourcing image annotation. Our first campaign with the tool is focused on annotating demographics in VGG Faces.
You can checkout that campaign at ajl.ai.
This repo is currently structured as a base open source annotator, and the VGG Faces annotation campaign together.
In the future, we'll be working to publish reusable image annotation tools, along with our data and findings. In the meantime, this is a good starting point if you're looking to run a large scale crowd image annotation campaign.
ansible
- deployment scripts for running the application on Ubuntu 16 serverbackend
- back-end api written in node/express/postgressfrontend
- front-end client written in react/redux/webpackjest
- unit and integration testsprototypes/
- exploratory interaction prototypes
sample-data
- initial sample data from researchersstatic
- initial static data for testingapi.md
- REST API reference documentationThis project exposes a REST API with the following endpoints:
More information available in api.md.
Image Annotator is configured to be deployable on any Ubuntu 16 server. All you need is Ansible 2.2x and root access to your desired target machine.
The current VGG Faces campaign is deployed to Bocoup's infrastructure which is orchastrated from bocoup/infrastructure-foundation.
In order to deploy to, or download reports from, this server, you will need to submit a PR to this (mitmedialab/ajl.ai) repo adding configuration for yourself to ansible/vars/users.yml
. Once this has been done, ask one the
project contributors can provision your account and give you the
"Vault password" needed to run the following commands:
This will open the secrets file in your default editor. Secret data is managed
by Ansible Vault. All secrets are stored in [ansible/vars/secrets.yml
]().
This will prepare a target machine with all system dependencies needed to
run Image Annotator and grant collaborators access to run deployments. You
will be prompted for both a Vault password and a SUDO password during this
task. You will have configured your sudo password at the beginning of the
project when you added yourself to ansible/vars/users.yml
.
This will clone your desired commit to the target machine, install all node dependencies, compile the site with webpack, apply any outstanding migrations to the database and restart the API server.
The database residing on the production server is backed up to S3 hourly. Running this for any target server will completely replace the database with the most recent backup. Take care not to do this for production unless you know what you are doing!
Running the following commands from the root directory of this repository will dump a CSV from our production database to your local machine.
This will export all annotation data to annotations.csv
. You'll be prompted
for a "Vault password" when running this.
This will export all feedback form data to feedback.csv
. You'll be prompted
for a "Vault password" when running this. TODO: implement a feedback form
in the UI. At the time of this writing we are using google forms for feedback.
The front end is running React, Redux, Webpack.
Run npm install
to install all front-end dependencies; then run npm start
to launch the webpack dev server. The application will then be available at localhost:8080.
The front end currently needs the local node/postgres backend running (installation instructions below).
These commands are available after installation within the ajl.ai/
directory:
npm test
: run unit tests with Jest, then lint on exitnpm run lint
: run ESLint to identify syntax & style issues in the codenpm run build
: generate a static build into ajl.ai/dist
npm run storybook
: launch React Storybook for component development (accessible at localhost:6006 by default)The backend is running node, PostgreSQL and an express based set of middlewares for the REST API we expose. In order to develop the application, you must have an installation of PostgreSQL.
There are numerous ways to run PostgreSQL. Choose the one that is most familiar to you!
If you have Docker on your machine, do the following:
docker run --name ia-pg -d -p 5432:5432 -i postgres
docker exec -it ia-pg su postgres -c 'createdb image-annotator'
npm run migrate:up
npm start
# stop postgres
docker stop ia-pg
# start postgres
docker start ia-pg
If you have VirtualBox and Vagrant on your machine, do the following:
printf "PGHOST=10.10.0.100\nPGUSER=image-annotator" > .env
vagrant up
npm run provision:vagrant
npm run migrate:up
npm start
# stop postgres
vagrant halt
# start postgres
vagrant up
If you want to install PostgreSQL locally on Ubuntu, do the following:
sudo apt-get install postgresql
sudo service postgresql stop
sudo sh -c "printf 'local all all trust\nhost all all 127.0.0.1/32 trust\nhost all all ::1/128 trust' > /etc/postgresql/9.5/main/pg_hba.conf"
sudo service postgresql start
createdb -U postgres image-annotator
npm run migrate:up
npm start
# stop postgres
sudo service postgresql stop
# start postgres
sudo service postgresql start
If you want to install PostgreSQL locally on OSX, do the following:
Ensure you have brew installed. Then, run this:
printf "PGUSER=$USER" > .env
brew install postgresql
brew services start postgresql
createdb image-annotator
npm run migrate:up
npm start
# stop postgres
brew services stop postgresql
# start postgres
brew services start postgresql
cd
into the prototypes directory you want to look at and run npm start
(Requires python, equivalent to running python -m SimpleHTTPServer 8000
) or a comparable static web server.
Navigate to the following prototypes in your browser:
This section clarifies the verbiage and terms used within the application code.
An attribute is the object representing a specific type of annotation a user will apply to any given image, such as demographic attributes like "perceived ethnicity" or positional values like "the location of the right eye."
Attribute properties:
Attribute objects have the shape
{
"name": "Attribute Name",
"type": "type-of-data",
"options": ["list", "of", "accepted", "values", "for", "multiple", "choice", "attributes"]
}
An annotation is the object representing the submitted value for a particular Attribute
.
Annotation properties:
Annotation objects have the shape
{
"name": "Attribute Name",
"value": "selected-value"
}
An image annotation is an object associating one or more Annotations
with the specific Image
to which they were applied. It contains an array of annotations, and the ID of their associated image.
ImageAnnotation objects have the shape:
{
"id": 3359,
"annotations": [{
"name": "Attribute Name",
"value": "selected-value"
}, {
"name": "Another Attribute Name",
"value": "selected-value"
}]
}
An image is a representation of an image to which Annotations
will be applied.
Image objects have the shape:
{
"id": 3359,
"url": "http://www.url.com/some-image.jpg",
"width": 250,
"height": 250
}
A workload is an object containing a list of Images
to be annotated.
Workload objects have the shape:
{
"id": 121,
"images": [{
"id": 3359,
"url": "http://url.com/some-image.jpg",
"width": 250,
"height": 250
}, {
"id": 3360,
"url": "http://url.com/other-image.jpg",
"width": 250,
"height": 250
}]
}
An annotated workload is a collection of ImageAnnotations
submitted for a specific Workload
.
Annotated Workload objects have the shape
{
"workloadId": 121,
"images": [{
"id": 3359,
"annotations": [{
"name": "Attribute Name",
"value": "selected-value"
}, {
"name": "Another Attribute Name",
"value": "selected-value"
}]
}, {
"id": 3360,
"annotations": [{
"name": "Attribute Name",
"value": "selected-value"
}, {
"name": "Another Attribute Name",
"value": "selected-value"
}]
}]
}