search

bit-bots / imagetagger

An open source online platform for collaborative image labeling
MIT License
267 stars 54 forks source link
crowdsourcing deep-learning image-annotation image-labeling images imagetagger label-images labeling robocup

readme

ImageTagger

This is a collaborative online tool for labeling image data.

If you are participating in RoboCup, you should not install your own instance but rather use the one provided by the Hamburg Bit-Bots (https://imagetagger.bit-bots.de). This enables you to use already labeled images from other teams and to share your own.

For a short overview of the functions please have a look at the following poster: https://robocup.informatik.uni-hamburg.de/wp-content/uploads/2017/11/imagetagger-poster.pdf

Table of Contents:

Features

Planned Features

Reference

This paper describes the Bit-Bots ImageTagger more in depth. Please cite if you use this tool in your research:

FIEDLER, Niklas, et al. ImageTagger: An Open Source Online Platform for Collaborative Image Labeling. In: RoboCup 2018: Robot World Cup XXII. Springer, 2018.

@inproceedings{imagetagger2018,
   author={Fiedler, Niklas and Bestmann, Marc and Hendrich, Norman},
   year={2018},
   title={ImageTagger: An Open Source Online Platform for Collaborative Image Labeling},
   booktitle={RoboCup 2018: Robot World Cup XXII},
   organization={Springer}
}

Installing and running ImageTagger

ImageTagger can be installed and run locally (best for development), in a docker container or in Kubernetes (used in our deployment).

Locally

In some of the following code snippets, the DJANGO_CONFIGURATION environment variable is exported. This defines the type of deployment by selecting one of our predefined configuration presets. If ImageTagger is running in a development environment, no export is necessary.

  1. Install the latest release

    You need to have Pipenv installed on your system.

    Replace v… with the latest release tag.

    git checkout v…
cd imagetagger
pipenv sync

  2. Setup a database server

    As a database server postgresql is required. Please seek a guide specific to your operating system on how to install a server and get it running.

    Once postgresql is installed, a user and database need to be set up for ImageTagger. Of course, the user and password can be changed to something else.

    CREATE USER imagetagger PASSWORD 'imagetagger';
CREATE DATABASE imagetagger WITH OWNER imagetagger;

  3. Select a Configuration preset

    When running ImageTagger as a developer, no step is necessary because a development configuration is used per default when not running as a docker based deployment. However if this is supposed to be a production deployment, export the following environment variable.

    Currently available presets are Dev and Prod

    export DJANGO_CONFIGURATION=Prod

  4. Configuring ImageTagger to connect to the database

    Please see the lower Configuration section on how to configure ImageTagger for your specific database credentials.

  5. Initialize the database

    Run the following to create and setup all necessary database tables:

    ./manage.py migrate

  6. Create a user

    ./manage.py createsuperuser

  7. Run the server

    ./manage.py runserver

In a production deployment it is necessary to run the following commands after each upgrade:

./manage.py migrate
./manage.py compilemessages
./manage.py collectstatic

for additional steps on some releases see instructions in UPGRADE.md

In-Docker

  1. Checkout the latest release

    git checkout v…

  2. Build the container image

    Note: the Dockerfile has been tested with podman as well.

    docker build -t imagetagger .

  3. Run the container

    docker run -it -p 8000:80 --name imagetagger imagetagger

    This step will not work out of the box because configuration still needs to be done. See the lower section about configuring ImageTagger on how to fix this.

  4. Create a user

    Note: This step requires a container running in the background.

    docker exec -it imagetagger /app/src/imagetagger/manage.py createsuperuser

About the Container

Kind Description
Volume /app/data is where persistent data (like images) are stored
Volume /app/config is where additional custom configuration files can be placed. See the Configuration section below
Environment ImageTagger can mostly be configured via environment variables
Ports The container internal webserver listens on port 80 for incoming connections.

On Kubernetes

  1. Follow the steps for In-Docker on how to build a container image

  2. Apply kubernetes configuration

    Note: This assumes you have access to a kubernetes cluster configured and can use kubectl

    We use kustomize for management of our kubernetes configuration. It can be applied by running the following commands:

    kustomize build . > k8s.yml
kubectl apply -f k8s.yml

Generated Kubernetes resources

Kind Name Description
Namespace imagetagger The namespace in which each resource resides
Deployment + Service imagetagger-postgres postgresql server which is required by ImageTagger. The replica count can be dialed down to 0 if an the usage of an external server is desired.
PersistentVolumeClaim imagetagger-database Where the postgresql server stores its data
ConfigMap imagetagger-postgres Configuration of the postgresql server. Also available inside the application server deployment so that settings can be referenced.
Deployment + Service imagetagger-web application server. Per default this deployment references the image imagetagger:local which is probably not resolvable and should be replaced by a reference to where your previously built container image is available.
PersistentVolumeClaim imagetagger-image-data Where the application server stores its images (and tools).
ConfigMap imagetagger-web Configuration of the application server. Mounted as environment variables. See Configuration for details.

Configuration

ImageTagger is a Django application and uses django-configurations for better configuration management.

ImageTagger is configured to use a Dev configuration when running locally and Prod when running in a container. This can be overridden via the environment variable DJANGO_CONFIGURATION.

For a list of available configuration values see settings.py. Towards the bottom is a list of values. These are taken from environment variables where the key is the variable name but with an IT_ prefix.

If completely custom configuration is desired, imagetagger/imagetagger/settings_local.py can be created in which a custom configuration class may be created. In Docker this file may be located at /app/config/settings.py so that mounting it should be simple. To use this custom configuration class, the environment variables DJANGO_SETTINGS_MODULE=imagetagger.settings_local and DJANGO_CONFIGURATION=MyCustomClass must be set.

If downloading zip files of Imagesets is desired, the feature can be enabled by settings IT_ENABLE_ZIP_DOWNLOAD to true. A separate zip generation daemon must then be started via the following command. This feature is enabled and running automatically in Docker based deployments.

export DJANGO_CONFIGURATION=Prod
./manage.py runzipdaemon

To create annotation types, log into the application and click on Administration at the very bottom of the home page.

Minimal production Configuration

In production, the following configuration values must be defined (as environment variables)

Key Description
IT_SECRET_KEY The django secret key used by ImageTagger. It is used for password hashing and other cryptographic operations.
IT_ALLOWED_HOSTS django ALLOWED_HOSTS as comma separated list of values. It defines the hostnames which this application will allow to be used under.
IT_DB_HOST Hostname (or IP-address) of the postgresql server. When deploying on kubernetes, the provided Kustomization sets this to reference the database deployment.
IT_DB_PASSWORD Password used for authenticating against the postgresql server.
IT_DOWNLOAD_BASE_URL Base-URL under which this application is reachable. It defines the prefix for generated download links.

Database Configuration

The following environment variables configure how the postgresql server is accessed

Key Required Default Description
IT_DB_HOST yes Hostname (or IP-address) of the postgresql server. When deploying on kubernetes, the provided Kustomization sets this to reference the database deployment.
IT_DB_PORT no 5432 Port on which the postgresql server listens.
IT_DB_NAME no imagetagger Database name on the postgresql server. ImageTagger requires full access to it.
IT_DB_USER no imagetagger User as which to authenticate on the postgresql server.
IT_DB_PASSWORD yes Password used for authentication.

Used dependencies

The ImageTagger relies on the following plugins, libraries and frameworks:

We are grateful to the maintainers and contributors of the respective projects.