datamade / how-to

📚 Doing all sorts of things, the DataMade way
MIT License
86 stars 12 forks source link

Add docs on Docker for local development of Python packages #107

Closed hancush closed 3 years ago

hancush commented 4 years ago

Documentation request

@fgregg figured out how to hack on local Python packages using containerized application setups. (Copied from https://github.com/dedupeio/dedupe-service/pull/1500#issuecomment-538135818.)

Notes on working developing a library dependency.

  1. Mount your local copy of the library to the service/s that need it https://github.com/dedupeio/dedupe-service/blob/1e9fd004f3f6fa88c4df92f7fa7fa247a482c641/docker-compose.yml#L23-L33
queue:
    container_name: dedupe-queue
    restart: always
    image: dedupeio:latest
    depends_on:
      - migration
    volumes:
      - .:/app
      - ${PWD}/api/local.py.example:/app/api/local.py
      - ${PWD}/api/secrets.py.example:/app/api/secrets.py
      - dedupe-tmp:/tmp
      - /Users/fgregg/work/dedupe-variable-number:/dedupe-variable-number <-- RELEVANT LINE
    command: python run_queue.py
  1. With the containers running,
> docker exec -it dedupe-queue pip install -e /dedupe-variable-number

N.b., Sometimes the order of installation is meaningful, e.g., in la-metro-councilmatic. If you install a dependency directly with pip, and it breaks the app, try replacing the relevant line in requirements.txt with -e /container/path/to/dependency, then reinstalling all the dependencies: docker-compose up -d && docker-compose exec app pip install -r requirements.txt.

  1. Restart the containers (so Python will reload libs)
  2. ~If you make changes to library you need to restart containers~ This is actually not necessary!

This comes up for Dedupe, as well as for django-councilmatic not infrequently. Let's add this strategy to the local development with Docker documentation.