search

django / django-box

VM to run the Django test suite. ARCHIVED Please use https://github.com/django/django-docker-box
BSD 3-Clause "New" or "Revised" License
67 stars 22 forks source link

Is anyone interested in Docker? #40

Closed kingbuzzman closed 5 years ago

kingbuzzman commented 6 years ago

I recently had to commit some code to django and needed to make sure to run and test django.

I created this: https://gist.github.com/kingbuzzman/3784b49bffa4f14b2b08575541ed7a1e

I wonder if anyone is interested in it.

  1. It maps a local django directory to where you run it from.
  2. Installs all the dependencies needed to run django.
  3. Gives you the prompt so you can mess with it to your hearts content.

If you find any issues with it, i'm all ears.

jarshwah commented 6 years ago

Yes, I think docker would be far easier to manage than the virtualbox we're currently using, so I'd be very interested in getting this project to that point.

A few things though:

Now, we don't actually need to support all of these services/combinations within the same node. I'd be happy enough with something like a docker-compose file with these services:

And some way of choosing which combination of services you want for a particular invocation of the test suite.

If we can reduce this project down to a docker-compose.yml file and some Dockerfiles, I'd be interested in trying to upstream them into the django/django repo.

orf commented 6 years ago

I had exactly the same thought a while back, and yesterday I came up with this: https://github.com/orf/django-docker-box

It's by no means perfect, but it runs the Django test suite across all supported backends (including Oracle!).

With docker-compose it's really simple to include these extra services like memcached (which I had completely forgotten about!) and Selenium. It's also pretty easy to use docker-selenium to run tests across multiple browsers.

Please let me know if you think it's a good starting point.

orf commented 6 years ago

As a side note, using this I've got Travis to run a comprehensive Django test suite with a matrix of every single supported Python and database version except Oracle (will require some extra setup to get instaclient into the builds, but it's completely possible). I think that's pretty neat.

Interestingly enough there seems to be a reliable failure with the Django provided test_sqlite.py and the caching tests that might be worth looking into. It seems the caching keys are getting mixed up somehow and possibly interacting - using two separate memcached instances fixes this.

jarshwah commented 6 years ago

That's awesome, nice work 👍

I haven't gone through the project thoroughly yet, but I'll do so in the next few days.

felixxm commented 6 years ago

@orf You do not need an instantclient since cx_Oracle >= 6.0 (see release notes):

_Simplify building cx_Oracle considerably by use of ODPI-C. This means that cx_Oracle can now be built without Oracle Client header files or libraries and that at runtime cxOracle can adapt to Oracle Client 11.2, 12.1 or 12.2 libraries without needing to be rebuilt. This also means that wheels can now be produced and installed via pip.

felixxm commented 6 years ago

@orf Sorry for confusing :disappointed: , it seems that basic instantclient is still required if you do not have Oracle database installed locally. The only thing that has been changed is that you do not need Oracle SDK anymore.

orf commented 6 years ago

Thanks for the clarification @felixxm! Their licensing is pretty annoying :( I've got the Oracle tests running using a third party image (https://hub.docker.com/r/sath89/oracle-12c/), but would love to get them to use the official image instead.

I've also just added support for GIS tests in docker-box, bringing the total compatibility matrix to 62 (excluding Oracle)!

Right now it's functional, but there is a bit of a mess with volumes and dependency installation. I mount Django's source code at a specific location and run the tests, which means that the extra dependencies are not available at image build time and so are re-installed every run.

I'd like to use COPY and RUN so this does not happen, but Django's .git directory on my machine is upwards of 240mb, meaning about 350mb has to be sent to the Docker daemon before any building can take place. It would be good to have a .dockerignore with .git in the Django repo to prevent this from happening.

Anyway, it needs some refactoring but any feedback would be appreciated.

orf commented 6 years ago

Thanks for the clarification @felixxm! Their licensing is pretty annoying :( I've got the Oracle tests running using a third party image (https://hub.docker.com/r/sath89/oracle-12c/), but would love to get them to use the official image instead. I've also just added support for GIS tests in docker-box, bringing the total compatibility matrix to 62 (excluding Oracle)!

Right now it's functional, but there is a bit of a mess with volumes and dependency installation. I mount Django's source code at a specific location and run the tests, which means that the extra dependencies are not available at image build time and so are re-installed every run.

I'd like to use COPY and RUN so this does not happen, but Django's .git directory on my machine is upwards of 240mb, meaning about 350mb has to be sent to the Docker daemon before any building can take place. It would be good to have a .dockerignore with .git in the Django repo to prevent this from happening.

Anyway, it needs some refactoring but any feedback when you have time @jarshwah would be appreciated.

jarshwah commented 6 years ago

Let's continue the conversation over on your repo - and then if django org is happy to adopt as the recommended solution in place of vagrant, we can migrate it over and add you as a contributor. I'll start up an issue over on your repo (here: https://github.com/orf/django-docker-box/issues/1)