search

OpenEnergyPlatform / oeplatform

Repository for the code of the Open Energy Platform (OEP) website. The OEP provides an interface to the Open Energy Family
http://openenergyplatform.org/
GNU Affero General Public License v3.0
61 stars 19 forks source link

Starting oeplatform from Docker fails #1622

Closed ebroda closed 3 months ago

ebroda commented 3 months ago

Description of the issue

I wanted to start the oeplatform from Docker as described in /docker/USAGE.md. Therefore I downloaded the docker-compose.yml file and put in inside a directory. Afterwards I was trying to set up the service by using (sudo) docker-compose up. Docker downloaded the two images and all necessary stuff around, but the initialization of oeplatform_1 fails with FileNotFoundError: [Errno 2] No such file or directory: '/app/ontologies/oeo'.

Looks like image is missing these files which are necessary to execute the oeplatform. I didn't changed anything on the repo and didn't build the images myself. I'm just taking the ones latest provided from GitHub.

Docker Log

oeplatform_1  | Checking for Configuration
oeplatform_1  | File /app/oeplatform/securitysettings.py exists, we do not copy the default configuration.
oeplatform_1  | Migrating Databases...
oeplatform_1  | Migrating Django database...
oeplatform_1  | Traceback (most recent call last):
oeplatform_1  |   File "manage.py", line 10, in <module>
oeplatform_1  |     execute_from_command_line(sys.argv)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
oeplatform_1  |     utility.execute()
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 413, in execute
oeplatform_1  |     self.fetch_command(subcommand).run_from_argv(self.argv)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 354, in run_from_argv
oeplatform_1  |     self.execute(*args, **cmd_options)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 398, in execute
oeplatform_1  |     output = self.handle(*args, **options)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 89, in wrapped
oeplatform_1  |     res = handle_func(*args, **kwargs)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 75, in handle
oeplatform_1  |     self.check(databases=[database])
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 423, in check
oeplatform_1  |     databases=databases,
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 76, in run_checks
oeplatform_1  |     new_errors = check(app_configs=app_configs, databases=databases)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 13, in check_url_config
oeplatform_1  |     return check_resolver(resolver)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 23, in check_resolver
oeplatform_1  |     return check_method()
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 416, in check
oeplatform_1  |     for pattern in self.url_patterns:
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 48, in __get__
oeplatform_1  |     res = instance.__dict__[self.name] = self.func(instance)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 602, in url_patterns
oeplatform_1  |     patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 48, in __get__
oeplatform_1  |     res = instance.__dict__[self.name] = self.func(instance)
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 595, in urlconf_module
oeplatform_1  |     return import_module(self.urlconf_name)
oeplatform_1  |   File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
oeplatform_1  |     return _bootstrap._gcd_import(name[level:], package, level)
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
oeplatform_1  |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
oeplatform_1  |   File "/app/oeplatform/urls.py", line 34, in <module>
oeplatform_1  |     url(r"^scenario-bundles/", include("factsheet.urls")),
oeplatform_1  |   File "/usr/local/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
oeplatform_1  |     urlconf_module = import_module(urlconf_module)
oeplatform_1  |   File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
oeplatform_1  |     return _bootstrap._gcd_import(name[level:], package, level)
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
oeplatform_1  |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
oeplatform_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
oeplatform_1  |   File "/app/factsheet/urls.py", line 2, in <module>
oeplatform_1  |     from factsheet import views
oeplatform_1  |   File "/app/factsheet/views.py", line 41, in <module>
oeplatform_1  |     Path(ONTOLOGY_ROOT, "oeo")
oeplatform_1  | FileNotFoundError: [Errno 2] No such file or directory: '/app/ontologies/oeo'
oep_oeplatform_1 exited with code 1

Alternative way

I also build the image on my own. Therefore I cloned the repository and then needed to find out how to build the image. Probably it would be a good idea, to add a short section to /docker/USAGE.md as it's not the usualy simple docker build ., but needs the -f flag. I used sudo docker build -t oeplatform -f docker/Dockerfile .

On first try this also showed a similar error:

$ sudo docker build -t oeplat -f docker/Dockerfile .
[+] Building 106.3s (13/13) FINISHED                                                                                                                                                                                                                       docker:default
 => [internal] load .dockerignore                                                                                                                                                                                                                                             0.0s
 => => transferring context: 77B                                                                                                                                                                                                                                              0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                          0.0s
 => => transferring dockerfile: 655B                                                                                                                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/python:3.7.10                                                                                                                                                                                                              0.5s
 => [internal] load build context                                                                                                                                                                                                                                             1.3s
 => => transferring context: 208.90MB                                                                                                                                                                                                                                         1.2s
 => CACHED [1/9] FROM docker.io/library/python:3.7.10@sha256:c9b155d64106b7c939cb82396251401418dbdd6ccf51a90c9ea4ede13162fa35                                                                                                                                                 0.0s
 => [2/9] RUN apt-get update     && apt-get install -y apache2 apache2-dev libapache2-mod-wsgi-py3    && rm -rf /var/lib/apt/lists/*                                                                                                                                         33.4s
 => [3/9] WORKDIR /app                                                                                                                                                                                                                                                        0.1s
 => [4/9] COPY requirements.txt /app/requirements.txt                                                                                                                                                                                                                         0.1s 
 => [5/9] RUN pip install -r requirements.txt                                                                                                                                                                                                                                70.8s 
 => [6/9] COPY docker/apache2.conf /etc/apache2/conf-enabled/oeplatform.conf                                                                                                                                                                                                  0.1s 
 => [7/9] COPY . /app                                                                                                                                                                                                                                                         0.5s 
 => [8/9] COPY docker/docker-entrypoint.sh /app/docker-entrypoint.sh                                                                                                                                                                                                          0.1s 
 => ERROR [9/9] RUN cp /app/oeplatform/securitysettings.py.default /app/oeplatform/securitysettings.py && python manage.py collectstatic --noinput && rm /app/oeplatform/securitysettings.py                                                                                  0.8s 
------                                                                                                                                                                                                                                                                             
 > [9/9] RUN cp /app/oeplatform/securitysettings.py.default /app/oeplatform/securitysettings.py && python manage.py collectstatic --noinput && rm /app/oeplatform/securitysettings.py:                                                                                             
0.703 ERROR:ontology.apps:The oeo release files in '/app/ontologies/oeo' are missing! The app can`t start.!
0.704 Traceback (most recent call last):
0.704   File "manage.py", line 10, in <module>
0.704     execute_from_command_line(sys.argv)
0.704   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
0.704     utility.execute()
0.704   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
0.704     django.setup()
0.704   File "/usr/local/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
0.704     apps.populate(settings.INSTALLED_APPS)
0.704   File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 122, in populate
0.704     app_config.ready()
0.704   File "/app/ontology/apps.py", line 38, in ready
0.704     f"The oeo release files in '{file_to_check}' are missing! The app can`t start. Please refer to the documentation: {DOCUMENTATION_LINKS.get('oeo_setup')}"
0.704 ImportError: The oeo release files in '/app/ontologies/oeo' are missing! The app can`t start. Please refer to the documentation: https://openenergyplatform.github.io/oeplatform/install-and-documentation/install/installation/#4-setup-the-openenergyontology-integation
------
Dockerfile:18
--------------------
  16 |     
  17 |     
  18 | >>> RUN cp /app/oeplatform/securitysettings.py.default /app/oeplatform/securitysettings.py && python manage.py collectstatic --noinput && rm /app/oeplatform/securitysettings.py
  19 |     
  20 |     EXPOSE 80
--------------------
ERROR: failed to solve: process "/bin/sh -c cp /app/oeplatform/securitysettings.py.default /app/oeplatform/securitysettings.py && python manage.py collectstatic --noinput && rm /app/oeplatform/securitysettings.py" did not complete successfully: exit code: 1

Interesting is, that in this case, explicitely a reference to the docs saying oeo missing is given.

ImportError: The oeo release files in '/app/ontologies/oeo' are missing! The app can`t start. Please refer to the documentation: https://openenergyplatform.github.io/oeplatform/install-and-documentation/install/installation/#4-setup-the-openenergyontology-integation

I'm not sure, why the docker-build in the CI/GitHub Actions succeeded, when it fails on my side. As I added the files accordingly, the build succeded, but the image still not worked, but with a different error message (see issue #1623).

Steps to Reproduce

Ensure you use the docker images provided from GitHub, not locally build ones.

  1. Download docker-compose.yml
  2. sudo docker-compose up
  3. See error above.

Ideas of solution

Context and Environment

Workflow checklist

jh-RLI commented 3 months ago

Thank you for sharing your progress with us and documenting this issue. I have also seen this before in one of the currently open PullRequests. It seems quite strange to me. I won't look at it again today, but sure will tomorrow.

My first guess would be that it's either a hickup or a bug in the django restframework package.

The only difference I've found so far is that the installation of the dependency is unexpected.

See the failing install action here: image

And compare it with one of the currently functioning ones. (last triggered 3 days ago) image

Then I stumbled across this: https://github.com/encode/django-rest-framework/issues/9300

ebroda commented 3 months ago

I think the comments are on the wrong issue, each.

From this comment (https://github.com/OpenEnergyPlatform/oeplatform/issues/1623#issuecomment-2004966583):

Currently you can only use the docker image for tag v0.18.0.

That image fails with the same error, also 0.17.2. Seems like the issue, missing step 4, is there already some time. 

$ sudo docker-compose up
Pulling oeplatform (ghcr.io/openenergyplatform/oeplatform:0.17.2)...
0.17.2: Pulling from openenergyplatform/oeplatform
0bc3020d05f1: Already exists
...
2e80eb32d142: Pull complete
...
3619674a0500: Pull complete
Digest: sha256:7964b83a9373a3c48af5af4e32d0d1212168685ce4a9b437c32722fb31e2741a
Status: Downloaded newer image for ghcr.io/openenergyplatform/oeplatform:0.17.2
Starting docker-oep_postgres_1 ... done
Recreating docker-oep_oeplatform_1 ... done
Attaching to docker-oep_postgres_1, docker-oep_oeplatform_1
postgres_1    | 
postgres_1    | PostgreSQL Database directory appears to contain a database; Skipping initialization
...
oeplatform_1  | Checking for Configuration
oeplatform_1  | File /app/oeplatform/securitysettings.py does not exist, copying default configuration.
oeplatform_1  | Migrating Databases...
oeplatform_1  | Migrating Django database...
oeplatform_1  | Traceback (most recent call last):
...
oeplatform_1  |   File "/app/factsheet/urls.py", line 2, in <module>
oeplatform_1  |     from factsheet import views
oeplatform_1  |   File "/app/factsheet/views.py", line 41, in <module>
oeplatform_1  |     Path(ONTOLOGY_ROOT, "oeo")
oeplatform_1  | FileNotFoundError: [Errno 2] No such file or directory: '/app/ontologies/oeo'
docker-oep_oeplatform_1 exited with code 1
jh-RLI commented 3 months ago

Oh, that's true. Was a bit late that day :)

Well, due to a docker setting that was overlooked for a long time. This problem goes back to the version where we introduced the ontologies folder years ago ... . Also, I stopped using Docker for a few months now. Thanks again for pointing it out.

In v0.18 i just added the check - so at least devs are guided on the documentation if the folder is missing.

ebroda commented 3 months ago

I added a oneliner to Dockerfile which automatizes getting and extracting the latest oeo and put in in #1631 https://github.com/OpenEnergyPlatform/oeplatform/pull/1631/commits/6e6e106bcbd96c533f006763711e374a369df36b

Also did small additions to the docker README.