tobybatch / kimai2

Docker containers for the kimai2 web application including docker-compose and kubernetes/helm deployment.
MIT License
183 stars 97 forks source link

[BUG] kimai/kimai2:fpm-alpine-1.9-prod container not working since last update #144

Closed Dwayne28 closed 3 years ago

Dwayne28 commented 4 years ago

Describe the bug We're using the docker image kimai/kimai2:fpm-alpine-1.9-prod with the recent update last week our kimai container is showing errors in the logs and is not working at all also the health status shows: unhealthy

The logs are mainly around these two messages:

  1. request.ERROR: Uncaught PHP Exception Symfony\Component\HttpKernel\Exception\BadRequestHttpException: "Untrusted Host "nginx"." at /opt/kimai/vendor/symfony/http-kernel/HttpKernel.php line 83

  2. General error: 1017 Can't find file: './kimai/kimai2_configuration.frm' (errno: 13 - Permission denied) at /opt/kimai/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:75)"}

we try to resolve using older images but this seems to occur from 1.6 and upwards with the given docker-compose.yml the only working version for us is 1.5 but we're missing some api methods there which are included since 1.8

Thanks in advance

To Reproduce Steps to reproduce the behaviour:

  1. Start the container
  2. Try to connect to my-ip/api/doc or my-ip
  3. See error
  4. find log from kimai container via docker logs containerID

Desktop (please complete the following information):

Command used to run the container

Docker compose file (with passwords redacted)

version: '3.5'
services:

  sqldb:
    image: mysql:5.7
    environment:
      - MYSQL_DATABASE=kimai
      - MYSQL_USER=kimaiuser
      - MYSQL_PASSWORD=kimaipassword
      - MYSQL_ROOT_PASSWORD=changemeplease
    volumes:
      - ../var/data/mysql:/var/lib/mysql
    command: --default-storage-engine innodb
    restart: unless-stopped
    healthcheck:
      test: mysqladmin -pchangemeplease ping -h localhost
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

  nginx:
    build: compose
    ports:
      - 80:80
    volumes:
      - ./compose/nginx_site.conf:/etc/nginx/conf.d/default.conf:ro
      - public:/opt/kimai/public:ro
    restart: unless-stopped
    depends_on:
      - kimai
    healthcheck:
      test:  wget --spider http://nginx/health || exit 1
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

  kimai:
    image: kimai/kimai2:fpm-alpine-1.9-prod
    environment:
      - APP_ENV=prod
      - TRUSTED_HOSTS=localhost, my-ip
      - ADMINMAIL=admin@kimai.local
      - ADMINPASS=changemeplease
      - DATABASE_URL=mysql://kimaiuser:kimaipassword@sqldb/kimai
    volumes:
      - public:/opt/kimai/public
      - var:/opt/kimai/var
      - ../var/lib/data:/opt/kimai/var/data
      # - ./ldap.conf:/etc/openldap/ldap.conf:z
      # - ./ROOT-CA.pem:/etc/ssl/certs/ROOT-CA.pem:z
    restart: unless-stopped
    healthcheck:
      test: wget --spider http://nginx || exit 1
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

  postfix:
    image: catatnight/postfix:latest
    environment:
      maildomain: neontribe.co.uk
      smtp_user: kimai:kimai
    restart: unless-stopped

volumes:
    var:
    public:

Additional context Add any other context about the problem here.

tobybatch commented 4 years ago

The PHP version was bumped in the last release and I guess this might be a cause of this. The error in the TRUSTED_HOSTS setting in the kimai section of your docker compare.

Try adding nginx to the trusted hosts line:

TRUSTED_HOSTS=localhost, my-ip, nginx

I am on holiday and only have my phone, bit that should fix it.

Dwayne28 commented 4 years ago

Thanks for the quick reply and for replying while on holiday hope you enjoy.

Both changes seems to do the trick. On my local machine everything is working correct.

But on my server which runs the setup mentioned above.

There is one more issue when you start a Timesheet and want to select an activity for a specific customer, the search (dropdown) show's only activities which are not assigned to a customer/project

the log in the kimai container shows this error:

NOTICE: PHP message: [2020-08-19 08:25:34] request.ERROR: Uncaught PHP Exception FOS\RestBundle\Exception\InvalidParameterException: "Parameter "project" of value "%project%" violated a constraint "Parameter 'project' value, does not match requirements '\d+'"" at /opt/kimai/vendor/friendsofsymfony/rest-bundle/Exception/InvalidParameterException.php line 68 {"exception":"[object] (FOS\RestBundle\Exception\InvalidParameterException(code: 0): Parameter \"project\" of value \"%project%\" violated a constraint \"Parameter 'project' value, does not match requirements '\d+'\" at /opt/kimai/vendor/friendsofsymfony/rest-bundle/Exception/InvalidParameterException.php:68)"}

the container on my local machine doesn't show this error/problem in the logs.

Thanks in advance again.

kevinpapst commented 4 years ago

Last one is not a docker issue and should be fixed with 1.10.

Dwayne28 commented 4 years ago

Thanks, okay any timeline when a docker image with 1.10 will be released?

kevinpapst commented 4 years ago

Try master image

Dwayne28 commented 4 years ago

I tried master which runs on kimai 1.8, if I'm correct? But same issue

I created a new Dockerfile from the current kimai repo: https://github.com/kevinpapst/kimai2 branch: master

It runs but still the same issue: NOTICE: PHP message: [2020-08-19 12:35:16] request.ERROR: Uncaught PHP Exception FOS\RestBundle\Exception\InvalidParameterException: "Parameter "project" of value "%project%" violated a constraint "Parameter 'project' value, does not match requirements '\d+'"" at /opt/kimai/vendor/friendsofsymfony/rest-bundle/Exception/InvalidParameterException.php line 68 {"exception":"[object] (FOS\RestBundle\Exception\InvalidParameterException(code: 0): Parameter \"project\" of value \"%project%\" violated a constraint \"Parameter 'project' value, does not match requirements '\d+'\" at /opt/kimai/vendor/friendsofsymfony/rest-bundle/Exception/InvalidParameterException.php:68)"} []

But I found that under the tab Doctor in the kimai UI

there is a file permission issue on the folder var/data/

maybe this is the issue I will see if I can fix this do you have any idea on this?

Thanks so far

*edit fixed the file permission -> issue still exists error message from above is still there

kevinpapst commented 4 years ago

Master should be master => almost 1.10. But I am not really a docker user, so I might be wrong here.

This is very likely a Javascript problem. At least I fix one with the exact same error message. But it only happened in rare edge cases with projects without activities/customers without projects.

Dwayne28 commented 4 years ago

Still couldn't find a solution. I found out that if I use 1.5 the problem does not exist but we're missing some API'calls which we need.

The install/setup is fresh we just setup one customer, one project (assigned to customer) and one activity (assigned to project and customer).

Clicking on the top right to start an activity selecting customer -> project -> activities are empty.

The weird thing is on my local machine it all works fine.

The machine we're running it on is a fresh vm with ubuntu 20.04 LTS and the external port we use is not 80 but this is anyway handled by docker.

kevinpapst commented 4 years ago

Whats in your browser console?

Edit: and what version does Kimai report?

tobybatch commented 4 years ago

@Dwayne28 I'm back home for a couple of days this weekend. I'll try and push a 1.10 docker image but you can build your own.

Pull the repo and run

./bin/build.sh 1.10

Should build 1.10 images.

jokay commented 4 years ago

Tried to reproduce the problem today but without success.

Dwayne28 commented 4 years ago

Thanks for investigated that much I really appreciate that.

I checked the browser console:

(when selecting projects dropdown in creating timesheet) myIP/api/projects?customer=%25customer%25&visible=1&start=%25begin%25&end=%25end%25 Response {"code":400,"message":"Bad Request"}

(when selecting customer dropdown in creating timesheet) myIP/api/activities?project=%25project%25&visible=1 Response {„code":400,"message":"Bad Request"}

And that is where the error in the create.js occurs.

Search for "<— error here Failed to load resource 400"

 C.ajaxTransport((function(e) {
                var t, r;
                if (y.cors || Gt && !e.crossDomain)
                    return {
                        send: function(i, o) {
                            var a, s = e.xhr();
                            if (s.open(e.type, e.url, e.async, e.username, e.password),
                            e.xhrFields)
                                for (a in e.xhrFields)
                                    s[a] = e.xhrFields[a];
                            for (a in e.mimeType && s.overrideMimeType && s.overrideMimeType(e.mimeType),
                            e.crossDomain || i["X-Requested-With"] || (i["X-Requested-With"] = "XMLHttpRequest"),
                            i)
                                s.setRequestHeader(a, i[a]);
                            t = function(e) {
                                return function() {
                                    t && (t = r = s.onload = s.onerror = s.onabort = s.ontimeout = s.onreadystatechange = null,
                                    "abort" === e ? s.abort() : "error" === e ? "number" != typeof s.status ? o(0, "error") : o(s.status, s.statusText) : o(Vt[s.status] || s.status, s.statusText, "text" !== (s.responseType || "text") || "string" != typeof s.responseText ? {
                                        binary: s.response
                                    } : {
                                        text: s.responseText
                                    }, s.getAllResponseHeaders()))
                                }
                            }
                            ,
                            s.onload = t(),
                            r = s.onerror = s.ontimeout = t("error"),
                            void 0 !== s.onabort ? s.onabort = r : s.onreadystatechange = function() {
                                4 === s.readyState && n.setTimeout((function() {
                                    t && r()
                                }
                                ))
                            }
                            ,
                            t = t("abort");
                            try {
                                s.send(e.hasContent && e.data || null) // <— error here Failed to load resource 400
                            } catch (e) {
                                if (t)
                                    throw e
                            }
                        },
                        abort: function() {
                            t && t()
                        }
                    }
            }
            )),
Dwayne28 commented 4 years ago

Some additional info: my external port is also not the default 80 or 443

myIP:10107/api/doc

I also cannot use the api via the swagger ui

kevinpapst commented 4 years ago

@tobybatch 1.10 is not yet tagged

@Dwayne28

Which time-tracking are you using? The URLs look pretty wrong. Are you sure that the first projects is right?

when selecting projects dropdown in creating timesheet myIP/api/projects?customer=%25customer%25&visible=1&start=%25begin%25&end=%25end%25

Changing a project should only trigger a call to api/activities...

Regarding the port: https://www.kimai.org/documentation/rest-api.html#using-the-swagger-ui

Dwayne28 commented 4 years ago

Timetracking Mode (*tested also other modes no difference only the call structure is a little different but still the error) [Default] start and end times can be edited

Select Customer dropdown myIP:10107/api/projects?customer=%25customer%25&visible=1&start=%25begin%25&end=%25end%25

Select Project dropdown myIP:10107/api/activities?project=%25project%25&visible=1

Select activity dropdown No XHR log found

kevinpapst commented 4 years ago

Okay, for some reason the javascript cannot find the linked select to fetch the data from them.

The JS should take should take the selected value from the customer dropdown and replace customer=%customer% with the value eg customer=10 and then ask the api for the projects of that customer.

Did you test eg. other browsers?

This is really interesting, because you are not the first user to report that problem. But no one was able to tell me how to reproduce it.

Dwayne28 commented 4 years ago

I tested with Firefox, Brave(chromium) and Safari

Docker Local Machine Creating Timesheet -> Selecting Customer Box (two calls) http://localhost/api/projects?customer=2&visible=1&start=2020-08-20T22:16:00&end= http://localhost/api/activities?globals=true&visible=1

Docker VM Machine Creating Timesheet -> Selecting Customer Box (one calls -> 400) myIp:10107/api/projects?customer=%25customer%25&visible=1&start=%25begin%25&end=%25end%25

The Swagger local.yaml works btw.

The issue is really weird since it is a fresh install running alongside with docker.

If I can help you in anyway let me know

*edit I copied the curl call and exchanged the path+queryParameters with the one from the localhost and I get correct results as expected

kevinpapst commented 4 years ago

@Dwayne28 I released Kimai 1.10 at the weekend, maybe you can test it again once the new docker image is there?

Dwayne28 commented 4 years ago

@kevinpapst thanks for providing the new version.

I created the docker with the build command @tobybatch provided. The build went thru without any issues.

Sadly the activities still not show up same error like before.

I'm out of ideas

kevinpapst commented 4 years ago

This is so weird, considering that it 1. works in one system and not in the other and 2. that it is frontend code that seems to fail which should be 100% identical in both installations. Did you try to use the same database and configuration from "Docker Local Machine" (where it works) in the "Docker VM Machine" container (where it doesn't work)? Just to eliminate the possibility of a data inconsistency problem and to verify that it is really the frontend that causes the issues.

Dwayne28 commented 4 years ago

"Docker VM Machine" & "Docker Local Machine" I have the mysql database mounted to the local filesystem

"Docker VM Machine" stopped docker; deleted the local mounted files; and copied the files from local to server; docker-compose start -d

Server Data is now identical to local. The issue still exists.

For the config I use everything as it is like a fresh installation

Maybe I create another vm and test it there.

kevinpapst commented 4 years ago

So the only difference is host and port, right? And the javascript works in one system and not the other ... Is there anything else in the javascript console regarding blocked resources, CORS issues or alike? I don't get it and would have to see it for myself, unfortunately I wasn't able to reproduce it.

Can you paste the commands you used to build the container that doesn't work? I am not a docker user...

Dwayne28 commented 4 years ago

Sry for the late reply.

To clarify I had also on the local machine the port changed to the one I use externally no issue on the local machine

Steps I did/do

  1. I cloned this repo
  2. changed docker-compose.yaml to the one below
  3. cmd: docker-compose up -d
  4. navigating to localhost:10107 | yourIp:10107
  5. login with admin@kimai.local pw: changemeplease
  6. customer -> create customer A; nothing else changed;
  7. project -> create Project A; for customer A; nothing else changed;
  8. activity -> create activity A; for customer A and Project A; nothing else changed;
  9. Press start activity on top right corner; selecting, customer; project; no activities show up

The image: kimai/kimai2:fpm-alpine-1.10-prod is not available from docker hub to create it use cmd: ./bin/build.sh 1.10 -b fpm-alpine -s prod but to speed things up you can also use the image: kimai/kimai2:fpm-alpine-1.9-prod which is available from dockerhub

To see the logs for container: docker ps copy CONTAINER ID you want docker logs -f container_id

version: '3.5'
services:

  sqldb:
    image: mysql:5.7
    environment:
      - MYSQL_DATABASE=kimai
      - MYSQL_USER=kimaiuser
      - MYSQL_PASSWORD=kimaipassword
      - MYSQL_ROOT_PASSWORD=changemeplease
    volumes:
      - ../var/data/mysql:/var/lib/mysql
    command: --default-storage-engine innodb
    restart: unless-stopped
    healthcheck:
      test: mysqladmin -pchangemeplease ping -h localhost
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

  nginx:
    build: compose
    ports:
      - 10107:80
    volumes:
      - ./compose/nginx_site.conf:/etc/nginx/conf.d/default.conf #:ro 
      - public:/opt/kimai/public # :ro
    restart: unless-stopped
    depends_on:
      - kimai
    healthcheck:
      test:  wget --spider http://nginx/health || exit 1
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

  kimai:
    # build:
    #   context: .
    #   dockerfile: Dockerfile_kimai
    image:  kimai/kimai2:fpm-alpine-1.10-prod #kimai/kimai2:latest-main #kimai/kimai2:fpm-alpine-1.9-prod
    environment:
      - APP_ENV=prod
      - TRUSTED_HOSTS=localhost,nginx, yourIp
      - ADMINMAIL=admin@kimai.local
      - ADMINPASS=changemeplease
      - DATABASE_URL=mysql://kimaiuser:kimaipassword@sqldb/kimai
    volumes:
      - public:/opt/kimai/public
      - var:/opt/kimai/var
      #- ../var/lib/data:/opt/kimai/var/data
      # - ./ldap.conf:/etc/openldap/ldap.conf:z
      # - ./ROOT-CA.pem:/etc/ssl/certs/ROOT-CA.pem:z
    restart: unless-stopped
    healthcheck:
      test: wget --spider http://nginx || exit 1
      interval: 20s
      start_period: 10s
      timeout: 10s
      retries: 3

  postfix:
    image: catatnight/postfix:latest
    environment:
      maildomain: neontribe.co.uk
      smtp_user: kimai:kimai
    restart: unless-stopped

volumes:
    var:
    public:
mariokorte commented 4 years ago

When will Version 1.10 be tagged?

tobybatch commented 3 years ago

I'm on vacation until September 2nd. I'll tag it then.

You can always pull the repo and build it yourself.

tobybatch commented 3 years ago

@Dwayne28 I'm pushing 1.10 and 1.10.1 right now, give it 30 mins and you should be able to use 1.10.1

kevinpapst commented 3 years ago

@tobybatch there was a problem with some empty dashboard widgets in 1.10.1 that seemed to occur only in some environments (combination of probably older PHP and ICU versions). No clue which systems were exactly affected - I only saw it happen in one customer environment. Thank you PHP for that consistent behavior resulting in production bugs 😁

Long story short: latest release is 1.10.2.

Dwayne28 commented 3 years ago

Sry for the late reply. Was already testing it the day you mentioned it. No change. I still need to setup a new vm and test it there.

@tobybatch thanks, tested it, same problem sadly

@kevinpapst I will try to build the container with this version.

thanks a lot, I will keep you informed when I tested on another vm