Install dockerized PartKeepr instance with OctoPart support

[marcolino7]

Hi, just installed latest version of Partkeepr 1.4.0 on docker, and added OctoPart API key on parameters.php. When I search for a component, partkeepr always return an empty list.

What can I check to start troubleshooting?

Thanks

[marcolino7]

When I make a serach on OctoPart, got this: 2021-02-04T18:19:36.877760328Z 192.168.1.3 - - [04/Feb/2021:18:19:36 +0000] "GET /api/octopart/query/?q=PHP79NQ08LT%2C127&_dc=1612462775638&page=1 HTTP/1.1" 500 425 "http://partkeepr.casa.me/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36", 2021-02-04T18:19:34.119112794Z 192.168.1.3 - - [04/Feb/2021:18:19:33 +0000] "GET /api/storage_locations?_dc=1612462772945&page=1&start=0&itemsPerPage=25&group=%7B%22property%22%3A%22categoryPath%22%2C%22direction%22%3A%22ASC%22%7D&order=%5B%7B%22property%22%3A%22category.categoryPath%22%2C%22direction%22%3A%22ASC%22%7D%2C%7B%22property%22%3A%22name%22%2C%22direction%22%3A%22ASC%22%7D%5D HTTP/1.1" 200 1989 "http://partkeepr.casa.me/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36"

[christianlupus]

The problem is that the PartKeepr version 1.4.0 is not up to date with the current OctoPart API. The latest release is no longer compatible. In the master branch there is work done towards this. You might want to look there. But this is a "known issue".

[marcolino7]

Hi, is there available a working docker image with full Partkeepr installed?

I am trying to compose by myself using docker folder and how to available here, but when container start and I open the setup page, I got this error in Apache:

[Sat Feb 06 00:30:45.785457 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP Warning:  require_once(/var/www/pk/web/../app/bootstrap.php.cache): failed to open stream: No such file or directory in /var/www/pk/web/app.php on line 6, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785548 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP Stack trace:, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785559 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP   1. {main}() /var/www/pk/web/app.php:0, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785707 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP Fatal error:  require_once(): Failed opening required '/var/www/pk/web/../app/bootstrap.php.cache' (include_path='.:/usr/local/lib/php') in /var/www/pk/web/app.php on line 6, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785725 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP Stack trace:, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785733 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP   1. {main}() /var/www/pk/web/app.php:0, referer: http://192.168.1.75:8082/setup/
192.168.1.191 - - [06/Feb/2021:00:30:45 +0000] "GET /bundles/partkeeprfrontend/images/favicon.ico HTTP/1.1" 200 817 "http://192.168.1.75:8082/setup/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36"

Thanks

[christianlupus]

Regarding a docker image for production see #1132. There is still work to be done. For now, I suggest going with the docker-compose setup by mhubig.

I just tried the included dev setup as in the upstream repository. For me, it is working without issues. Did you follow the guide? If this is not the issue, you might need to be more verbose in order to help you out there.

[marcolino7]

Hi @christianlupus I tried to take Dockerfile from mhubig, and replace part where it download 1.4.0 stable, with a git clone of repository and some folder moving, like this:

git clone https://github.com/partkeepr/PartKeepr.git
mv ./PartKeepr/* ./
rm -r PartKeepr/

but when I start container it immediately stop after few second with exit code 255.

I also tried to follo guide here https://github.com/partkeepr/PartKeepr/tree/master/docker#setting-up-a-development-environment, as from your suggestion, but when I start container and point browser to setup page, I got a blank page, and the error I put in my last post:

[Sat Feb 06 00:30:45.785457 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP Warning:  require_once(/var/www/pk/web/../app/bootstrap.php.cache): failed to open stream: No such file or directory in /var/www/pk/web/app.php on line 6, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785548 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP Stack trace:, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785559 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP   1. {main}() /var/www/pk/web/app.php:0, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785707 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP Fatal error:  require_once(): Failed opening required '/var/www/pk/web/../app/bootstrap.php.cache' (include_path='.:/usr/local/lib/php') in /var/www/pk/web/app.php on line 6, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785725 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP Stack trace:, referer: http://192.168.1.75:8082/setup/
[Sat Feb 06 00:30:45.785733 2021] [php7:notice] [pid 35] [client 192.168.1.191:29566] PHP   1. {main}() /var/www/pk/web/app.php:0, referer: http://192.168.1.75:8082/setup/
192.168.1.191 - - [06/Feb/2021:00:30:45 +0000] "GET /bundles/partkeeprfrontend/images/favicon.ico HTTP/1.1" 200 817 "http://192.168.1.75:8082/setup/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36"

even if I set PARTKEEPR_FORCE_UPDATE to yes as suggested. Can you help me to make a Dockerfile adapting mhubig code and make latest version working?

Many Thanks

[christianlupus]

The thing with PARTKEEPR_RESET_FORCE is something specific to the development environment. I fear there are some things going completely wrong.

In order to help you, I need a few information from you:

• what do you want to achieve? Special installation requirements?
• Do you want to set up a development environment or a production environment?
• You seem to prefer docker over a Host based installation, right?
• latest branch in repo or latest published release?

Starting from there, I need to know what you have already achieved and how you tried to set up things. Are you using docker plainly or with docker-compose?

If docker plain: what containers are running yet? What is missing? How are these connected? I need to know your setup.

If docker-compose: you could share your config file at best or give me a link to it if you use a foreign one.

Out of curiosity: how much experience with Linux do you have? Is the docker container running on a Linux or windows host?

[marcolino7]

Hi @christianlupus , here is answer to your question:

what do you want to achieve? Special installation requirements? No special requirments, I need normal installation with Octopart Working

Do you want to set up a development environment or a production environment? Is a "production" enviroment for private use. I will use Partkeepr to keep track of my component and its datasheet in my small basement where I play with electronic

You seem to prefer docker over a Host based installation, right? I prefer docker, because I use it on my small home enviroment

latest branch in repo or latest published release? I Need version that work in all basic function, included Octopart

Starting from there, I need to know what you have already achieved and how you tried to set up things. Are you using docker plainly or with docker-compose? I fireup a working Partkeepr in a docker container from mhubig image. For me could be enought, but in version contained in last docker image from mhubig, Octopart is not wotking

If docker-compose: you could share your config file at best or give me a link to it if you use a foreign one. I did follow gude here: https://github.com/partkeepr/PartKeepr/tree/master/docker#setting-up-a-development-environment step by step, but result is a white page when I load setup. As told before I just need a stable version but with Octopart intergration working. For me could be enough a mhubig docker image, but with Octopart integration working.

Out of curiosity: how much experience with Linux do you have? Is the docker container running on a Linux or windows host? I have basic known of linux, I use it for my home servers (home automation, security camera, etc) but I am not very Expert. I use docker on linux.

Many Thanks Marco

[christianlupus]

As Octopart is not part of any release, as far as I know, you will need to access the master branch for now. Unfortunately, this is by definition not stable but the latest development branch. So if issues arise, you might report them here. Docker is perfectly fine as it will allow containing the ancient PHP version in a defined environment.

I assume you did use the provided docker-compose.yml file in the repo. If you changed anything there, please tell me so.

Go to the project folder (where the docker-compose.yml file lives) and issue the following commands:

docker-compose stop app
docker-compose rm app
docker-compose up -d app

This will/should rebuild the container. Do not try to access it in the browser. Wait about 30 seconds and then have a look at docker-compose logs app. The last messages should be something like "ready for connections" or so (I do not know the exact wording, it's from apache). It might take even longer than a few minutes as there are quite some steps involved to build the app. Once the ready-to-serve message has shown up, please post the logs either here or as a gist so I can examine if there is something strange in it.

Edit: Another test you can do is to run the following command (post output here as well):

docker-compose run --rm --entrypoint bash app ls /var/www/pk -l

[marcolino7]

Hi, no modification to docker-compose.yml. I ran 3 commands you provide and i got this logs. After more than 5 minutes there is not a "ready-to-serve message":

Attaching to partkeepr_dev_app_1
app_1     | No configuration was found yet. A defult configuration was generated.
app_1     | Please visit the setup page if you encounter issues.
app_1     | /user-entrypoint.sh: 11: /user-entrypoint.sh: cannot create app/config/parameters.php: Permission denied
app_1     | /user-entrypoint.sh: 12: /user-entrypoint.sh: cannot create app/config/parameters.php: Permission denied
app_1     | Loading composer repositories with package information
app_1     | Installing dependencies (including require-dev) from lock file
app_1     | Warning: The lock file is not up to date with the latest changes in composer.json. You may be getting outdated dependencies. It is recommended that you run `composer update` or `composer update <package name>`.
app_1     | Package operations: 111 installs, 0 updates, 0 removals
app_1     |
app_1     |
app_1     |   [RuntimeException]
app_1     |   /var/www/pk/vendor does not exist and could not be created.
app_1     |
app_1     |
app_1     | install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] [<packages>]...
app_1     |
app_1     |
app_1     | Warning: require_once(/var/www/pk/app/bootstrap.php.cache): failed to open stream: No such file or directory in /var/www/pk/app/console on line 10
app_1     |
app_1     | Call Stack:
app_1     |     0.0003     352224   1. {main}() /var/www/pk/app/console:0
app_1     |
app_1     |
app_1     | Fatal error: require_once(): Failed opening required '/var/www/pk/app/bootstrap.php.cache' (include_path='.:/usr/local/lib/php') in /var/www/pk/app/console on line 10
app_1     |
app_1     | Call Stack:
app_1     |     0.0003     352224   1. {main}() /var/www/pk/app/console:0
app_1     |
app_1     | PHP Warning:  require_once(/var/www/pk/app/bootstrap.php.cache): failed to open stream: No such file or directory in /var/www/pk/app/console on line 10
app_1     | PHP Stack trace:
app_1     | PHP   1. {main}() /var/www/pk/app/console:0
app_1     | PHP Fatal error:  require_once(): Failed opening required '/var/www/pk/app/bootstrap.php.cache' (include_path='.:/usr/local/lib/php') in /var/www/pk/app/console on line 10
app_1     | PHP Stack trace:
app_1     | PHP   1. {main}() /var/www/pk/app/console:0
app_1     | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.19.0.2. Set the 'ServerName' directive globally to suppress this message
app_1     | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.19.0.2. Set the 'ServerName' directive globally to suppress this message
app_1     | [Sun Feb 07 16:09:28.360775 2021] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.1.33 configured -- resuming normal operations
app_1     | [Sun Feb 07 16:09:28.360860 2021] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'

and running docker-compose run --rm --entrypoint bash app ls /var/www/pk -l I got:

Creating partkeepr_dev_app_run ... done
/bin/ls: /bin/ls: cannot execute binary file
ERROR: 126

[christianlupus]

OK, we know you have got a permission issue on your machine. The entrypoint script is not able to initialize PartKeepr correctly.

As soon as these messages appear, the entrypoint script has finished (but failed in your case) and apache is started:

app_1     | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.19.0.2. Set the 'ServerName' directive globally to suppress this message
app_1     | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.19.0.2. Set the 'ServerName' directive globally to suppress this message
app_1     | [Sun Feb 07 16:09:28.360775 2021] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.1.33 configured -- resuming normal operations
app_1     | [Sun Feb 07 16:09:28.360860 2021] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'

Let's try these ones

docker-compose run --rm app ls /var/www/pk -l
docker-compose run --rm app ls /var/www/pk -ln

Additionally: On the host, where did you install/download the PK repo (just for sake of writing correct paths here)? What user is the owner (on the host) of the files (aka who has run git clone)? What UID is that of the user? What is configured in the docker-compose.yml for GITHUB_DEBUG_UID?

[marcolino7]

Hi, here is the results of 2 commands you ask me to run:

root@docker:~/PartKeepr/docker/development# docker-compose run --rm app ls /var/www/pk -l
Creating volume "partkeepr_dev_db" with default driver
Creating partkeepr_dev_app_run ... done
Calling manual command ls /var/www/pk -l as user www-data
total 356
-rw-r--r-- 1 root root  12004 Feb  7 16:00 CHANGELOG.md
-rw-r--r-- 1 root root   2482 Feb  7 16:00 CONTRIBUTING.md
-rw-r--r-- 1 root root  35146 Feb  7 16:00 LICENSE
-rw-r--r-- 1 root root    818 Feb  7 16:00 LICENSE.3rdparty
-rw-r--r-- 1 root root   2011 Feb  7 16:00 README.md
-rw-r--r-- 1 root root    513 Feb  7 16:00 THANKS
drwxr-xr-x 5 root root   4096 Feb  7 16:00 app
-rw-r--r-- 1 root root     50 Feb  7 16:00 app.json
-rw-r--r-- 1 root root    103 Feb  7 16:00 build.properties
-rw-r--r-- 1 root root   4383 Feb  7 16:00 build.xml
-rw-r--r-- 1 root root   3894 Feb  7 16:00 composer.json
-rw-r--r-- 1 root root 245772 Feb  7 16:00 composer.lock
drwxr-xr-x 6 root root   4096 Feb  7 16:00 data
drwxr-xr-x 4 root root   4096 Feb  7 16:00 docker
drwxr-xr-x 5 root root   4096 Feb  7 16:00 documentation
drwxr-xr-x 3 root root   4096 Feb  7 16:00 src
drwxr-xr-x 5 root root   4096 Feb  7 16:00 theme
drwxr-xr-x 4 root root   4096 Feb  7 16:00 web
root@docker:~/PartKeepr/docker/development#

root@docker:~/PartKeepr/docker/development# docker-compose run --rm app ls /var/www/pk -ln
Creating partkeepr_dev_app_run ... done
Calling manual command ls /var/www/pk -ln as user www-data
total 356
-rw-r--r-- 1 0 0  12004 Feb  7 16:00 CHANGELOG.md
-rw-r--r-- 1 0 0   2482 Feb  7 16:00 CONTRIBUTING.md
-rw-r--r-- 1 0 0  35146 Feb  7 16:00 LICENSE
-rw-r--r-- 1 0 0    818 Feb  7 16:00 LICENSE.3rdparty
-rw-r--r-- 1 0 0   2011 Feb  7 16:00 README.md
-rw-r--r-- 1 0 0    513 Feb  7 16:00 THANKS
drwxr-xr-x 5 0 0   4096 Feb  7 16:00 app
-rw-r--r-- 1 0 0     50 Feb  7 16:00 app.json
-rw-r--r-- 1 0 0    103 Feb  7 16:00 build.properties
-rw-r--r-- 1 0 0   4383 Feb  7 16:00 build.xml
-rw-r--r-- 1 0 0   3894 Feb  7 16:00 composer.json
-rw-r--r-- 1 0 0 245772 Feb  7 16:00 composer.lock
drwxr-xr-x 6 0 0   4096 Feb  7 16:00 data
drwxr-xr-x 4 0 0   4096 Feb  7 16:00 docker
drwxr-xr-x 5 0 0   4096 Feb  7 16:00 documentation
drwxr-xr-x 3 0 0   4096 Feb  7 16:00 src
drwxr-xr-x 5 0 0   4096 Feb  7 16:00 theme
drwxr-xr-x 4 0 0   4096 Feb  7 16:00 web
root@docker:~/PartKeepr/docker/development#

On the host, where did you install/download the PK repo (just for sake of writing correct paths here)? I do a git clone in home directory of root user

What user is the owner (on the host) of the files (aka who has run git clone)? root

What UID is that of the user? 0 (root)

What is configured in the docker-compose.yml for GITHUB_DEBUG_UID? the default one, i did not touch docker-compose.yml. GITHUB_DEBUG_UID: 1000

[christianlupus]

OK, I see. The problem is that the user 1000 in the container tries to install the instance but as the files belong to root (id 0) this is not allowed.

I suggest to create a new user on the host machine with useradd -mU partkeepr then log in as the named user (or do sudo -iu partkeepr) and use that user to clone the repository. So it will be something like this

# useradd -mU partkeepr
# sudo -iu partkeepr
$ git clone https://github.com/partkeepr/PartKeepr.git
$ cd PartKeepr/docker/development
$ touch github.env
$ exit
# cd PasrtKeepr/docker/development
# id -u partkeepr

The id -u will return the uid of the partkeepr user.

In the next step you have to replace it twice in the docker-compose.yml file where GITHUB_DEBUG_UID is. Continue with the steps as in the tutorial waiting for the database.

# nano docker-compose.yml
# docker-compose up -d
....

Make sure the other docker containers are no longer running as the ports might interfere (docker-compose down).

[marcolino7]

Hi, following your suggestion I did more steps ahead. Container si started, and I finished setup Wizard. I choose "Basic Authentication" as suggested, but as soon I load Partkeepr i got this:

Partkeepr do not ask me for username and password, but show me an Username/Password Invalid error.

[christianlupus]

OK, this seems to have installed the PartKeepr successfully (at least internally). Can you please try with a private browser window? You could check in the browser web development tools if the browser did send an authentication header.

[marcolino7]

Hi,tried from Private Browsing session same result. Attached HAR file from Chrome development console

Partkeeper.zip

[christianlupus]

Hello. There is the default login with admin:admin enabled: See here. Just go in your git repository to app/config and open the file parameters.php. Look close to the end for the line

$container->setParameter('partkeepr.frontend.auto_login.enabled', true);

and replace the true by false. That should do the trick.

[marcolino7]

Hi, Now I can Login, many Thanks. Last part missing. Where I have to add Octopart API Key? I added to docker-compose.yml in this way:

services:
    app:
        image: partkeepr/development:latest

        # Use this instead if you want to develop the docker infrastructure
        #build:
            #context: ./app
            #args:
                #SRC_IMAGE: partkeepr/base-dev:latest

        ports:
            - 8082:80
        volumes:
            - "../..:/var/www/pk"
        env_file:
            - ./github.env
        environment:
            GITHUB_DEBUG_UID: 9002
            PARTKEEPR_OCTOPART_APIKEY: xxxxxxxxx-xxxxxxxxxxxx-xxxxxxxxx-xxxxxxxx
            #ADD_PHPINFO_FILE: 1
            #PARTKEEPR_FORCE_UPDATE: "yes"

but octopart integrazione does not work:

Should I change parameters.php? Can I pass as parameter to container?

Thanks

[christianlupus]

Should I change parameters.php?

Yes, you need to change that file manually. It will not be regenerated automatically. This is due to the fact that the docker container is intended for development.

[marcolino7]

Hi @christianlupus I set

$container->setParameter('partkeepr.octopart.apikey', 'xxxxxxxxxx-2011-xxxxx-xxxxxxxx-xxxxxxx');

into parameters.php, then

docker-compose stop app
docker-compore rm app
docker-compore up -d app

but message that Octopart is not configured, still persist. Something else to do? Thanks

Marco

[christianlupus]

I think this should work. Can you please have a look at the logs (docker-compose logs app) and see if there is something visible?

I suggest you prepare everything till the point where the API call for importing needs to be done, start a continuous logging session (docker-compose logs --tail 10 -f app), and then try to start the import. That way you see relatively clearly all the messages that have just arrived.

[marcolino7]

Nothing in log when I click on Octopart Button.

[christianlupus]

Not even a short message that the remote access has happened? I will have to test on my instance but I know there was a message logged once when Octopart was queried.

[marcolino7]

Hi, this night I removed all, clone again repository, reconfigure all, and now Octopart Work Fine. I connect Development version to my old 1.4.0 Database and all works fine. I am still using this verion to add all my Inventory and for now I have no issue at all.

Kindly regards and thanks for help.

Marco

PS only one issue. Export in Excel work, but Excel is not able to open generated file. I have to use CSV

[christianlupus]

OK, as the main issue (dockerized setup) is working in general, I will close this issue here.

Your last issue might be related to #646. If not, you should open a new issue to keep things clean.