build-base.php file is missing as of 1.70.0, broken setup_database

[jaxjexjox]

It is entirely possible, my lack of knowledge is holding me back, but I'm unable to roll a fresh install because build base is missing.

docker exec librenms setup_database

can not be run, without this file.

Am I doing something wrong? 1.70.0 1.70.1 :latest

All the same Previous versions, it's there.

File location should be /opt/librenms/ inside the container.

[jaxjexjox]

(Note, I realise these notes are pretty tewrrible - but it's straight forward, the file is literally missing from the container upon exec and looking inside the path - the file IS there on older builds)

[jarischaefer]

Thanks for reporting this issue. Please try version 1.70.1-1 or latest.

[jaxjexjox]

Thank you! That appears to be fixed. I'm a newbie so I suspect I'll be back with some other stuff soon but this is nailed.

Thanks again.

[vistalba]

Run into the same issue as well with docker latest so I've to switch back to 1.69 and it was working again.

[jaxjexjox]

Delete your image entirely and pull it again? It worked for me last night - about 21 hours ago.

[vistalba]

No, this dosn't help for me.

Tried tags:

• 1.69 -> OK
• 1.70 -> NOK
• 1.70.1 -> NOK
• latest -> NOK

I'm using kubernetes and imagePullPolicy: Always and I also cleared docker container cache on the nodes with docker system prune.

[jaxjexjox]

I am not experienced enough to help further, I assume you've exec'd into the container and can clearly see the file is missing.

Hopefully the dev can help. Heck I can't even figure out how to get a new copy running at all on my new test machine!

Good luck.

[jarischaefer]

@vistalba Please try 1.70.1-1 or latest corresponding to the digest sha256:d6e79167a9ab2b3a58d4f2c9ba8aaa6afd04c6334fc661b68bfeb4ec6d643c40. You may see the digest when running docker pull.

docker pull jarischaefer/docker-librenms
Using default tag: latest
latest: Pulling from jarischaefer/docker-librenms
Digest: sha256:d6e79167a9ab2b3a58d4f2c9ba8aaa6afd04c6334fc661b68bfeb4ec6d643c40
Status: Image is up to date for jarischaefer/docker-librenms:latest
docker.io/jarischaefer/docker-librenms:latest

You can also use docker images and make sure the image id matches either 9a4ba5780083 or bda62cd6aee5:

docker images jarischaefer/docker-librenms
REPOSITORY                     TAG                 IMAGE ID            CREATED             SIZE
jarischaefer/docker-librenms   latest              9a4ba5780083        2 days ago          1.2GB
jarischaefer/docker-librenms   1.70.1-1            bda62cd6aee5        2 days ago          1.2GB

As for Kubernetes, I am not sure why the policy does not work. Perhaps you need to recreate the instance before it takes effect.

[vistalba]

As already said... it doesn't work with all of the 1.70 images:

~$ kubectl logs -n mgmt librenms-0 -c init-librenmsdb
Could not open input file: /opt/librenms/build-base.php

Configued image is jarischaefer/docker-librenms:1.70.1-1 and I've also tried latest, 1.70.1 and 1.70. The only image that is working at the moment is 1.69.

~$ sudo docker images jarischaefer/docker-librenms
REPOSITORY                     TAG                 IMAGE ID            CREATED             SIZE
jarischaefer/docker-librenms   1.70.1-1            bda62cd6aee5        2 days ago          1.2GB

As you can see I've deleted all the old cached images and get this new one.

Edit: The image gets new pulled with tag latest but still sticks on the same error:

Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  115s  default-scheduler  Successfully assigned mgmt/librenms-0 to k8s-node-01
  Normal  Pulling    115s  kubelet            Pulling image "jarischaefer/docker-librenms:latest"
  Normal  Pulled     1s    kubelet            Successfully pulled image "jarischaefer/docker-librenms:latest" in 1m53.780619175s
  Normal  Created    0s    kubelet            Created container init-librenmsdb

[E-t-z]

Can confirm, that file does not exist indeed:

root@LibreNMS:/# test -f /opt/build-base.php       
root@LibreNMS:/#   

root@LibreNMS:/# sudo find /opt -name build-base.php          
root@LibreNMS:/#                               

Using Tag: Latest aka version: 1.70.1-1

[vistalba]

With tag latest set:

root@librenms-0:/# test -f /opt/build-base.php
root@librenms-0:/#

root@librenms-0:/# find /opt -name build-base.php
root@librenms-0:/#

The file doen't exists on this path. I think path should be /opt/librenms/ as there are many files.. but as you can see, there is no build-base.php file:

root@librenms-0:/# ls -lh /opt/librenms/
total 1.3M
-rw-r--r--   1 librenms librenms  12K Dec  4 11:51 AUTHORS.md
-rw-r--r--   1 librenms librenms   94 Dec  4 11:51 CHANGELOG.md
-rw-r--r--   1 librenms librenms   93 Dec  4 11:51 CODE_OF_CONDUCT.md
-rw-r--r--   1 librenms librenms  170 Dec  4 11:51 CONTRIBUTING.md
-rw-r--r--   1 librenms librenms  35K Dec  4 11:51 LICENSE.txt
drwxr-xr-x  18 librenms librenms 4.0K Dec  4 11:51 LibreNMS
-rw-r--r--   1 librenms librenms  11K Dec  4 11:51 README.md
-rwxr-xr-x   1 librenms librenms 8.6K Dec  4 11:51 addhost.php
-rwxr-xr-x   1 librenms librenms  956 Dec  4 11:51 adduser.php
-rwxr-xr-x   1 librenms librenms 1.8K Dec  4 11:51 alerts.php
drwxr-xr-x  17 librenms librenms 4.0K Dec  4 11:51 app
-rwxr-xr-x   1 librenms librenms 1.7K Dec  4 11:51 artisan
-rwxr-xr-x   1 librenms librenms 6.2K Dec  4 11:51 billing-calculate.php
drwxr-xr-x   3 librenms librenms 4.0K Dec  4 11:51 bootstrap
drwxr-xr-x   2 librenms librenms 4.0K Dec  4 11:51 cache
-rwxr-xr-x   1 librenms librenms 3.2K Dec  4 11:51 check-services.php
-rw-r--r--   1 librenms librenms 4.9K Dec  4 11:52 composer.json
-rw-r--r--   1 librenms librenms 360K Dec  4 11:51 composer.lock
drwxr-xr-x   1 librenms librenms 4.0K Dec  4 11:51 conf.d
drwxr-xr-x   1 librenms librenms 4.0K Dec  4 11:51 conf.internal.d
drwxr-xr-x   2 librenms librenms 4.0K Dec  4 11:51 config
-rw-r--r--   1 librenms librenms 1.4K Dec  4 11:51 config.php
-rw-r--r--   1 librenms librenms 1.7K Dec  4 11:51 config.php.default
-rwxr-xr-x   1 librenms librenms  352 Dec  4 11:51 config_to_json.php
drwxr-xr-x   2 librenms librenms 4.0K Dec  4 11:51 contrib
-rwxr-xr-x   1 librenms librenms  880 Dec  4 11:51 cronic
-rw-r--r--   1 librenms librenms  15K Dec  4 11:51 daily.php
-rwxr-xr-x   1 librenms librenms  15K Dec  4 11:51 daily.sh
drwxr-xr-x   6 librenms librenms 4.0K Dec  4 11:51 database
-rwxr-xr-x   1 librenms librenms  517 Dec  4 11:51 delhost.php
-rwxr-xr-x   1 librenms librenms  14K Dec  4 11:51 discovery-wrapper.py
-rwxr-xr-x   1 librenms librenms 4.8K Dec  4 11:51 discovery.php
-rwxr-xr-x   1 librenms librenms 2.2K Dec  4 11:51 dist-pollers.php
drwxr-xr-x  10 librenms librenms 4.0K Dec  4 11:51 doc
drwxr-xr-x   8 librenms librenms 4.0K Dec  4 11:51 html
drwxr-xr-x   9 librenms librenms 4.0K Dec  4 11:51 includes
-rwxr-xr-x   1 librenms librenms  975 Dec  4 11:51 irc.php
-rwxr-xr-x   1 librenms librenms 1.7K Dec  4 11:51 librenms-service.py
-rw-r--r--   1 librenms librenms  490 Dec  4 11:51 librenms.cron
-rw-r--r--   1 librenms librenms  961 Dec  4 11:51 librenms.nonroot.cron
drwxr-xr-x   2 librenms librenms 4.0K Dec  4 11:51 licenses
-rwxr-xr-x   1 librenms librenms 1.8K Dec  4 11:51 lnms
drwxrwxr-x   2 librenms librenms 4.0K Dec  7 11:20 logs
drwxr-xr-x 260 librenms librenms  20K Dec  4 11:51 mibs
drwxr-xr-x   2 librenms librenms 4.0K Dec  4 11:51 misc
-rw-r--r--   1 librenms librenms 7.7K Dec  4 11:51 mkdocs.yml
-rw-r--r--   1 librenms librenms 446K Dec  4 11:51 package-lock.json
-rw-r--r--   1 librenms librenms 1.7K Dec  4 11:51 package.json
-rwxr-xr-x   1 librenms librenms 3.8K Dec  4 11:51 pbin.sh
-rw-r--r--   1 librenms librenms 1.5K Dec  4 11:51 phpunit.xml
-rwxr-xr-x   1 librenms librenms  753 Dec  4 11:51 ping.php
-rwxr-xr-x   1 librenms librenms 7.6K Dec  4 11:51 poll-billing.php
-rwxr-xr-x   1 librenms librenms  14K Dec  4 11:51 poller-wrapper.py
-rwxr-xr-x   1 librenms librenms 6.1K Dec  4 11:51 poller.php
-rwxr-xr-x   1 librenms librenms  836 Dec  4 11:51 renamehost.php
-rw-r--r--   1 librenms librenms   50 Dec  4 11:51 requirements.txt
drwxr-xr-x   6 librenms librenms 4.0K Dec  4 11:51 resources
drwxr-xr-x   2 librenms librenms 4.0K Dec  4 11:51 routes
drwxrwxr-x   2 librenms librenms 4.0K Dec  7 11:20 rrd
drwxr-xr-x   5 librenms librenms 4.0K Dec  4 11:51 scripts
-rw-r--r--   1 librenms librenms  543 Dec  4 11:51 server.php
-rwxr-xr-x   1 librenms librenms  14K Dec  4 11:51 services-wrapper.py
-rwxr-xr-x   1 librenms librenms 8.8K Dec  4 11:51 snmp-scan.py
-rw-r--r--   1 librenms librenms  880 Dec  4 11:51 snmpd.conf.example
-rwxr-xr-x   1 librenms librenms  510 Dec  4 11:51 snmptrap.php
drwxr-xr-x   2 librenms librenms  12K Dec  4 11:51 sql-schema
drwxrwxr-x   6 librenms librenms 4.0K Dec  7 11:20 storage
-rwxr-xr-x   1 librenms librenms  544 Dec  4 11:51 syslog.php
drwxr-xr-x   8 librenms librenms 4.0K Dec  4 11:51 tests
-rwxr-xr-x   1 librenms librenms 5.6K Dec  4 11:51 validate.php
drwxr-xr-x  53 librenms librenms 4.0K Dec  4 11:52 vendor
-rw-r--r--   1 librenms librenms  601 Dec  4 11:51 webpack.mix.js

Edit:

When I use tag 1.69 it looks fine:

root@librenms-0:/opt/librenms# find /opt -name build-base.php
/opt/librenms/build-base.php

Contend of build-base.php on 1.69:

root@librenms-0:/opt/librenms# cat /opt/librenms/build-base.php
#!/usr/bin/env php
<?php
/**
 * build-base.php
 *
 * Create database structure.
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 *
 * @link       http://librenms.org
 * @copyright  2017 Tony Murray
 * @author     Tony Murray <murraytony@gmail.com>
 */
if (! isset($init_modules)) {
    $opts = getopt('ldh:u:p:n:t:s:');

    $map = [
        'h' => 'DB_HOST',
        'u' => 'DB_USERNAME',
        'p' => 'DB_PASSWORD',
        'n' => 'DB_DATABASE',
        't' => 'DB_PORT',
        's' => 'DB_SOCKET',
    ];

    // set env variables
    foreach ($map as $opt => $env_name) {
        if (isset($opts[$opt])) {
            putenv("$env_name=" . $opts[$opt]);
        }
    }

    $init_modules = ['nodb', 'laravel'];
    require __DIR__ . '/includes/init.php';

    set_debug(isset($opts['d']));

    $skip_schema_lock = isset($opts['l']);
}

require __DIR__ . '/includes/sql-schema/update.php';

exit($return);

[E-t-z]

Fun part is, that it does run fine on my deployment, without it... :confused:

[jarischaefer]

Yes, build-base.php should not exist (see https://github.com/librenms/librenms/commit/8977bb50adead7b8992eadd55c64f68ba93787ff), that is fully intended.

The content of /usr/local/bin/setup_database must be

#!/bin/bash -eu

/sbin/setuser librenms php /opt/librenms/includes/sql-schema/update.php

However, if your file looks like this, then you are still using one of the broken tags (1.70, 1.70.1):

#!/bin/bash -eu

/sbin/setuser librenms php /opt/librenms/build-base.php

[E-t-z]

@jarischaefer thx, just discovered that as well, when parsing LibreNMS repository, that file indeed should not exist anymore.

[vistalba]

Ooooooooooh! Now I get the point.

I've used a init-container to configure user etc. with this command:

        command:
        - su
        - "-p"
        - "librenms"
        - "-c"
        - "cd /opt/librenms && php /opt/librenms/build-base.php && php /opt/librenms/adduser.php ${USERNAME} ${PASSWORD} 10 ${EMAIL}"

Now if this file didn't exist anymore, I should leave it away from the ini-container command as this will fail the whole thing to start :(

So I think I can just remove this part of the init-container command. Anything else I should use instead of that?

[jaxjexjox]

Tutorials seem to imply we need to run a command which calls the build-base.php file.

What is the new method of initialising the database and application?

[jaxjexjox]

Command is "docker exec librenms setup_database" - this fails and it's part of this tutorial. https://jasonloong.com/2017/synology-snmp-network-monitoring-with-librenms-docker/

Presumably, there needs to be a new tutorial / command to initialise the DB.

[jarischaefer]

The setup_database script is exactly intended for that, to provide a stable interface regardless of internal changes made in LibreNMS. Therefore, you should be able to continue using setup_database in order to migrate tables. It is simply a wrapper that calls whatever script is necessary to populate the database. In this case, LibreNMS is using Laravel's built-in system, leading to this process: setup_database wrapper => LibreNMS script => Laravel migrations

@jaxjexjox Could you provide more information regarding the failure of setup_database? You can consider it a bug if it does not work.

@vistalba Perhaps you could replace php /opt/librenms/build-base.php with a call to setup_database. If you would like, I could add another script (similar to create_admin), which allows you to specify user, password and e-mail. The script would ensure that any future changes to /opt/librenms/adduser.php would not break your init process.

[jaxjexjox]

Sorry, perhaps I'm confusing the issue and my lack of skills messing things up.

I interpreted, someone elses post as claiming build-base.php should no longer be included as part of LibreNMS, as if it's intentional?

You have manually added the file (??) in doing so, my problems are infact resolved. If it's not supposed to be included and you remove it, then I'm back to being unable to setup a new build.

edit: Sorry, it's your own post here: https://github.com/jarischaefer/docker-librenms/issues/124#issuecomment-739833420

Look, as long as I can run this command docker exec librenms setup_database and it works, I'm happy, I don't care what files it calls at the end of the day.

[jarischaefer]

Look, as long as I can run this command docker exec librenms setup_database and it works, I'm happy, I don't care what files it calls at the end of the day.

It will work with the proper docker images mentioned here. You can run docker exec librenms cat /usr/local/bin/setup_database to verify the running image.

[vistalba]

@vistalba Perhaps you could replace php /opt/librenms/build-base.php with a call to setup_database. If you would like, I could add another script (similar to create_admin), which allows you to specify user, password and e-mail. The script would ensure that any future changes to /opt/librenms/adduser.php would not break your init process.

Sounds like a good idea. But what I need is a script I can execute not just a "docker exec" command. I need a path inside the docker container.

For docker exec librenms setup_database can I just execute the script on /usr/local/bin/setup_database ?

[jarischaefer]

@vistalba See here for the new feature (create_user script). You may reference the script by its full path, e.g. /usr/local/bin/create_user or /usr/local/bin/setup_database.