OpenDroneMap / NodeODM

A lightweight REST API to access aerial image processing engines such as ODM or MicMac
https://github.com/OpenDroneMap/NodeODM/blob/master/docs/index.adoc
GNU Affero General Public License v3.0
207 stars 150 forks source link

Invalid configuration file config-default.json error #223

Closed sbonaime closed 2 weeks ago

sbonaime commented 3 weeks ago

On an HPC system I want to start on nodeODM with singularity in a start.slurm file

#!/usr/bin/bash
#source .bashrc

#SBATCH --partition=ncpushort
#SBATCH --job-name ODM-CLUSTER
#SBATCH --ntasks-per-node=10
#SATCH --mem=12G
#SBATCH --mail-type=END
#SBATCH --mail-type=FAIL
#SBATCH --output /gpfs/users/bonaime/logs/slurm-openmp-%j.out

cd $HOME
cd git/NodeODM/

#singularity caches
SINGULARITY_CACHEDIR=/gpfs/scratch/bonaime/tmpfs
SREGISTRY_TMPDIR=/gpfs/scratch/bonaime/tmpfs
SINGULARITY_WORKDIR=/gpfs/scratch/bonaime/tmpfs

srun --nodes=1 singularity run  -H /gpfs/scratch/bonaime  --writable-tmpfs node

I run this script with

sbatch  --parsable ipgp.slurm

and here is the log


Invalid configuration file config-default.json
Invalid configuration file config-default.json
Invalid configuration file config-default.json
Invalid configuration file config-default.json
Invalid configuration file config-default.json
Invalid configuration file config-default.json
Invalid configuration file config-default.json
Invalid configuration file config-default.json
srun: error: ncpu004: tasks 1-4,7: Exited with exit code 1
Invalid configuration file config-default.json
Invalid configuration file config-default.json
srun: error: ncpu004: tasks 0,5-6,8-9: Exited with exit code 1

I did not modify the config-default.json is

{
        "instance": "node-OpenDroneMap",
        "odm_path": "/gpfs/users/bonaime/git/ODM/",

        "logger": {
                "level": "info",
                "maxFileSize": 104857600,
                "maxFiles": 10,
                "logDirectory": ""
        },

        "port": "auto",
        "deamon": false,
        "parallelQueueProcessing": 1,
        "cleanupTasksAfter": 2880,
        "test": false,
        "testSkipOrthophotos": false,
        "testSkipDems": false,
        "token": "",
        "authorizedIps": [],
        "maxImages": ""
}

I got the nodeodm and clusterodm with singularity pull --force --disable-cache docker://opendronemap/nodeodm:latest and singularity pull --force --disable-cache docker://opendronemap/cluster:latest I then rename nodeodm to node I think it is a different problem as #33

sbonaime commented 2 weeks ago

I have the same error if I remove the config-default.json file. So my problem is that I don't know how to give to singularity the config-default.json path

sbonaime commented 2 weeks ago

If I bind git/nodeODM to /var/www , I have another error :


singularity  run --compat --writable-tmpfs   -B $PWD:/var/www    node
internal/modules/cjs/loader.js:905
  throw err;
  ^

Error: Cannot find module 'minimist'
Require stack:
- /var/www/config.js
- /var/www/index.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:101:18)
    at Object.<anonymous> (/var/www/config.js:21:12)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Module.require (internal/modules/cjs/loader.js:974:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/var/www/config.js', '/var/www/index.js' ]
}
sbonaime commented 2 weeks ago

I found a solution I don't understand but it is working ! I just made a shell connect to the node image

singularity shell --bind $PWD:/var/www  nodeodm_latest.sif
Singularity> cd /var/www
npm install --production
exit

and then I am able to run it !

singularity run --bind $PWD:/var/www  nodeodm_latest.sif

I will make another tutorial for clusterODM , NodeODM later, HPC and singularity