Make it compatible with "manual scans".

[jlegrand62]

It should be possible to use the "pipeline" to reconstruct manually acquired pictures.

[jlegrand62]

We use the example dataset Chirsuta_wildplant1 on AWS in database_fabfab. It consists of a set of 55 manually acquired pictures (with a smartphone).

Start by making a copy to work with

export ROMI_DB='/home/scanner/database_jlegrand'
cp database_fabfab/Chirsuta_wildplant1 $ROMI_DB

First recreate a dataset compatible with romidata

Recreate the images directory

cd $ROMI_DB/Chirsuta_wildplant1
mkdir images
mv *.jpg images/.

Add some metadata

touch metadata.json

Add the following to metadata.json:

{
    "object": {
        "age": "N/A",
        "culture": "N/A",
        "environment": "outdoor",
        "experiment_id": "romi demo outdoor plant",
        "object": "plant",
        "plant_id": "Chirsuta_1",
        "sample": "whole plant",
        "species": "Cardamine hirsuta",
        "stock": "WT",
        "treatment": "none"
    }
}

Start the docker container

As the next steps require romi softwares, you need to start a docker container named romiscan

docker run --runtime=nvidia --gpus all --env PYOPENCL_CTX='0' -v $ROMI_DB:/home/scanner/db -v /home/scanner/configs:/home/scanner/configs -it roboticsmicrofarms/romiscan:demo bash

Then use the import tool

romi_import_folder $ROMI_DB/Chirsuta_wildplant1 /data/romi/db/outdoor_plant_1 --metadata my_plant/metadata.json

NOTE: there is no images directory in the metadata directory. It is supposed to contain JSON files associated to the pictures...

Finally start the reconstruction

Let's not be too ambitious and stat with the task Colmap:

romi_run_task Colmap db/chirsuta_wt1/ --config configs/original_pipe_0.toml

I get some error due to the bounding_box being required in the metadata:

INFO     [luigi] logging configured via *.conf file from fileConfig
INFO     [luigi-interface] Informed scheduler that task   Colmap_True____feature_extrac_3bbfcb1413   has status   PENDING from fileConfig
INFO     [luigi-interface] Informed scheduler that task   images   has status   DONE from fileConfig
INFO     [luigi-interface] Done scheduling tasks from fileConfig
INFO     [luigi-interface] Running Worker with 1 processes from fileConfig
INFO     [luigi-interface] [pid 18] Worker Worker(salt=314860193, workers=1, host=455e1903d52a, username=scanner, pid=18) running   Colmap(output_file_id=out, scan_id=, upstream_task=ImagesFilesetExists, matcher=exhaustive, compute_dense=False, cli_args={"feature_extractor": {"--ImageReader.single_camera": "1", "--SiftExtraction.use_gpu": "1"}, "exhaustive_matcher": {"--SiftMatching.use_gpu": "1"}, "model_aligner": {"--robust_alignment_max_error": "10"}}, align_pcd=True, calibration_scan_id=) from fileConfig
ERROR    [luigi-interface] [pid 18] Worker Worker(salt=314860193, workers=1, host=455e1903d52a, username=scanner, pid=18) failed    Colmap(output_file_id=out, scan_id=, upstream_task=ImagesFilesetExists, matcher=exhaustive, compute_dense=False, cli_args={"feature_extractor": {"--ImageReader.single_camera": "1", "--SiftExtraction.use_gpu": "1"}, "exhaustive_matcher": {"--SiftMatching.use_gpu": "1"}, "model_aligner": {"--robust_alignment_max_error": "10"}}, align_pcd=True, calibration_scan_id=) from fileConfig
Traceback (most recent call last):
  File "/home/scanner/.local/lib/python3.7/site-packages/luigi/worker.py", line 191, in run
    new_deps = self._run_get_new_deps()
  File "/home/scanner/.local/lib/python3.7/site-packages/luigi/worker.py", line 133, in _run_get_new_deps
    task_gen = self.task.run()
  File "/home/scanner/romiscan/romiscan/tasks/colmap.py", line 66, in run
    bounding_box = self.find_bounding_box()
  File "/home/scanner/romiscan/romiscan/tasks/colmap.py", line 58, in find_bounding_box
    "Cannot find suitable bounding box for object in metadata")
OSError: Cannot find suitable bounding box for object in metadata

[jlegrand62]

Try to hack it by adding a workspace entry in the metadata.json:

    "workspace": {
        "x": [
            100,
            700
        ],
        "y": [
            100,
            700
        ],
        "z": [
            -207,
            110
        ]
    }

This get Colmap to stop complaining!

[jlegrand62]

But, as it could be expected, this also generates an error:

INFO     [luigi] logging configured via *.conf file from fileConfig
INFO     [luigi-interface] Informed scheduler that task   Colmap_True____feature_extrac_3bbfcb1413   has status   PENDING from fileConfig
INFO     [luigi-interface] Informed scheduler that task   images   has status   DONE from fileConfig
INFO     [luigi-interface] Done scheduling tasks from fileConfig
INFO     [luigi-interface] Running Worker with 1 processes from fileConfig
INFO     [luigi-interface] [pid 30] Worker Worker(salt=646446456, workers=1, host=455e1903d52a, username=scanner, pid=30) running   Colmap(output_file_id=out, scan_id=, upstream_task=ImagesFilesetExists, matcher=exhaustive, compute_dense=False, cli_args={"feature_extractor": {"--ImageReader.single_camera": "1", "--SiftExtraction.use_gpu": "1"}, "exhaustive_matcher": {"--SiftMatching.use_gpu": "1"}, "model_aligner": {"--robust_alignment_max_error": "10"}}, align_pcd=True, calibration_scan_id=) from fileConfig
ERROR    [luigi-interface] [pid 30] Worker Worker(salt=646446456, workers=1, host=455e1903d52a, username=scanner, pid=30) failed    Colmap(output_file_id=out, scan_id=, upstream_task=ImagesFilesetExists, matcher=exhaustive, compute_dense=False, cli_args={"feature_extractor": {"--ImageReader.single_camera": "1", "--SiftExtraction.use_gpu": "1"}, "exhaustive_matcher": {"--SiftMatching.use_gpu": "1"}, "model_aligner": {"--robust_alignment_max_error": "10"}}, align_pcd=True, calibration_scan_id=) from fileConfig
Traceback (most recent call last):
  File "/home/scanner/.local/lib/python3.7/site-packages/luigi/worker.py", line 191, in run
    new_deps = self._run_get_new_deps()
  File "/home/scanner/.local/lib/python3.7/site-packages/luigi/worker.py", line 133, in _run_get_new_deps
    task_gen = self.task.run()
  File "/home/scanner/romiscan/romiscan/tasks/colmap.py", line 117, in run
    points, images, cameras, sparse, dense, bounding_box = colmap_runner.run()
  File "/home/scanner/romiscan/romiscan/colmap.py", line 423, in run
    raise Exception(msg)
Exception: Empty sparse point cloud!ø
            The bounding box is probably wrong, check workspace in metadata.

No kidding... I love that kind of message.

[jlegrand62]

Maybe, if the original author took the time to describe what the "bounding box" is used for and how to parametrize it, I would have some idea of what to do next...