Problems with launching a LOADed pycv from the python interface

[Iximiel]

This possibly means that, due to these GIL issues, also "run pycv from plumed in python" is not working? I though @Iximiel was testing this at least locally, is this correct? Otherwise I think we need a test

I tried, and I think I need to find a solution to this:

what():  Can't add new modules after the interpreter has been initialized

Originally posted by @Iximiel in https://github.com/plumed/plumed2/issues/1129#issuecomment-2406935974

[Iximiel]

@GiovanniBussi I think I found a (duct taped) solution, that looks like this:

cmake_minimum_required(VERSION 3.20)
project(PYCV VERSION 0.1.0 LANGUAGES CXX)
set (CMAKE_CXX_STANDARD 17)

find_package(Python COMPONENTS Interpreter Development)
find_package(pybind11 CONFIG)
message(STATUS "pybind11 found: ${pybind11_VERSION}")

exec_program(plumed
ARGS info --include-dir
OUTPUT_VARIABLE PLUMED_INCLUDE_DIR
)
message(STATUS "Plumed include dir: ${PLUMED_INCLUDE_DIR}")

pybind11_add_module(plumedCommunications PlumedPythonEmbeddedModule.cpp)
add_library(PythonCVInterface SHARED ActionWithPython.cpp  PythonCVInterface.cpp  PythonFunction.cpp)
target_link_libraries(PythonCVInterface PUBLIC pybind11::embed plumedKernel)
target_link_libraries(plumedCommunications PUBLIC plumedKernel PythonCVInterface)

target_include_directories(PythonCVInterface PUBLIC ${PLUMED_INCLUDE_DIR}/plumed)
target_include_directories(plumedCommunications PUBLIC ${PLUMED_INCLUDE_DIR}/plumed)

This basically gives you the readied Python module and the so with the actions.

Since as is now a proof of concept, you have to:

• put the shared object module (named like plumedCommunications.cpython-39-x86_64-linux-gnu.so) into your python modules directory
• have the shared object libPythonCVInterface.so with the action available to the python module when you are calling it with plmd.cmd("readInputLine","LOAD FILE=./libPythonCVInterface.so") followed by plmd.cmd("readInputLine","cvPY: PYCVINTERFACE IMPORT=pycv").

I want to understand how to make this a wheel, that may solve A LOT of problems, maybe making pyCV a package with https://github.com/pybind/scikit_build_example

and on top of that solve the puzzle of having both the action.so available to be LOADed by plumed and to be LD_LINKED to the module