Closed drmoose closed 6 months ago
Hello,
Thank you for taking the time to write a repro.
In your case, you can use an amalgamation utility I wrote. It is designed to not include twice the same file, so that it should do what you expect.
Here is a commented example:
from codemanip.amalgamated_header import AmalgamationOptions, write_amalgamate_header_file
amalgamation_options = AmalgamationOptions()
# Base dir for the include paths
amalgamation_options.base_dir = '.'
# All possible subdirs for includes
amalgamation_options.include_subdirs = ["./"]
# Start of the local includes (so that we do not include <string> for example)
# In your example, I had to add "./" to the include path, so that it is handled by local_includes_startwith
amalgamation_options.local_includes_startwith = './'
# You wlll need a main header that includes all the API you want to expose
amalgamation_options.main_header_file = 'dog.hpp'
# Destination file for the amalgamated header
amalgamation_options.dst_amalgamated_header_file = 'amalgamated_header.hpp'
# Let's write the amalgamated header
write_amalgamate_header_file(amalgamation_options)
Beware, I had to change one line in your example:
with open('dog.hpp', 'w') as fd:
if TRIGGER_THE_BUG:
fd.write('#include "./animal.hpp"\n') # I had to add "./" here, so that it is handled by local_includes_startwith
else:
fd.write(ANIMAL_CLASS)
fd.write(DOG_SUBCLASS)
And if you want the full working code:
from litgen import LitgenOptions, write_generated_code_for_file
# litgen_error.py
TRIGGER_THE_BUG = True
TRIGGER_PURE_VIRTUAL_COMPILER_ERROR = False
ANIMAL_CLASS = """
#include <string>
class Animal {
public:
virtual std::string go(int n_times) = 0;
virtual std::string name() { return "unknown"; }
virtual bool is_furry()%s
};
""" % (
"=0;" if TRIGGER_PURE_VIRTUAL_COMPILER_ERROR else '{return true;}'
)
DOG_SUBCLASS = """
class Dog : public Animal {
public:
std::string go(int n_times) override {
std::string result;
for (int i=0; i<n_times; ++i)
result += bark() + " ";
return result;
}
virtual std::string bark() { return "woof!"; }
};
"""
CPP_OUTPUT_STUB = """
#include "dog.hpp"
#include <pybind11/pybind11.h>
namespace py = pybind11;
// <litgen_glue_code>
// </litgen_glue_code>
void py_init_module_test_output(py::module& m) {
// <litgen_pydef>
// </litgen_pydef>
}
PYBIND11_MODULE(test_output, m) { py_init_module_test_output(m); }
"""
PYI_OUTPUT_STUB = """
# <litgen_stub>
# </litgen_stub>
"""
HEADER_FILES = ['dog.hpp']
if TRIGGER_THE_BUG:
HEADER_FILES.append('animal.hpp')
with open('animal.hpp', 'w') as fd:
fd.write(ANIMAL_CLASS)
with open('dog.hpp', 'w') as fd:
if TRIGGER_THE_BUG:
fd.write('#include "./animal.hpp"\n') # I had to add "./" here, so that it is handled by local_includes_startwith
else:
fd.write(ANIMAL_CLASS)
fd.write(DOG_SUBCLASS)
with open('test_output.cpp', 'w') as fd:
fd.write(CPP_OUTPUT_STUB)
with open('test_output.pyi', 'w') as fd:
fd.write(PYI_OUTPUT_STUB)
HEADER_FILES = ['animal.hpp', 'dog.hpp']
#
# Using the amalgamated_header.py module from codemanip (provided by litgen)
#
from codemanip.amalgamated_header import AmalgamationOptions, write_amalgamate_header_file
amalgamation_options = AmalgamationOptions()
# Base dir for the include paths
amalgamation_options.base_dir = '.'
# All possible subdirs for includes
amalgamation_options.include_subdirs = ["./"]
# Start of the local includes (so that we do not include <string> for example)
# In your example, I had to add "./" to the include path, so that it is handled by local_includes_startwith
amalgamation_options.local_includes_startwith = './'
# You wlll need a main header that includes all the API you want to expose
amalgamation_options.main_header_file = 'dog.hpp'
# Destination file for the amalgamated header
amalgamation_options.dst_amalgamated_header_file = 'amalgamated_header.hpp'
# Let's write the amalgamated header
write_amalgamate_header_file(amalgamation_options)
conf = LitgenOptions()
conf.class_override_virtual_methods_in_python__regex = '^Animal$|^Dog$'
write_generated_code_for_file(
conf,
#input_cpp_header_files=HEADER_FILES,
input_cpp_header_file='amalgamated_header.hpp',
output_cpp_pydef_file='test_output.cpp',
output_stub_pyi_file='test_output.pyi',
)
Ah, thank you. I'd found a workaround parsing free functions and class members in separate passes (with parent classes inlined for the class members), but yours is better.
Could write_amalgamate_header_file
be added to the online docs? I see a reference to it in the DasLib
example but I didn't see an explanation of why that approach needed to be chosen over passing a list of files to write_generated_code_for_files
.
When litgen reads a subclass that's defined in the same file as its base class, it will correctly generate bindings for all methods defined on the base class. If you move the base class to a separate header file, though, it does not.
If the base class includes a pure-virtual method which the subclass does not override, this generates an error at compile time because the generated trampoline class is also missing the PYBIND_OVERRIDE_PURE for that method.
Unfortunately, using the
code_preprocess_function
to replace#include
s with their contents does not appear to be a viable workaround, as it results in multiple definitions for the base class.Reproduction
Here's a python script that writes, generates, and tries to compile a minimally-reproducing example, adapted from the
Dog : Animal
inheritance example in the docs.Setting
TRIGGER_BUG
toTrue
in the above code putsclass Animal
in its own file, which results inclass Dog
not having bindings for::name()
or::is_fuzzy()
. SettingTRIGGER_BUG
toFalse
generates the expected litgen code.