Make USD built with python and without python C++ ABI compatible

[ARFoisy]

Is your feature request related to a problem? Please describe. To run USD in specialized processing environments, we would like a minimal USD runtime that has a lighter footprint.  In particular, the minimal USD runtime is compiled with option “—no-python”.

At the same time, the code compiled and linked against minimal USD should be able to load and run against a full USD runtime that has been compiled with option “—python”, for example when the environment is a DCC.

When building USD with “—python” and “—no-python”, we would like that the resulting libraries be ABI compatible with respect to the non-python C++ related code.

Describe the solution you'd like We will be working on this issue and submitting a PR that will attempt to regularize vtables, member definitions, and other ABI issues between builds with “—python” and “—no-python” options.

In particular, code guarded by #ifdef PXR_PYTHON_SUPPORT_ENABLED that affects address alignment will be exposed to both “—python” and “—no-python” options and the resulting code definitions will be adjusted according the active option.

The above exposes some python related classes when compiling with “—no-python”. The affected classes will have two implementations with static_assert safeguards to check for size and alignment. For example (some includes and comments have been removed to show the pattern):

#ifdef PXR_PYTHON_SUPPORT_ENABLED

PXR_NAMESPACE_OPEN_SCOPE

// We define the empty stub for ABI compatibility even if Python support is 
// enabled so we can make sure size and alignment is the same.
class TfPyObjWrapperStub
{
public:
    static constexpr std::size_t Size = 16;
    static constexpr std::size_t Align = 8;

private:
    std::aligned_storage<Size, Align>::type _stub;
};

/// \class TfPyObjWrapper
///
/// Boost Python object wrapper.
/// …
#ifdef PXR_PYTHON_SUPPORT_ENABLED
class TfPyObjWrapper
    : public boost::python::api::object_operators<TfPyObjWrapper>
{
    typedef boost::python::object object;

public:

    TF_API TfPyObjWrapper();

    TF_API TfPyObjWrapper(object obj);

    object const &Get() const {
        return *_objectPtr;
    }

    TF_API PyObject *ptr() const;

    friend inline size_t hash_value(TfPyObjWrapper const &o) {
        return (size_t) o.ptr();
    }

    TF_API bool operator==(TfPyObjWrapper const &other) const;

    TF_API bool operator!=(TfPyObjWrapper const &other) const;

private:

    // Befriend object_operators to allow it access to implicit conversion to
    // boost::python::object.
    friend class boost::python::api::object_operators<TfPyObjWrapper>;
    operator object const &() const {
        return Get();
    }

    // Store a shared_ptr to a python object.
    std::shared_ptr<object> _objectPtr;
};

static_assert(sizeof(TfPyObjWrapper) == sizeof(TfPyObjWrapperStub),
              "ABI break: Incompatible class sizes.");
static_assert(alignof(TfPyObjWrapper) == alignof(TfPyObjWrapperStub),
              "ABI break: Incompatible class alignments.");

#else // PXR_PYTHON_SUPPORT_ENABLED

class TfPyObjWrapper : TfPyObjWrapperStub
{
};

#endif // PXR_PYTHON_SUPPORT_ENABLED

Describe alternatives you've considered We considered fully isolating the python implementation by having zero python types in the .h files. Hiding the python-aware code is more involved. For example, some template functions that use python types in low-level libraries are being instantiated with types which are only defined in downstream libraries.

The above could be “future work”. At this moment we are working to have the same address alignment of declarations with properly stubbed definitions when compiling with “—python” and “—no-python”.

Additional context Ref.: Previous discussions on USD Group: https://groups.google.com/g/usd-interest/c/BQmL9BwwlZo

[ARFoisy]

This is for the Pixar USD GitHub

[ARFoisy]

Will submit to Pixar