NLOHMANN_DEFINE_TYPE_INTRUSIVE doesn't work with "json_fwd.hpp"

[AndreyOrb]

Description

I wrote a code that uses json.hpp. My class adds relevant conversions with a macro NLOHMANN_DEFINE_TYPE_INTRUSIVE.

class Rect
{
public:
    Rect() = default;

    float _x1{};

    NLOHMANN_DEFINE_TYPE_INTRUSIVE(Rect, _x1)
};

I then replaced json.hpp with json_fwd.hpp to allow forward declaration of nlohmann::json. Unfortunately, json_fwd.hpp doesn't contain NLOHMANN_DEFINE_TYPE_INTRUSIVE macro. I tried to overcome the issue by copying NLOHMANN_DEFINE_TYPE_INTRUSIVE to json_fwd.hpp file, but compilation still fails with errors:

error C2676: binary '[': 'nlohmann::json_abi_v3_11_2::json' does not define this operator or a conversion to a type acceptable to the predefined operator (compiling source file Rect.cpp)

error C2027: use of undefined type 'nlohmann::json_abi_v3_11_2::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_11_2::adl_serializer,std::vector<uint8_t,std::allocator>>' (compiling source file Rect.cpp)

If I understand right, basic_json class must declare that it has .at / operator[]

Reproduction steps

include "json_fwd.hpp"

class Rect
{
public:
    Rect() = default;

    float _x1{};

    NLOHMANN_DEFINE_TYPE_INTRUSIVE(Rect, _x1)
};

Expected vs. actual results

Expected: Compilation succeeds with NLOHMANN_DEFINE_TYPE_INTRUSIVE and json_fwd.hpp Actual: Exceptions thrown:

error C2676: binary '[': 'nlohmann::json_abi_v3_11_2::json' does not define this operator or a conversion to a type acceptable to the predefined operator (compiling source file Rect.cpp)

error C2027: use of undefined type 'nlohmann::json_abi_v3_11_2::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_11_2::adl_serializer,std::vector<uint8_t,std::allocator>>' (compiling source file Rect.cpp)

Minimal code example

No response

Error messages

No response

Compiler and operating system

Windows 10 / Microsoft Visual Studio Professional 2019

Library version

3.11.2

Validation

• [ ] The bug also occurs if the latest version from the develop branch is used.
• [ ] I can successfully compile and run the unit tests.

[AndreyOrb]

This is the grabbed macro that I used: json_addon.h

[gregmarr]

If I understand right, basic_json class must declare that it has .at / operator[]

That's correct, you must have the full class definition and not just the forward declaration to use the NLOHMANN_DEFINE_TYPE_INTRUSIVE macro.

[AndreyOrb]

Some background story: I use json.hpp to jsonify a class (Rect, in the example above). But there are also other classes, that contain STLs. Huge vectors, for example. When it comes to converting such classes, it is extremely slow. So I though to mark json.hpp for full optimization. But it will not work, because it is included in Rect's header file. I then replaced json.hpp with json_fwd.hpp, and included the json.hpp in Rect's .cpp.

But now there's a problem, because NLOHMANN_DEFINE_TYPE_INTRUSIVE macros fail.

I see several options from here: 1) Some workaround for adding NLOHMANN_DEFINE_TYPE_INTRUSIVE in json_fwd.hpp Then there's a problem with operator[] 2) Move NLOHMANN_DEFINE_TYPE_INTRUSIVE into .cpp Then there's a problem, because the Rect is declared in .h 3) Switching to NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE Then there's a problem, because Rect's float _x1 is actually a protected field. There are many protected fields in other classes as well. I will have to break encapsulation.

[gregmarr]

You should be able to do something like this in your class definition with json_fwd:

friend void to_json(nlohmann::json& nlohmann_json_j, const Rect& t);
friend void from_json(const nlohmann::json& nlohmann_json_j, Rect& t);

and then in your .cpp:

void to_json(nlohmann::json& nlohmann_json_j, const Rect& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, _x1)) }

void from_json(const nlohmann::json& nlohmann_json_j, Rect& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, _x1)) }

If that works, it could potentially be added as a pair of macros like this:

#define NLOHMANN_DECLARE_TYPE_INTRUSIVE(Type)  \
    friend void to_json(nlohmann::json& nlohmann_json_j, const Type& nlohmann_json_t); \
    friend void from_json(const nlohmann::json& nlohmann_json_j, Type& nlohmann_json_t)

#define NLOHMANN_IMPLEMENT_TYPE_INTRUSIVE(Type, ...)  \
    void to_json(nlohmann::json& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) } \
    void from_json(const nlohmann::json& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) }

[AndreyOrb]

Your approach actually worked out! The only problem that a compiler complains now is a template class.

After adding template to Rect class, I get exception: Argument list for class template "Rect" is missing.

template <class T>
class NnRect
{

I expended the macro and got:

void to_json(nlohmann::json& nlohmann_json_j, const NnRect& nlohmann_json_t)
{
    nlohmann_json_j["_x1"] = nlohmann_json_t._x1;
}

Adding template part would fix it:

template <class T>
void to_json(nlohmann::json& nlohmann_json_j, const NnRect<T>& nlohmann_json_t)
{
    nlohmann_json_j["_x1"] = nlohmann_json_t._x1;
}

Is there a way to edit NLOHMANN_IMPLEMENT_TYPE_INTRUSIVE macro to include template parameters?

[gregmarr]

I'm not sure that could be done in general. It could to done as a separate macro for a fixed number of template arguments, since it already has a variable argument list.

#define NLOHMANN_IMPLEMENT_TYPE_INTRUSIVE_TEMPLATE(Type, ...)  \
    template<typename NLOHMANN_TEMPLATE_TYPE> void to_json(nlohmann::json& nlohmann_json_j, const Type<NLOHMANN_TEMPLATE_TYPE>& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) } \
    template<typename NLOHMANN_TEMPLATE_TYPE> void from_json(const nlohmann::json& nlohmann_json_j, Type<NLOHMANN_TEMPLATE_TYPE>& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) }

[AndreyOrb]

Well, it worked with forward declaration + explicit instantiation.

I will add here some adjustments that I did:

json_fwd_addon.h:

pragma once

#include "json_fwd.h"

#define NLOHMANN_DECLARE_TYPE_INTRUSIVE(Type, ...)  \
    friend void to_json(nlohmann::json& nlohmann_json_j, const Type& nlohmann_json_t); \
    friend void from_json(const nlohmann::json& nlohmann_json_j, Type& nlohmann_json_t);

#define NLOHMANN_DECLARE_TYPE_INTRUSIVE_TEMPLATE(Type, ...)  \
    template<class T> friend void to_json(nlohmann::json& nlohmann_json_j, const Type<T>& nlohmann_json_t); \
    template<class T> friend void from_json(const nlohmann::json& nlohmann_json_j, Type<T>& nlohmann_json_t);

#define NLOHMANN_DECLARE_TYPE_INTRUSIVE_VIRTUAL(Type, ...)  \
    virtual void to_json_v(nlohmann::json& nlohmann_json_j) const; \
    NLOHMANN_DECLARE_TYPE_INTRUSIVE(Type)

json_addon.cpp:

include "json.hpp"

#define NLOHMANN_IMPLEMENT_TYPE_INTRUSIVE(Type, ...)  \
    void to_json(nlohmann::json& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) } \
    void from_json(const nlohmann::json& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) }

#define NLOHMANN_IMPLEMENT_TYPE_INTRUSIVE_TEMPLATE(Type, ...)  \
    template<class T> void to_json(nlohmann::json& nlohmann_json_j, const Type<T>& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) } \
    template<class T> void from_json(const nlohmann::json& nlohmann_json_j, Type<T>& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) }

#define NLOHMANN_IMPLEMENT_TYPE_INTRUSIVE_VIRTUAL(Type, ...)  \
    void Type::to_json_v(nlohmann::json& nlohmann_json_j) const { Type nlohmann_json_t = *this; NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) } \
    void to_json(nlohmann::json& nlohmann_json_j, const Type& nlohmann_json_t) { nlohmann_json_t.to_json_v(nlohmann_json_j); } \
    void from_json(const nlohmann::json& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) }

I could not use json_obj.get(), but from_json worked just fine.