Improve relations to play nicer with server APIs (e.g. Retrofit, GSON, Jackson, Moshi, ...)

[greenrobot]

If you get object data from another source (e.g. server) you can make those data objects also ObjectBox entities. One thing has to be considered: relations on ObjectBox currently rely on ToOne and ToMany types, which 3rd party libraries are not aware of.

A quick work around would probably be to walk through the object graph and replace List objects with ToMany objects.

However, we could also look into supporting plain java.util.Lists in some way. A put would have to detect the type and do some additional syncing to figure out what to do. Or maybe just plainly put all objects in the list?

[FabianTerhorst]

That would fix #95 for me.

[greenrobot]

Not 100% certain how to implement it. Somewhat hesitant to enable putting entire lists as this may "encourage" it, which just might be wrong in other scenarios. Maybe having a new marker annotation to enable it?

[FabianTerhorst]

An annotation for automatic inserting lists would be great.

[bloemy7]

Is there any news on this? Currently running into this exact same issue (of toOne's not being recognised by a 3rd party), and would love to keep on using ObjectBox without having to build costly wrappers around everything...

[Vincent-Carrier]

Seconded. Meanwhile, out of curiosity, what is the fix you guys are using? Something like order.forEach { order -> order.customer.target = order } ?

[bloemy7]

Yep, with .setTarget(order). Seems like the right way around this for the moment.

[KishoreBabuIN]

Yes please. Would like to use an ObjectBox "entity" as a model of Retrofit responses too. Using Moshi for the json adapter. Please suggest a way to use these together.

[ecramer]

Any ETA on this? Looking to move from Realm to ObjectBox but this issue is making it difficult.

[ArthurSav]

I concur in regards to having an annotation based solution @ToOne @ToMany POJOS or in this case entities should not be closely tied to a database.

Having something like:

@Many List<Stuff> stuff

Would solve many of the issues here

[indrakumarprajapat]

I have a class with ToMany object, and using retrofit as rest client. I am getting this error while loading data,

FATAL ERROR field com.company.entity.Receipt.receipt_items has type io.objectbox.relation.ToMany, got java.util.ArrayList

This is my class structure.

@Entity
public class Receipt implements Serializable {

    public ToMany<ReceiptItem> receipt_items;

    @Id
    public long id;
    public long receipt_no;
}

Issue is arraylist not mapping to ToMany object.

[greenrobot-team]

@indrakumarprajapat Is this a JSON parsing error? For example with GSON you might have to create a type adapter for ToMany. -ut

[indrakumarprajapat]

okay, I got it, but can you help me, how can I do that. i have little bit idea. but not sure how to do it exacly.

[greenrobot-team]

@indrakumarprajapat Sorry, not here. Please look in the GSON documentation or ask for example on Stack Overflow for help. -ut

[indrakumarprajapat]

Sorry for late reply, the issue is fixed by type adapter. thanks.

[ferdavs]

I don't like this solution but it works. you have to implement JsonDeserializer.

    public  class MyDeserializer<T extends FromJson> implements JsonDeserializer<T> {
        @Override
        public T deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
            try {
                T var = ((Class<T>) typeOfT).newInstance();
                var.fromJson(json.getAsJsonObject());
                return var;
            } catch (InstantiationException e) {
                e.printStackTrace();
            } catch (IllegalAccessException e) {
                e.printStackTrace();
            } catch (Throwable e) {
                e.printStackTrace();
            }

            return null;
        }
    }

create Gson object to use with Retrofit

 Gson gson= new GsonBuilder()
                .registerTypeHierarchyAdapter(FromJson.class, new MyDeserializer())
                .create();

Retrofit retrofit = new Retrofit.Builder()
                .client(client)
                .baseUrl(baseUrl)
                .addConverterFactory(GsonConverterFactory.create(gson))
                .addCallAdapterFactory(RxJava2CallAdapterFactory.create())
                .build();

FromJson interface and some example.

public interface FromJson {
    void fromJson(JsonObject jo) throws JSONException;
}

@Entity
public class Status implements FromJson  {
    public long id;
    public int code;
    public String message;

    public void fromJson(JsonObject jo) throws JSONException {
            this.code = jo.get("code").getAsInt();
            this.message = jo.get("message").getAsString();
    }
}

@Entity
public class Login implements FromJson  {
    public long id;
    public ToOne<User> user;
    public ToOne<Status> status;

    @Override
    public void fromJson(JsonObject jo) throws JSONException {
            Status s = new Status();
            s.fromJson(jo.get("status").getAsJsonObject());
            this.status.setTarget(s);

            User u = new User();
            u.fromJson(jo.get("user").getAsJsonObject());
            this.user.setTarget(u);
    }
}

Any example using TypeAdapter? example provided by @greenrobot-team is dead.

[VincentJ1989]

@indrakumarprajapat How does the TypeAdapter work? Could U show me a simple demo ? Thks :)

[markwhitaker]

I'm not sure any of the comments here directly reflect the case of simple nested objects, of the type that crop up all the time in JSON data feeds, e.g.:

{
   "name": "Acme Inc.",
   "location": {
      "longitude": 1.23456,
      "latitude": 6.54321
   }
}

Which might deserialize to...

public class Location
{
   private double longitude;
   private double latitude;
   // getters, setters etc.
}

@Entity
public class Business
{
   private String name;
   // HOW TO PERSIST THIS??
   private Location location;
}

These are a problem too, but we don't usually need an entity relationship here. Something more like Room's @Embedded annotation would be sufficient.

(I think this is more accurately covered by https://github.com/objectbox/objectbox-java/issues/217 which is marked as a duplicate of this issue.)

[ferdavs]

objectbox+Retrofit

[lgengsy]

I gave up because of this problem, and I don't think this data structure design is reasonable enough.

[greenrobot]

@lgengsy Could you provide a little bit info for us, so we can fix it?

[uiiang]

@indrakumarprajapat How does the TypeAdapter work? Could U show me a simple demo ? Thks :)

[greenrobot]

Quick poll: Do you mostly need this for List? Or is ToOne a hard requirement?

[lgengsy]

At first, I thought this project could help me deal with data caching problems quickly. I tried it. When I met toMany and toOne, I found the project was getting more and more complicated. It took me a day or two to adjust my data structure. Finally, I found it too hard to change back.

[Riabkov]

Any new solutions or workarounds? Or better look for another ORM? (Room?) Creating an adapter for each entity not really an option

[hetang]

Any solutions for ToOne or ToMany to work with Retrofit?

[MhmDSmdi]

Is there any solution for deserialize ToOne Object Using Gson Converter ? @greenrobot @greenrobot-team

[suleymanccelik]

Is there any update or any solution for working with Gson?

[wb1992321]

@greenrobot @greenrobot-team Can you summarize the specific implementation? It is best to add to the document

[Xset-s]

I have the same issue. Please, implement annotation for ToOne.

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of io.objectbox.relation.ToOne (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)

[xuzh2017]

java.lang.IllegalArgumentException: field .bean.SubscribeBean.cates has type io.objectbox.relation.ToMany, got java.util.ArrayList

@Backlink(to = "subscribeBean") private ToMany cates;

I still can't find the exactly method to resolve this problem,could you give some example?retro+objectbox

[Xset-s]

Any news??

[MhmDSmdi]

I can't fix the problem of Objectbox with GSON for deserialize ToOne Object but I'm using Moshi now and fix it by some trick. 1.you must create same model class but Instead of ToOne, declare main Class.(ObjectResponse) 2.create a JsonAdapter class and use FromJson , ToJson method for convert Json to your object 3.configure Moshi Object (add JsonAdapter class to Moshi) for more information see this: https://github.com/square/moshi#another-example

[VenkateshPrasadSV]

@greenrobot Is there any update for this issue? Manually parsing JSON is becoming difficult. I have used a workaround for saving the list but assume if we have a list inside another list, it becomes more complex to solve with the workaround. I am using GSON

[racofix]

@greenrobot @greenrobot-team How to solve the issue?

[greenrobot-team]

Recently had to work with ObjectBox and Gson. If you don't want to add temporary fields the only other solution I could come up with is to write a custom deserializer for each entity. (Edit: but the general advice is still, keep your JSON model separate from your database model to avoid having to do this at all!)

@Entity class User {
    private ToOne<Status> status;
    private ToMany<Address> addresses;
    // TODO getters/setters
}

class UserDeserializer implements JsonDeserializer<User> {
    @Override
    public User deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context)
            throws JsonParseException {
        JsonObject userJson = json.getAsJsonObject();
        User user = new User(); // ToOne and ToMany initialized here through code injected by ObjectBox plugin

        // ToOne
        JsonObject statusJson = userJson.getAsJsonObject("status");
        Status status = context.deserialize(statusJson, Status.class);
        user.getStatus().setTarget(status);

        // ToMany
        JsonArray addressesJson = userJson.getAsJsonArray("addresses");
        Type addressListType = new TypeToken<ArrayList<Address>>(){}.getType();
        ArrayList<Address> addresses = context.deserialize(addressesJson, addressListType);
        user.getAddresses().addAll(addresses);

        // TODO parse other properties

        return user;
    }
}

-Uwe

[kibotu]

are there any plans on having auto generated type adapters or any reducing boilerplate implementations in the future?

[ArthurSav]

One drop of oil can ruin a tank full of clean water.

Revisited ObjectBox after 2 years in hopes of seeing some improvements but i see this issue (among others) still persists.

I'm afraid issues like this is a deal breaker for many mobile developers that care about quality code. It's not just the boilerplate code to deal with ToOne ToMany, it's also that my app has to be aware of unnecessary business logic of a 3rd party lib in many places.

Maybe in another 2 years :)

[francispinsoro]

I develop this solution, based on this post. http://www.jhr8.com/p/574396eb1412?utm_campaign=maleskine&utm_content=note&utm_medium=seo_notes&utm_source=recommendation

I assing to gson a deserializer of a invented class 'JsonBox', and all entities extends from that class, and i change the statement 'ToMany', through the interface 'List'.

This is the deserializer:

public class JsonBoxDeserializer<T extends JsonBox> implements JsonDeserializer<T> {

    private Gson gsonBox;

    public JsonBoxDeserializer(){
        gsonBox = MyGson.getGsonBox();
    }

    @Override
    public T deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
        try {
            T var = gsonBox.fromJson(json, typeOfT);
            for (Map.Entry<String, JsonElement> entry : json.getAsJsonObject().entrySet()) {
                try {
                    Field field = var.getClass().getDeclaredField(entry.getKey());
                    field.setAccessible(true);
                    if (field.getType() == List.class) {
                        filling((List) field.get(var), ((Class) (((ParameterizedType) field.getGenericType()).getActualTypeArguments()[0])), entry.getValue(), context);
                    } else if (field.getType() == ToOne.class) {
                        filling((ToOne<? extends JsonBox>) field.get(var), ((Class) (((ParameterizedType) field.getGenericType()).getActualTypeArguments()[0])), entry.getValue(), context);
                    }

                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
            return var;
        } catch (Throwable e) {
            e.printStackTrace();
        }
        return null;
    }

    protected <T> void filling(List<T> toMany, Class<T> type, JsonElement jsonElement, JsonDeserializationContext context) {
        if (toMany == null)
            return;
        toMany.clear();
        if (jsonElement.isJsonArray()) {
            JsonArray array = jsonElement.getAsJsonArray();
            for (int i = 0; i < array.size(); i++) {
                JsonElement element = array.get(i);
                T item = context.deserialize(element, type);
                toMany.add(item);
            }
        }
    }

    protected <T extends JsonBox> void filling(ToOne<T> toOne, Type typeOfT, JsonElement jsonElement, JsonDeserializationContext context) {
        if (toOne == null)
            return;
        T one = context.deserialize(jsonElement,typeOfT);
        toOne.setTarget(one);

    }

}

I instance gson excluding List and ToOne.

public static Gson getGsonBox(){
        return new GsonBuilder().setExclusionStrategies(new ExclusionStrategy() {
            @Override
            public boolean shouldSkipField(FieldAttributes f) {
                return false;
            }

            @Override
            public boolean shouldSkipClass(Class<?> clazz) {
                return clazz == List.class || clazz == ToOne.class;
            }
        }).create();
    }

And the first deserializer to call:

new GsonBuilder() .registerTypeHierarchyAdapter(JsonBox.class, new JsonBoxDeserializer<>()) .create();

[damonbakker]

The automatic filling was quite important in a lot of my code that used greendao and retrofit

I did a lot of syncing with retrofit on a lot of endpoints. Spend a good amount of time refactoring to allow for objectbox with daocompat.

Would really love to see some compatibility here. I'll try to implement some of the suggestions above, potentially reverting to greendao if it doesn't work out nicely.

[ajans]

I wanted to give my point of view on how I handle this scenario:

I was pondering how to do this and what would be considered as a best practice. This article gave some insightful pointers on why it is not a good idea to use the Retrofit-response-model as the model for ObjectBox-operations: https://proandroiddev.com/the-real-repository-pattern-in-android-efba8662b754

TLDR of this article:

• Models used for remote communication (Retrofit-models aka DTOs (data transfer objects)) should not be used as domain-model entities used by an app, as changes to the DTO would affect the whole codebase for the app
• Therefore, there should be a mapping-process between DTOs and domain-model entities and a repository-class should always return domain-model entities to the app
• This way, if anything changes on the backend-side, only the mapping in the respective repository-class has to be adapted

With this way of thinking, I was able to mitigate this issue in a clean way without having to resort to custom deserializers or other hacky solutions. So, my BaseRepository-code to refresh a list of data from a remote API and save it to ObjectBox basically looks something like this:

/**
 * Type parameters:<br/>
 * DTO: domain transfer object (remoteapi-gson-models)br/>
 * DM: domain model (app-models)<br/>
 */
abstract class BaseRepository<DTO, DM : DomainModel>(
    val service: RemoteApi,
    private val localDataSource: LocalDataSource<DM>
) {
    // ...
    suspend fun refreshList() {
        wrapEspressoIdlingResource {
            val dtoData: Result<List<DTO>?> = fetchList()
            if (dtoData is Result.Success) {
                val mappedData = mapList(dtoData.data)
                saveToDatabase(mappedData)
            } else if (dtoData is Result.Error) {
                throw dtoData.exception
            }
        }
    }

    fun mapList(dtoData: List<DTO>?): List<DM>? {
        logTrace()
        if (dtoData == null || dtoData.isEmpty())
            return null

        val result: ArrayList<DM> = ArrayList()
        for ((index, sourceItem) in dtoData.withIndex()) {
            val mappedModel = map(index, sourceItem)
            result.add(mappedModel)
        }

        return result
    }

    abstract fun map(index: Int, dtoItem: DTO): DM

refreshList is used from a ViewModel-LiveData.

First step of refreshList is to get the DTO-data from the remote-API via fetchList.

Second step is to map this data with the mapList and map functions. The concrete implemention of the map-function instantiates the domain-model-entities, sets their properties and adds related mapped sub-entities and puts them into the ToMany-fields.

After mapList, the mapped data (in your domain-model entities) then gets persisted into the ObjectBox-database in the saveToDatabase-method.

There is a very similar implementation of this steps for single DTO-item-responses from the remote-API.

To use this code, you derive a Repository-class from BaseRepository for a pair of DTO and domainmodel-entity that you want to persist in your objectbox from this. In the concrete Repository, you implement the map-function, the remote-API call (called from fetchList) and define the generic type of the LocalDataSource-interface in the constructor. The concrete LocalDataSource-implementation class for the used ObjectBox-entity knows the specific Box or Boxes for used entities and uses those boxes to get or put the entity-instances or provide ObjectBoxLiveData-instances with the specific queries.

If you read the article above and its recommendations, you might think this is a bit of cheating. That is true, if you would follow the recommendations consequently, you would not use ObjectBox-annotations and ToMany in your domain-model entities at all (keep them pure of any persistence and communication framework stuff), but I found it works quite well in my case as a tradeoff between pragmatism and purity as I didn't want to introduce another model-layer and its mapping at the time.

The one thing I had a bit of trouble with, was unit-testing the ViewModels and the repository-classes' mapping with ObjectBox running on desktop, as in my multi-module project, the ToMany-property would not get initialised and failed the tests.

I worked around that by abstracting ObjectBox away behind a LocalDataSource-interface (injected into the Repository-constructor) , thus faking it for the unit-tests and doing the "Initialization magic" trick mentioned in the objectbox-docs.

So, my TLDR recommendation is:

Don't use your Gson- or Moshi-models as ObjectBox-models.

At least separate those two data-source models by mapping one onto the other in your repository-classes. This way, you can use ObjectBox the way you like and can even separate your remote-API-client out of your app-project into its own project that you don't have to build all the time but only if something changes on your backend.

P.S.: if you wonder why the map-function takes an incremented index as a first parameter, this is being used in a @BaseEntity order-property to preserve the order of DTO-items coming from the remote-API. So if you have something like top-items with an arbitrary serverside-defined order to them, you don't have to duplicate that ordering-business-logic in your app in the ObjectBox-query. You can simply use the order-property. You could also use ObjectBox-generated IDs, but I found this to be quite helpful with assignable IDs in my ObjectBox-models.

[Mustafa-Altameemi]

At the final stage ToMany<> just messed up on all my project.

[Ditscheridou]

Any updates here? Im currently looking into Objectbox + mapstruct, this ToMany/ToOne messes up my whole mapping process.

[Ditscheridou]

@greenrobot-team ?

[mecoFarid]

@ArthurSav

Maybe in another 2 years :)

Well, here is your 2 years

[mecoFarid]

Recently had to work with ObjectBox and Gson. If you don't want to add temporary fields the only other solution I could come up with is to write a custom deserializer for each entity.

@Entity class User {
    private ToOne<Status> status;
    private ToMany<Address> addresses;
    // TODO getters/setters
}

class UserDeserializer implements JsonDeserializer<User> {
    @Override
    public User deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context)
            throws JsonParseException {
        JsonObject userJson = json.getAsJsonObject();
        User user = new User(); // ToOne and ToMany initialized here through code injected by ObjectBox plugin

        // ToOne
        JsonObject statusJson = userJson.getAsJsonObject("status");
        Status status = context.deserialize(statusJson, Status.class);
        user.getStatus().setTarget(status);

        // ToMany
        JsonArray addressesJson = userJson.getAsJsonArray("addresses");
        Type addressListType = new TypeToken<ArrayList<Address>>(){}.getType();
        ArrayList<Address> addresses = context.deserialize(addressesJson, addressListType);
        user.getAddresses().addAll(addresses);

        // TODO parse other properties

        return user;
    }
}

-Uwe

We have more than 800 models tied with ToOne/ToManyrelations, would be fun to implement what you recommend. One more thing; we must not forget to do the same for future relations or else the whole project will be doomed

[Mustafa-Altameemi]

Well, as I have gone deep through the library source code, so (I think) the only solution for @greenrobot-team is to start building a new DB from scratch. It's very disappointing to see an experienced developers team fall into such a mistake.

Anyway @greenrobot-team thanks for your efforts.

[ajans]

Well, as I have gone deep through the library source code, so (I think) the only solution for @greenrobot-team is to start building a new DB from scratch. It's very disappointing to see an experienced developers team fall into such a mistake.

Anyway @greenrobot-team thanks for your efforts.

As a long-time user of ObjectBox (currently working on the 3rd app with it, 2 of them have professional context), I am very curious to hear from you specific arguments based on specific ObjectBox-code to support your bold statement.

In my opinion it is a mistake to judge the whole ObjectBox-ecosystem based on a client-library for a specific platform (even if it is the primary and most mature one).

Yes, the handling of ToOne and ToMany-properties does not play very nice with e.g. Retrofit directly, but as I pointed out in my comment above, I think it is a fundamental design-flaw in one's app-code to not separate a Retrofit-POJO from an app-internal domain-model and its related ObjectBox-entity for the reasons stated in my earlier comment and the article I quoted.

[ArthurSav]

Forgot to unsubscribe from this. Here's my 2cents:

After many years of coding i realise why plain old sqlite is still relevant. The headaches caused by 3rd party libs are usually not worth the switch but that is obvious only after a lot of hard work has gone into a project.

[AfricanDev]

UserDeserializer 

can you provide sample on how to use UserDeserializer @greenrobot

[greenrobot-team]

@AfricanDev Can you please stop spamming on issues! It's enough to post once on one issue.

Look at the Gson user guide on how to de/serialize custom types. https://github.com/google/gson/blob/master/UserGuide.md