@Unique: replace on conflict flag

[greenrobot]

Right now, @Unique throws an exception if there's a violation.

We could also offer to replace old entities with newer ones, e.g. by doing a @Unique(replaceOnConflict = true).

Keep in mind: If there are multiple unique properties, on entity might replace several others.

[avcial]

Actually requirement is not completely replacing, just update fields which are not annotated as @Id or @Unique

[greenrobot]

@avcial Not sure if I follow. Example?

[kordianbruck]

@greenrobot you can update the fields instead of delete+create ("replace") objects that have a unique property (like primary keys and explicit @unique), but that's just an implementation detail. Difference is, that PKs will stay the same an FK will not break while updating a unique key with replaceOnConflict.

[avcial]

@kordianbruck you are soo right As i mentioned before here, mine and in my opinion the most common case is that caching server datas to device for giving faster responses, but the necessity of UpdateOnConflict is that what if my users last name has changed but i don't want to lose my ObjectBox db ObjectId and relations on it. Scenario : Get datas from WebAPi you have empty table, you inserted all of them with logic [long id,string ServerGuid, string FirstName, string LastName]. Then you want to update your ObjectBox user table because you know you have some new users or some of your users might change their name or maybe they got married and their last name changed and you don't want to show old datas on your screen. So you get the new list from your api and then you give it to a function whic adds new users with new objectIds but it also checks the older records based on your second column(string ServerGuid) that is because your Server doesn't know ObjectBox ObjectId's of your devices and when you got data which are inserted before you just check the fileds(FirstName and LastName) which are not annotated by @Id or @Unique or maybe they are annotated on consciously like @UpdateOnConflict and update them with new datas which comes from the current entity.

[greenrobot]

Yeah, it makes sense to keep the ID, because it is used in relations and you do not want to cut those off. So it should rather be @Unique(onConflict = UPDATE) with some enum {FAIL, REPLACE, UPDATE}.

We could extend further extend the enum with ABORT_TX (e.g. check SQLite ON CONFLICT).

HOWEVER, I'm getting doubts if it's a good idea to tie it to an annotation. Because annotations on two different properties could be contradicting. And also, behavior is less flexible.

The more flexible approach would be to overload the put method and pass either a ConflictResolution enum or flags (more hacky but more future proof) additionally.

[kordianbruck]

There is a bit more specific documentation in the SQLite docs: https://sqlite.org/lang_conflict.html

So yes, its a good idea and general industry standard when talking about SQLite based things: https://developer.android.com/reference/android/arch/persistence/room/OnConflictStrategy

Maybe SQLite is not the reference here, but I think when devs come from using an ORM like room, they at least expect similar ways how those things work :+1:

[avcial]

I know the logic will be a mess but when you add the exception throwing on conflict at ver2.0.0 i expected that maybe you would return the original entity inside the error so we can do whatever we want to do with conflicted row, maybe try to put it again after some changes??

[greenrobot]

@kordianbruck Not sure what you are arguing for, could you explain? Do it as annotations? SQLite supports conflict resolution within column definition (inflexible), but also inside INSERT/UPSERT (flexible). I tend to prefer the flexible option, e.g. box.put(person, REPLACE) because at another code section you might want to do box.put(person, IGNORE) because you might have another use case.

@avcial Maybe be a mess for lists, but OK for single entities; e.g. Person conflict = box.putOrGet(person). Same with lists would be possible but you would loose the connection of offending and existing entity.

[avcial]

if performance is good enough it does't matter if i loose connection. maybe the method will return me the list which is handled for the choosen strategy(ies)??

[kordianbruck]

@greenrobot sorry, I could've been more clear on that: I would vouch to use the same naming conventions as in SQLite for ROLLBACK, ABORT and FAIL so that developers coming freshly to objectbox already can guess what that flag is doing from their previous SQLite experience.

For our use cases we would be fine with annotated support. If the more flexible option is doable without modifying the API too much that also might be a good idea - no clue tho, how useful it is.

[avcial]

Throwing exception break's box.put(Collection ) method loop, there is a need for quick solution for that

[Mistic92]

Yep, I can't update via put method. I'm getting crash when entity with unique annotation exists but I just want to update

[avcial]

Any upgrade or time estimation?

[wheelergames]

Jumping on the bandwagon for this. I want to do put(Collections) with Entities that have a Unique field. If I have a clash the try catch just throws an exception and stops. I'd love to have that overloaded method to say on unique clash, do update or something like that.

[greenrobot]

Couple of questions to better understand requirements:

1. Do you use @Unique to emulate e.g. a string ID?
2. Or, do you have multiple @Unique properties in a single entity or do you plan to?
3. Any preference to put the conflict strategy on @Unique or, more flexible, with put()? And why?

[wheelergames]

To answer your questions 1) Yes 2) No 3) no pref, maybe in put() as I'm not an expert on annotations

So my use case might have that extra level of annoyance/complication, as my Box has a ToMany with it...

I want to have a box/entity of words, to form a dictionary, then I want a separate box for a difficulty level, which links on many to many to the words.

i.e. a word like cat will exist on easy and hard words, but a word like supercilious would only be in hard

I will upload all the words from one dictionary e.g. easy, which will populate the Word Box and the WordDifficulty Box, then upload all the hard words, which will

1) add a new word if needed 2) if it exists add a 'row' in the WordDifficulty for hard

(I'd like to do the put for this as a List so use the collections version of put)

[avcial]

Good to be aware about conflicts on Uniqly annotated fields but throwing exception breaks put function and not giving any referance of conflicted entity, to update or do something with that entity, i have to querry it to find which entity was registered to db before, i need to know that, if its possible i need to make changes on that verry quickly like change its "LastSyncDate" field.

[wheelergames]

The error message actually says which IDs have clashed, but you'd need to somehow regex the string in order to find that out, which is not exactly ideal!

[mesterj]

@Unique(replaceOnConflict = true) this behaviour will default? I think true would be better then throwing exception. When new version will available? I am waiting :-)

[avcial]

any news?

[Queatz]

Should be in the .put, because one may want to do different things in different flows. Looking for this feature, too. :+1:

[greenrobot]

@Queatz Do you have a real (seen in practice) use case?

Both options have up- and down-sides. With put, there can be multiple unique properties that work differently. And a put may span over relations... Thus, I'd like to continue with property-based on-conflicts.

[greenrobot]

Current proposal: @Unique has the default on-conflict strategy FAIL; any unique violation throws an exception.

@Unique(onConflict=IGNORE): ignore the offending object (no changes to the existing conflicting object). If there are multiple unique properties in an entity, this strategy is evaluated first: if the property conflicts, no other properties will be checked for conflicts.

@Unique(onConflict=REPLACE): the offending object replaces the existing conflicting object (deletes it). If there are multiple properties using this strategy, a single put can potentially replace (delete) multiple existing objects.

@Unique(onConflict=UPDATE): the offending object overwrites the existing conflicting object while keeping its ID. Thus, all relations pointing to the existing entity stay intact. This is useful for a "secondary" ID, e.g. a string "ID". Within an entity, this strategy may be used once only (update target would be ambiguous otherwise).

Multiple strategies in an entity are processed in this order: IGNORE, FAIL, REPLACE, UPDATE.

[Queatz]

In my app, I basically have a simple implementation of Offline First:

@Entity
class Message {
    @Id long localId;
    String id;
    ...
}

When a user sends a message, it creates a new Message with a null id. When a sync happens, the Message is sent to the server and an id is returned, which then gets set on the Message. Any subsequent load from the server will update the local model.

However, there is a slight chance that the Message can be loaded from some other server endpoint before the id is successfully returned and set on the local model. So I end up with duplicates.

Ideally I could do:

@Unique(onConflict=UPDATE) for any object coming from the server @Unique(onConflict=IGNORE) for any object created locally (i.e. when the server ID is added to the local object, and that ID already exists on an object due to a race condition, just ignore adding the ID, because the model is outdated anyways.

For this app, I could get by with only being able to pick one option per field, however I can foresee running into a wall there by having different requirements in different scenarios.

[greenrobot]

@Queatz Don't see duplicates in this scenario. Check this example and let me know if and where we diverged:

1. Your entity with localId 42 is being uploaded
2. Server assigns id "foo"
3. "foo" object gets to the client on a different route, it's inserted with localId 43 in another thread (race condition to 1. step)
4. First step (upload) gets "foo" back; with onConflict=UPDATE it will update into localID 43 which got there first (object with ID 42 is removed)

No dups. However, any relations to localId 42 will be deleted. If that is a problem, server-side queuing should solve it. Nobody said sync is easy... ;-)

[Queatz]

Sync is hard xD Can't wait to play around with objectbox sync when it's ready. :)

onConflict=UPDATE would solve the dupe, but then any extra details the server added will be removed (if i understand correctly)

[aToO57]

Hello, any ETA for this feature ?

[Queatz]

Would be good to know a ballpark ETA as well. If it's ~1 month out then I get to skip writing workaround logic.

[meteoral]

Is there any news for this feature?

[ajay-a1]

any ETA for this feature?

[saeednt]

is gonna be released any time soon?

[lucamtudor]

Does this look like a reasonable workaround when you need to update the local db with some objects from a server?

val fromServer = Contact(uidString = UUID.randomUUID().toString(), name = "Tudor")
copyToStoreOrUpdate_insteadOfSimplePut(fromServer)

fun copyToStoreOrUpdate_insteadOfSimplePut(fromServer: Contact): Contact {
    val box = ObjectBox.store.boxFor<Contact>()

    val localIdForExisting: Long = box.localIdFor(fromServer.uidString)
    val contactWithLocalId: Contact = fromServer.copy(localId = localIdForExisting)

    val finalLocalId: Long = box.put(contactWithLocalId)

    return contactWithLocalId.copy(localId = finalLocalId)
}

fun Box<Contact>.localIdFor(id: String): Long {
    return query { equal(Contact_.uidString, id) }
        .findIds()
        .firstOrNull() ?: 0
}

@Entity
data class Contact(
    @Id var localId: Long = 0, // required by ObjectBox

    // TODO Replace @Index with @Unique(onConflict=REPLACE), once it lands in ObjectBox, or when we get `String` @Id
    @Index
    val uidString: String,

    val name: String
)

[Queatz]

Is there anything we can do to help move this guy along?

[GiulianoFranchetto]

Is this issue still in the roadmap? When will you release it?

[lucamtudor]

any updates?

[Queatz]

Should we send a :beer:?

[lucamtudor]

Are you guys actively working on this? This is a major pain-point for us!

We can't properly use relations in our models and let ObjectBox figure out when an object is new or needs updates, so we have to do a manual look-up each time we want to save something.

We used Realm & started migration to ObjectBox because of better API, better support for Architecture Components & no weirdness caused by the no-copy model. And because Realm is so damn slow at implementing features requested by the community (there still are a couple of issues from 2017 I'm still waiting for).

We're building a chat platform with lots of features that would benefit from ObjectBox's upcoming sync. But if I can't build the app today without making lots of error & bug prone workarounds there won't be a day when we'll happily pay for the sync service.

I moved away from SQL in 2014 and never looked back. Realm worked pretty well for not very complex scenarios & it was still better than writing tons of boilerplate code for SQL. ObjectBox came along at it seemed to be something even better, especially in the context of Android Architecture Components. But for a slightly more complex scenario that doesn't use Long ids, everything falls apart. I'm seriously considering going back to SQL. Room became a pretty solid library. It even supports suspend functions!

If the ObjectBox team is not interested in working on this, or has other priorities, just say so! Don't keep us waiting for months on end, for nothing.

Thanks, and sorry for the long & ranty message!

[greenrobot]

For once, some background here: we were in the middle of implementing this when we realized that this feature might impact data synchronization in a complex way. E.g. this feature brings implicit deletes; and with potentially multiple data states across clients and servers; this might become problematic. We want to implement this feature in a future safe way that plays well with sync.

[mehdiyari]

@greenrobot iam using objectbox 2.3.4 on android and i have problem for update with model that have one unique property i try this comment but io.objectbox.annotation.Unique annotation does not require any argument

[Queatz]

@mehdiyari We are all waiting patiently for this feature to become available. :)

[halcyonashes]

Still no update on this?

[Queatz]

Bump. Still really really need this.

[mataide]

We all need this or String as ID

[Queatz]

Is this still going to happen?

[greenrobot-team]

See https://github.com/objectbox/objectbox-java/issues/509#issuecomment-491735260 on why this is delayed until sync ships, still valid.

[Reginer]

New year is coming .

[Queatz]

Just leaving a note here that we are still waiting and rooting for the team 🙂

[vpotvin]

Curious if there is an update and what people are doing as an optimal workaround for now?

My scenario is pretty basic offline-first with server syncing, attempting to use @Unique to keep local id separate from server side id. So far the only obvious solution I see is try to put every object in the database iteratively.

[Queatz]

@vpotvin that's what I do as well. It's not always perfect and breaks down quite often for various reasons. It's sad my social network has been broken for over 2 years because of this one issue, but there are no good alternatives to ObjectBox, so I've just implemented the iterative approach the best that I can.

[Queatz]

@vpotvin to be a little more detailed, this is my approach in Kotlin pseudo:

fun handleServerModels(results: Array<ModelResult>) {
    val serverIds = results.map { it.id }
    val existing = objectBox.box(Model::class.java).query(Model_.id.oneOf(serverIds)).build().find()

    results.forEach {serverModel ->
        val localModel = existing.find { it.id == serverModel.id }
        if (localModel != null) // then update local
        else // create new
   }
}

My full implementation does a little more than that, but here for reference:

internal fun <T : BaseObject, R : ModelResult> handleFullListResult(
        results: List<R>?,
        clazz: Class<T>,
        idProperty: Property<T>,
        deleteLocalNotReturnedFromServer: Boolean,
        createTransformer: (R) -> T,
        updateTransformer: ((T, R) -> T)?) {
    results ?: return

    val serverIdList = results.map { it.id!! }.toSet()

    on<StoreHandler>().findAll(clazz, idProperty, serverIdList).observer { existingObjs ->
        val existingObjsMap = HashMap<String, T>()
        for (existingObj in existingObjs) {
            existingObjsMap[existingObj.id!!] = existingObj
        }

        val objsToAdd = mutableListOf<T>()
        val idsToAdd = mutableSetOf<String>()

        for (result in results) {
            if (idsToAdd.contains(result.id!!)) continue
            idsToAdd.add(result.id!!)

            if (!existingObjsMap.containsKey(result.id!!)) {
                objsToAdd.add(createTransformer.invoke(result))
            } else if (updateTransformer != null) {
                objsToAdd.add(updateTransformer.invoke(existingObjsMap[result.id!!]!!, result))
            }
        }

        on<StoreHandler>().store.tx({
            on<StoreHandler>().store.box(clazz).query()
                    .`in`(idProperty, objsToAdd.map { it.id!! }.toTypedArray())
                    .build()
                    .remove()

            if (deleteLocalNotReturnedFromServer) {
                on<StoreHandler>().removeAllExcept(clazz, idProperty, serverIdList, false)
            }

            on<StoreHandler>().store.box(clazz).put(objsToAdd)
        })
    }
}