search

realm / realm-java

Realm is a mobile database: a replacement for SQLite & ORMs
http://realm.io
Apache License 2.0
11.45k stars 1.75k forks source link

@SerializedName annotation for defining Class/Field names #1470

Open beeender opened 9 years ago

beeender commented 9 years ago

We need something similar to GSON's SerializedName annotation which can map a different name field from JSON string to RealmObject's property.

eg:

{ "dog_name": "abc"}
class Dog {
    @SerializedName("dog_name")
    private String name;
}
navjotbedi commented 8 years ago

Any update on this feature ?

taltstidl commented 7 years ago

I'd really like to use Realms Json methods, but the way they currently work they are unfortunately too limited to be of any use to me. In addition to a @SerializedName annotation, I'd also need a @SerializedObjectId(s)Name annotation that allows linking of related objects using their ids. Most web APIs only return the id of the related object, which makes it impossible to use Realms Json methods (as far as I know).

So, for example, the following should work:

{"id": 1, "relatedId": 1, "relatedIds": [1, 2, 3]}
class TestObject extends RealmObject {
    // different serialized name in json representation
    @SerializedName("id") @PrimaryKey
    long mId;

    // one-to-one relationship in json representation
    @SerializedObjectIdName("relatedId")
    RelatedObject mRelatedObject;

    // one-to-many relationship in json representation
    @SerializedObjectIdsName("relatedIds")
    RealmList<RelatedObject> mRelatedObjects;
}

Using those annotations the Realm Json methods should be able to properly transfer the information encoded in the Json to the Realm object model, including its relationships. Having those annotations would solve quite a few issues for me with partially updating objects (see issue #3500) and would reduce the amount of code needed and make it more efficient.

cmelchior commented 7 years ago

@TR4Android I have browsed through the code and compiled a list of things that needs to be done (I probably forgot something though). If you decide to start on this, I would strongly encourage you to open a PR as soon as possible so we can follow progress along the way and adjust if needed. It would make sense for us to make a feature branch which you can make smaller PR's against instead of trying to cram everything into one big PR against master. It will be a lot easier for us to review.

Let me know. Cheers.

Overall design goal

Make it possible to use a Java specific name when instead of being forced to adopt the name being used by a JSON API or another platform.

USE CASE: My JSON input does not match my preferred Java class. This is useful if using our various createFromJson methods

USE CASE: I'm sharing a schema with iOS and would like my own names as the chosen names break normal Java conventions.

With this approach we combine these two use cases. Is that okay or will it bite us later so we don't want to mix JSON and Schema concerns?

TODO List:

Conclussion

Am I missing something on this list @realm/java and do you have other points? I believe @zaki50 and @kneth were the last to touch these parts of the code.

Zhuinden commented 7 years ago

I don't think JSON concerns should be mixed with schema concerns.

cmelchior commented 7 years ago

I have the same feeling, but it gets a bit tricky, because cross-platform schemas are a big topic for us right now 😄 

public class Foo extends RealmObject {

  // This is definitely eecky
  @SerializedName("my_stupid_json_name")
  @FieldName("internal_name")
  private String name;
}

Thinking about this. Maybe this is actually the best time to actually split our JSON API away from Realm, because a @SerializedName for JSON would only make sense if you used our JSON API, if you use e.g GSON it would just be confusing that Realm also had a similar annotation.

Instead we should create adapters for the various JSON imports:

- `io.realm:android-json-adapter:1.0.0` : Basically our existing API just moved to a seperate dependency. This library could then include specialized annotations like `@SerializedName`

- `io.realm:gson-adapter:1.0.0` : GSON type adapters

- `io.realm:android-streaming-json-adapter:1.0.0` : Our API 11 streaming adapters

Alternatively they get configured through the new realm closure block introduced in 2.0

It would have the following implications:

taltstidl commented 7 years ago

@cmelchior I tend to agree with @Zhuinden, I wouldn't want to mix the field and json names. Splitting this to its own library might make sense, as it would allow room for more superior support without making the core library more cluttered. I'd gladly add export APIs to this as well. Might make sense to name this realm-import or similiar and have it available for more platforms.

I'll probably need some guidance on the implementation then. Should the supplementary library alter the proxy classes? How should I split the library from the core one? How should I name the new classes for importing/exporting json?

I'll definitely create a PR as soon as possible, so you can track progress and intervene when I make mistakes on my side. I'm already exited to see this in a future release as it will make my app code that much simpler, and hopefully more performant!

One other thing, for the annotation I'd like to be able to build links to other objects using its primary key. That is currently the main reason I'm not using other json libraries, as I'm looking to build the relationships more natively. So, I'd like to see an option for that in the API design as well (see my previous comment).

Zhuinden commented 7 years ago

@cmelchior considering additional annotation processors can be executed on the same RealmObjects now that annotations aren't consumed, adding additional adapters (or adapter factories akin to Retrofit2) would make sense similarly to how "Rx support" is opt-in by adding the RealmObservableFactory.

cmelchior commented 7 years ago

@TR4Android I didn't consider the possibility to automatically hook up references, but you are right, it would also be a nice feature if done automatically. Especially since it is a lot of tedious code to do manually.

Basically by splitting the library we don't have to be as conservative with features, so export capabilities also make a lot more sense. I would however expect we want to share a lot of code between annotation processors, so we would need to find a way to do that.

We probably need to have an internal discussion first though as there are a lot of moving parts to this, but if you have some good ideas for how the public API could look like for this I am all ears.

@Zhuinden Yes, not consuming the annotations is definitely a major part of this as this allow us to mix any number of adapters. While the Retrofit factory method is very nice, keep in mind that right now we accept 3 different forms of JSON Input: InputStream, String and JSONObject/Array, so I don't think we can get away with just having a factory method somewhere as those 3 types are basically 3 different adapters.

taltstidl commented 7 years ago

@cmelchior Yes, you should probably have an internal discussion first. I'm not sure yet how the structure of the annotation processor should look, this is something I'd first have to investigate more. Any ideas would be appreciated. The public API would be pretty much what I have outlined in the other issue.

cmelchior commented 7 years ago

I'm mostly talking about the createFromJson* methods and friends. With the adapter approach I don't think it would make sense if they hang on the Realm object anymore, but they need to be somewhere (wishes for Kotlin extension methods)

cmelchior commented 7 years ago

Just throwing ideas at the the wall:

// Suggestion for the Realm closure
realm {
    gsonAdapters = true
    jsonStringAdapters = true
    jsonStreamAdapters = true 
    jsonObjectAdapters = true
}

// Version 1

Realm.init(context) // Automatically detect and register relevant adapters

// Combine all current methods so they accept `Object`
// This enable any adapter to "register" for a specific subclass and handle that
// Will reduce number of methods from 12 -> 4, and be backwards compatible with
// the proper adapter used. Will make the method signature less readable and 
// crash if called with no adapter installed.
Realm.createObjectFromJson(Class, Object);
Realm.createAllFromJson(Class, Object);
Realm.createOrUpdateObjectFromJson(Class, Object);
Realm.createOrUpdateAllFromJson(Class, Object);

// Version 2

// Each adapter exposes a init method + a controller class
// Having a really hard time convincing myself this is a good idea

// Init
RealmConfig config = getConfig()
RealmJson.init(realmConfig) // Controller class for each adapter (RealmJson/RealmJsonStream/RealmJsonObjects)

// Usage
realm.beginTransaction();
RealmJson.createOrUpdateObject(Realm, Class, Object);
realm.commitTransaction();
beeender commented 7 years ago

Split the json to another project would be great, especially we will have a chance to switch to javapoet from there. But the new project probably needs to be more generic to support other data converters than JSON, eg. flatbuffer.

taltstidl commented 7 years ago

Yes, I would agree that splitting this into its own project sounds like the best solution. This gives us the freedom to add additional import formats as well (XML, CSV, you name it...), though I'd focus on JSON as it's probably the most widely used format.

In order to prepare for splitting, a few design considerations need to be made first:

Where do the import/export methods go?

What are the import/export methods named?

How are the import/export method realized?

Chances are that I missed something above, but that's the general direction I had in mind. First-class support for importing and exporting common formats can greatly increase the performance of the code and the speed of development using Realm!

taltstidl commented 7 years ago

@cmelchior Any update on that internal discussion? Thanks! 😉 By the way, do you have any statistics on how often the InputStream flavors are used? I'm not sure I have seen that approach very often, mostly only the JSONObject flavor is used.

cmelchior commented 7 years ago

Hi @TR4Android Sorry not yet. I'm writing up a proposal with use cases and solutions right now just so we are sure we got all present and future use cases covered.

We don't have any stats for when either is used. A gut feeling is that most people go through GSON and similar instead of using our methods for it. After that, it would assume that String/JSONObject are the most frequently used, yes.

taltstidl commented 7 years ago

@cmelchior No problem, I'm as interested in getting this right as you guys are! I appreciate the time you're investing in a proposal, keep up the good work :+1: If there's anything I can help with let me know!

taltstidl commented 7 years ago

@cmelchior Any update? :wink:

cmelchior commented 7 years ago

Hi @TR4Android Sorry for the late turn-around time, we just came back from a company wide get-together. I posted some thoughts in #3758

dhuma1981 commented 6 years ago

Any update on this issue? Eagerly waiting for it.

abou7mied commented 6 years ago

Until this issue is resolved I have made a workaround in my App, let's say I have Chat.java

public class Chat extends RealmObject {
    @Index
    @SerializedName("c")
    private String code = "";

    @Index
    @SerializedName("t")
    private int type;
}

And RealmUtil.java

public class RealmUtil {

    private static HashMap<String, HashMap<String, String>> classesFieldsMap = new HashMap<>();

    static {
        initClasses(Chat.class);
    }

    private static void initClasses(Class cls) {
        if (!classesFieldsMap.containsKey(cls.toString())) {
            Field[] fields = cls.getDeclaredFields();
            HashMap<String, String> fieldsMap = new HashMap<>();
            for (Field a : fields) {
                SerializedName annotation = a.getAnnotation(SerializedName.class);
                if (annotation != null) {
                    fieldsMap.put(annotation.value(), a.getName());
                }
            }
            classesFieldsMap.put(cls.toString(), fieldsMap);
        }
    }

    public static JSONObject mapGsonObjectToRealm(Class cls, JSONObject object) throws JSONException {
        JSONObject newUserObj = new JSONObject();
        HashMap<String, String> fieldsMap = classesFieldsMap.get(cls.toString());
        for (String setKey : fieldsMap.keySet()) {
            String mappedKey = fieldsMap.get(setKey);
            if (object.has(setKey))
                newUserObj.put(mappedKey, object.get(setKey));
        }
        return newUserObj;
    }
}

By using reflection initClasses method will map each of the Gson SerializedName of the class to the corresponding names for Realm record.

And mapGsonObjectToRealm method will create a new object of the old one with new fields names.

Usage:

JSONObject newObject = RealmUtil.mapGsonObjectToRealm(Chat.class, new JSONObject().put("c", "this_is_code").put("t", "this_is_type"));
final JSONArray jsonArray = new JSONArray();
jsonArray.put(newObject);
Realm.getDefaultInstance().executeTransactionAsync(new Realm.Transaction() {
    @Override
     public void execute(Realm realm) {
        realm.createOrUpdateAllFromJson(Chat.class, jsonArray);
    }
});
Zhuinden commented 6 years ago

Realm.getDefaultInstance().executeTransactionAsync(

I wonder how many articles I need to write about how wrong this is before people stop doing it.

Btw, are you sure you need to do this reflection magic manually? I think this is what Gson does with gson.fromJson(string, class).

Please be aware that this probably wouldn't work without the right Proguard rules.

abou7mied commented 6 years ago

@Zhuinden gson.fromJson will return an object of the class param and If some fields are missing in the data received from the server, say (t field in Chat.java) then It will be 0 in object and will be saved 0 in Ream DB. But with JSONObject Realm will just update the existing fields.

Zhuinden commented 6 years ago

@abou7mied AH i understand, I guess that makes sense.

Make sure you apply proper Proguard configuration though!

Zhuinden commented 6 years ago

Jokes aside, the ability to unlink the schema field name from the actual Java field name using an annotation would be nice, and should be completely independent from JSON adapters.

felixklauke commented 6 years ago

Any update here? I would prefer a solution without this hacky gson way. We would really appreciate a simple solution with a single annotation.

abou7mied commented 6 years ago

Any updates? I made a lib for my project to map from json keys to realm java-keys and use createOrUpdateObjectFromJson for the mapped object but it would be good to make it using Realm without workarounds

ishaileshmishra commented 6 years ago

Hi there, Is there any way to get the list of values declared in @FieldName("internal_name"), let's suppose I have declared 10 fields annotated with FieldName with values in it, So how would I get 10 values?

cmelchior commented 6 years ago

Depends on what you need. If you need only the fields annotated that way, then the only way is to use reflection on the model class. If you use realm.getSchema().get(Foo.class.getSimpleName()).getFields() it will return all the internal field names in the class.

ishaileshmishra commented 6 years ago

Hi @cmelchior Thanks for your quick reply, I need only fields annotated by @FieldName

cmelchior commented 6 years ago

Then you need to use reflection. This should work:

        for(Field field  : Foo.class.getDeclaredFields()) {
            if (field.isAnnotationPresent(RealmField.class)) {
                RealmField ann = field.getAnnotation(RealmField.class);
                String internalName = ann.name();
            }
        }
ishaileshmishra commented 6 years ago

Thanks @cmelchior for your support, it worked.

abou7mied commented 5 years ago

Hi guys, I made a library to help me to map SerializedNames to Realm names.

I don't know if it is a good practice to do such a workaround or not, anyway I would like to share it with you https://github.com/abou7mied/realm-useful-helpers

BoD commented 4 years ago

Just wondering if this will be released at some point? In the meantime I'll be using https://github.com/cmelchior/realmfieldnameshelper which seems perfect - but it would be nicer to have this directly in Realm.

Zhuinden commented 4 years ago

I mostly wish for RealmFieldNamesHelper to be updated to be incremental annotation processing