apache / fury

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.
https://fury.apache.org/
Apache License 2.0
2.97k stars 218 forks source link

FR: Only serialize annotated fields #1735

Closed Frotty closed 1 month ago

Frotty commented 1 month ago

Is your feature request related to a problem? Please describe.

I want to move an existing project to fury, but right now data fields in classes are marked with an annotation. Fury only supports not de-/serializing transient fields, but it's not convenient to mark a majority of fields as transient.

Describe the solution you'd like

I would like there to be an annotation that allows me to mark only the fields of a class that should be de-/serialized.

Additional context

This was discussed @chaokunyang in https://www.reddit.com/r/java/comments/1cll68f/comment/ld3suns/?context=3 and he is generally supportive. However, there seems to be no real guide for me to start implementing this. Perhaps someone else can do it in a shorter amount of time.

urlyy commented 1 month ago

@chaokunyang I'm interested in this issue. However , as it's my first time delving into the Java version of Fury, I'm not quite sure if my approach is correct. By using DEBUG, I've traced the code and found a method Descriptor::createAllDescriptorsMap(...) which obtains the list of fields of the object being serialized, and the descriptorMap ultimately participates in Serializer codegen. Here is already a check on the Ignore annotation; I need to modify the logic. Do you have some advice? To be honest I don't know if it's enough to modify this method , maybe I should explore more. And what's the name of the annotation? FuryOnly?Include?

image

Frotty commented 1 month ago

Cool, would be nice to see this implemented. I suppose there must be a configuration or mode, or annotation on the class itself, to switch the fury behavior to only serialize marked fields. Otherwise, might be quite straightforward. Currently, the annotation is @Exposed but I don't really care about the name.

chaokunyang commented 1 month ago

The implementation should not be complex, but the naming is an issue. This is a public API, how do we name this annotation?

urlyy commented 1 month ago

@Frotty We have added an annotation named Expose. And if fileds of a Class are annotated with both @Expose and @Ignore, it will throw a RuntimeException. We can only use one kind.

@Data
@AllArgsConstructor
private static class ExposeFields {
  @Expose int f1;
  @Expose long f2;
  long f3;
  @Expose ImmutableMap<String, Integer> map1;
  ImmutableMap<String, Integer> map2;
}

@Test
public void testExposeFields() {
  Fury fury = Fury.builder().requireClassRegistration(false).build();
  ImmutableMap<String, Integer> map1 = ImmutableMap.of("1", 1);
  ImmutableMap<String, Integer> map2 = ImmutableMap.of("2", 2);
  ExposeFields o = serDe(fury, new ExposeFields(1, 2, 3, map1, map2));
  assertEquals(1, o.f1);
  assertEquals(2, o.f2);
  assertEquals(0, o.f3);
  assertEquals(o.map1, map1);
  assertNull(o.map2);
}

@Data
@AllArgsConstructor
private static class ExposeFields2 {
  @Expose int f1;
  @Ignore long f2;
  long f3;
}

@Test
public void testExposeFields2() {
  Fury fury = Fury.builder().requireClassRegistration(false).build();
  assertThrows(RuntimeException.class, () -> serDe(fury, new ExposeFields2(1, 2, 3)));
}
Pil0tXia commented 1 month ago

Good job. Might we consider this issue as completed?

Frotty commented 1 month ago

I wasn't able to test it yet, when will it be available in a release?

chaokunyang commented 1 month ago

I wasn't able to test it yet, when will it be available in a release?

You can use snapshot jar for test. A release jar will be released by the end of this week