search

square / wire

gRPC and protocol buffers for Android, Kotlin, Swift and Java.
https://square.github.io/wire/
Apache License 2.0
4.24k stars 570 forks source link

proto3 Working Document #279

Closed swankjesse closed 4 years ago

swankjesse commented 9 years ago

As kicked off by issue 210, here’s what we need for proto3:

Goal: All done September 1?

vanvlack commented 8 years ago

@swankjesse looks like proto3 support got bumped to 2.1?

swankjesse commented 8 years ago

That's our current plan. What features of proto3 are you most excited about?!

vanvlack commented 8 years ago

@swankjesse all the things. JSON support will be cool, but not having to rely on optional/required will be cool.

rzimmer commented 8 years ago

@swankjesse Support for Any to replace removed extension support would be great.

vanvlack commented 8 years ago

@swankjesse now that proto3 is official has been released, what's the timeline on Wire supporting it in full?

swankjesse commented 8 years ago

I can't promise any specific timelines.

Proto3 requires some big changes to the runtime. We’ll need to use int instead of Integer for example, which may complicate some things. It's also not easy to migrate from proto2 to proto3, which will allow our urgency to migrate.

vanvlack commented 8 years ago

It's also not easy to migrate from proto2 to proto3

Understandable. Would you think about selectively supporting proto3 syntax/features over time, vs a large push?

JakeWharton commented 8 years ago

We're already doing that.

vanvlack commented 8 years ago

We're already doing that. -- @JakeWharton

Do you have a priority list around what you are pulling in or when? I ask as some things like the following would be nice to see at some point:

ShreyK commented 7 years ago

Are you continuing to add support for proto3?

JakeWharton commented 7 years ago

We continue to plan to add support for proto3. Unfortunately there's not a lot of momentum around it.

On Thu, Mar 16, 2017 at 2:51 PM Shrey Khosla notifications@github.com wrote:

Are you continuing to add support for proto3?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/square/wire/issues/279#issuecomment-287156197, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEEEZjZupQfjT0GCqNo1jMc1DxsC7ilks5rmYSmgaJpZM4FcOln .

jverne commented 7 years ago

What would "JSON Support" provide when working with, say, Retrofit? Putting aside the data representation specifics (i.e., maps) who is the target audience?

I think this means that we could use Wire directly as the Retrofit convertor (instead of using Wire to create classes that are then consumed by something like GsonConvertorFactory). That is, I could more closely model the JSON that is the source of the truth, and not have to worry what happens if Gson can't grok something. (Which has not happened, but it keeps me up at nights.)

I fully admit I might be missing the point, but if this is the general gist then I vote JSON support.

swankjesse commented 7 years ago

I’ve got reservations around proto3’s JSON mapping. It’s not completely trivial, and also incompatible with the way Wire does Gson. For example, it recommends mapping field_name to fieldName which I think is needless complexity.

... I’ve also got more general reservations around proto3. It conflates absent with zero, which is very incompatible with proto2.

jverne commented 7 years ago

Yeah, I'm not a fan of the name mangling. My proto, which I use to create classes from a very complicated JSON structure (like over 10k lines of generated code) uses the exact field names, and ideally I wouldn't have to change that.

I'm OK with wire just being a build step. I'm just wondering what the other wins are for JSON support.

fluxxion82 commented 7 years ago

I ran into a problem with using the proto3 schema, which you may already be aware of. Absent String fields are still trying to get encoded and this is causing a NPE.

rickkln commented 6 years ago

Does the above issue still reflect the current status of proto3 on wire?

swankjesse commented 6 years ago

We’re going to resume work on it.

rickkln commented 6 years ago

Great! Thanks for the update 👍

DHosseiny commented 5 years ago

Scalars become non-nullable, but message fields remain nullable.

dkiefner commented 5 years ago

Hello, do you have a timeline/roadmap when proto3 support will be available? We're about to assess if we want to wait for Wire3 to be released or using Googles runtime with protobuf-javalite to use proto3 in our apps 🙂

swankjesse commented 5 years ago

I think proto3 is a regression. We aren't rushing to support it in Wire.

rickkln commented 5 years ago

If you're not actively working on supporting it, it might be best to close this issue and document that status in the README? We delayed a project to remove wire from our app based on the previous message in this thread (in September) that yall were resuming work on proto3. Given the news that you're actually not working in it we probably have to take another look at moving away from wire and I'm sure the information could be similarly helpful to others.

swankjesse commented 5 years ago

Sorry for the thrash, we’re trying to find a nice way forward. The surface-level differences between proto2 and proto3 are small, but the behavior-level consequences are very large.

rickkln commented 5 years ago

No problem, thanks for making open source stuff 😃

Agree that the "behavior-level consequences are very large". That is the problem for us, as on iOS/Web/Backend/API levels we use proto3, on Android we ended up with proto2 because the team went with wire not realizing how big this gap is. This does cause us issues every now and then and the choice is basically waiting for wire to add proto3 or moving away from it.

malachid commented 5 years ago

Is the checklist in the description currently up-to-date?

mellster2012 commented 4 years ago

+1 for full proto3 support - thank you!

Clement-Jean commented 4 years ago

How can I start helping on that ?

sweetbot commented 4 years ago

Are we still expecting proto3 support in Q1 2020? If not any time soon, could you please suggest an alternative solution to Wire, for generating Kotlin files from proto3 files in Android project during compile time? Because all of our backend system and iOS are using proto3. Thanks.

ZacSweers commented 4 years ago

@sweetbot nothing in this thread said "Q1 2020"

garyp commented 4 years ago

@sweetbot You might want to check out https://github.com/streem/pbandk (disclaimer: I'm one of the developers). It supports proto3 and generates Kotlin multiplatform code (currently supports Kotlin/JVM and Kotlin/JS; Kotlin/Native is still to do). It's not as mature as Wire, but we'd love some community contributions! We use it in production and all of the basic protobuf features are supported.

Also, I have the utmost respect for the Wire library and its authors. We probably would have ended up using Wire if proto3 support had been available. I'm only mentioning an alternative here because someone asked. There's a lot of similarity between the two libraries, but there are also places where we've made different design decisions, especially in how to best represent things in Kotlin.

Tooto commented 4 years ago

all stuff about proto3 is not available for developers, right?

oldergod commented 4 years ago

We're aiming at releasing a RC in 2 weeks.

nlgtuankiet commented 4 years ago

Just 1 question, for now, I'm using proto2 message syntax, I'm encoding a message a save it into a file (and also decoding a message from that file). When switching to proto3 syntax, can I still decode that file?

Tooto commented 4 years ago

We're aiming at releasing a RC in 2 weeks.

Is there any alpha version right now? I should start project at monday, so i need to make a decision which library to use. I really want to use wire

kalyaganov commented 4 years ago

Hello guys! Any update on this?

nlgtuankiet commented 4 years ago

Please don't thumb down me :( I just want to know when this release or there is any pre-release version that we can try at this time

JakeWharton commented 4 years ago

Updates about the availability of proto3 will be posted when they are ready. Feel free to hit "subscribe" on the issue to be notified of such events. Until then, asking will yield no response.

nlgtuankiet commented 4 years ago

Thank you guys for all the hard work, we all appreciate it

sweetbot commented 4 years ago

Hooray! I think the announcement is pretty close.

yschimke commented 4 years ago

Not sure where to put some random trivia, so stuffing it here for google to index. I ended up talking to a server using Jackson, using wire/proto3. It ends up being a very atypical situation where the java kotlin class uses "parsed_command", but the proto3 json uses "parsedCommand". Common sense would suggest the opposite is more common. Anyway for Jackson I needed this to fix the handling of the wire code generated classes.

class WireProto3PropertyNamingStrategy : PropertyNamingStrategy.PropertyNamingStrategyBase() {
  override fun translate(propertyName: String): String {
    if (propertyName.contains('_')) {
      return propertyName.replace(regex) { matchResult ->
        matchResult.groups[1]!!.value.toUpperCase()
      }
    }

    return propertyName
  }

  companion object {
    val regex = "_([a-z])".toRegex()
  }
}