search

mapbox / mapbox-gl-style-spec

76 stars 38 forks source link

Mark all features which are not supported in all SDKs as experimental & API unstable #653

Closed lucaswoj closed 7 years ago

lucaswoj commented 7 years ago

Credit for this elegant design goes to @jfirebaugh 🙇

When we add new features, especially when we're in the process of porting features across platforms, we often need to revise our initial API design.

We currently do a poor job of communicating to our users which APIs are stable and which APIs are likely to change.

We should prominently mark that any feature which is not supported in all SDKs as "experimental" in the SDK documentation. We should explain that an experimental feature's API may change in a future release.

1ec5 commented 7 years ago

We should prominently mark that any feature which is not supported in all SDKs as "experimental" in the SDK documentation. We should explain that an experimental feature's API may change in a future release.

How does this relate to the SDKs’ use of semantic versioning? The Android and iOS SDKs and node-mapbox-gl-native are all past v1.0, which means a change to an “experimental” API would require a major version bump, just like a change to any other API. The iOS SDK has carved out some wiggle room for undocumented APIs, but this issue seems to be about documented features.

jfirebaugh commented 7 years ago

The proposal is to exclude features marked as experimental from semantic versioning requirements. They would be free to change in backward-incompatible ways without a major version change for the SDK as a whole.

1ec5 commented 7 years ago

They would be free to change in backward-incompatible ways without a major version change for the SDK as a whole.

I’d be against a general strategy of marking new APIs as unstable. Backwards compatibility is a big deal for iOS and macOS applications. As far as I can tell, the culture around CocoaPods, Carthage, and Cocoa frameworks in general doesn’t allow for parts of the API to be exempted from semantic versioning requirements. There’s no such thing as an “alpha class” or “beta method”. We have three strategies for mitigating this inflexible interpretation, in order of preference:

These strategies aren’t a replacement for striving to release a stable API for each new feature.

1ec5 commented 7 years ago

I neglected to mention a very important fourth mitigation strategy: prereleases. Android and iOS SDK releases spend a lot of time in gestation. Alpha and beta releases allow us to finalize a feature’s design and implementation before we expect developers to rely on it in production code. The iOS SDK has specific criteria for when a build qualifies as an alpha, beta, or release candidate. With a prerelease process in place, there should be no need to mark a feature as “experimental” and “unstable” in a final release.

lucaswoj commented 7 years ago

I neglected to mention a very important fourth mitigation strategy: prereleases. Android and iOS SDK releases spend a lot of time in gestation. Alpha and beta releases allow us to finalize a feature’s design and implementation before we expect developers to rely on it in production code.

While a feature may be well designed and stable within the context of the iOS SDK, porting the feature to another platform often inspires / requires design changes.

Leaving an experimental API undocumented

This sounds like a solid approach.

  1. A majority of the time, features are implemented in GL JS first and will have their "experimental" designation lifted by virtue of their implementation in the native SDKs.
  2. The rest of the time, we create a healthy pressure on ourselves to get parity before doing a SDK release.
  3. The rare times we can't get parity before an SDK release, we'll include an undocumented API to be honest about the API's stability
1ec5 commented 7 years ago

To reiterate, undocumented APIs might as well not exist: hypothetically, no examples about alternative projections would be published until alternative projections are supported on all platforms, and any blog posts would avoid implying that developers can try out the API. We’d indeed create pressure to get parity on each platform before calling alternative projections done.

The downside is that Web developers would be highly unlikely to use the undocumented API, so there would be less feedback from developers that could help us refine the implementation. The solution is for GL JS to publish prereleases – that was my point in https://github.com/mapbox/mapbox-gl-style-spec/issues/653#issuecomment-273316475. Ensuring feature parity across platforms entails slowing down GL JS’s release cycle (but adding prereleases mid-cycle). I think v1.0 would be a good opportunity to formalize a prerelease strategy for GL JS.

jfirebaugh commented 7 years ago

I don't think it's tenable for GL JS to publish all releases as "prerelease" in periods where it has features that the mobile SDKs do not, and where the implementation in mobile SDKs is likely to require specification or API design changes. Concretely, that would mean that GL JS would have to be on "prerelease" status right now, and for the past year or more.

@1ec5 Can I kindly ask you to reconsider your resistance to at least trying out the proposal here? We have to introduce flexibility somewhere. I believe that it's worth trying out the flagged approach, even though it may deviate from certain cultural norms. As @lucaswoj notes, in most cases it's the GL JS implementation that will come first and be flagged as experimental, and where the reverse is true we can test the waters with small features first. If it doesn't work, we'll find out and try something else. But let's not be afraid to experiment (heh).

1ec5 commented 7 years ago

Concretely, that would mean that GL JS would have to be on "prerelease" status right now, and for the past year or more.

Right, we’d obviously need to avoid a prolonged beta testing phase. This speaks to the “healthy pressure” @lucaswoj refers to.

Can I kindly ask you to reconsider your resistance to at least trying out the proposal here? We have to introduce flexibility somewhere. I believe that it's worth trying out the flagged approach, even though it may deviate from certain cultural norms.

Perhaps I’m misunderstanding the purpose of this ticket. I understood the idea to be that every SDK would at times have to introduce an unstable API, documented as such but otherwise identical to a stable API, until a compatible API is implemented across all six platforms. The iOS SDK has to be a good citizen on its platform, and part of that is abiding by semver as developers understand it. I’m just the messenger.

If we document a documented API in the iOS SDK and tell developers to use it (whether in examples, blog posts, or compatibility tables), the API isn’t exempt from semver no matter how much we claim the contrary. After all, if a developer requires version ~> 3.4 of our pod and follows our examples to the tee, using this unstable API, their code breaks as soon as we release v3.5.0 with a backwards-incompatible change to the unstable API. As soon as they run pod update, they wind up with v3.5.0 and a build failure or, worse, unexpected behavior at runtime.

The status quo of releasing a feature to GL JS first seems to work well regardless of whether we document the feature as being unstable. The reverse, where a feature comes to a native SDK first, requires us to try to anticipate future porting problems during beta testing. But we aren’t perfect at predicting the future, so that’s where the three strategies in https://github.com/mapbox/mapbox-gl-style-spec/issues/653#issuecomment-273311404 come in, especially the first strategy of providing compatibility shims.

If prominently marking “experimental” APIs works for GL JS, that’s great. But I don’t think the proposed approach can be applied to the iOS SDK.

lucaswoj commented 7 years ago

This issue was moved to mapbox/mapbox-gl-js#4171