[Idea] Less-thick clients

[1fish2]

Moving more logic from the mobile app to the server would have several advantages:

• Share more code between TBA web, Android, and soon iOS and Windows Phone clients.
• Keep them all consistent about content details such as Bug #734 (recognizing the winner). Tournament rules are complex and year-specific.
• Allow faster deployment of bug fixes, esp. on iOS where it takes a week or two to push an update through Apple's approval process.
• Better support year-specific information, such as displaying Ranking Points instead of (or in addition to) quals match score points in a year with RPs.
• Reduce the QA work to do.

The idea is to move more of the domain logic and high level information design choices from the client code to the server code. Clients would implement platform-specific design details while letting the server provide data such as what fields to display, how to title them, which team numbers to mark specially as "favorite" and/or "surrogate in that match". The server would not specify layout details such as colors and margins.

That is, make the clients "less thick" (not "thin clients" like HTML viewers).

This doesn't force all clients to have all the same features, UI, navigation, or appearance. It does mean the information they display will agree.

Open design questions include:

• What info to include and exclude in the abstract presentation data?
• Should the client fetch the presentation data with the content data in each API response, or fetch it much less often? If the latter, the content data reply could contain a token for its needed (immutable, versioned) presentation data so the client will know if it needs to fetch that.
• Which API calls should aggregate content data of multiple models? Aggregation is OK as long as it doesn't interfere with fixing #638 . That is, each model that gets stored in a local DB row needs a last-modified timestamp so data rows received via different API calls/GCMs/Firebase do not replace older rows received from unexpired network caches.

I asked a friend if Google Now uses a "thin client" approach that sends content + layout info to the clients so it can support more flexible and dynamic layout without having to modify the mobile client code. The reply:

Yea, in the beginning all the cards used custom rendering, but now they've moved to a module approach where they have a set of predefined modules that can make up a card and the server just provides details for the modules for an individual card. The modules are fixed, so it's not super flexible, but if you define the modules well, you can still have a wide variety of card types while still keeping them pretty consistent.

This is interesting! It's not just presentation information abstracted back from client-specific details. It composes little modules of such information into the cards.

See More Server Side Config #715

See API v3 Discussion #1507, "look at trying to move things that currently require client-side computations to be part of the API proper."

[phil-lopreiato]

I'm absolutely on board with this - if it means we can deliver a better product, I see no reason why we shouldn't look into it.

I think #715 is going to play the biggest part in this - we don't want to include regular data and meta-presentation data in the same endpoints, so we're going to end up with a separate data source for the presentation part.

We may be able to take some inspiration from how Facebook handles configuration/releasing for their mobile apps.

• Paper about FB handles config management: https://research.facebook.com/publications/holistic-configuration-management-at-facebook/
• Great talk by Chuck Rossi, FB's releng guy: https://youtu.be/Nffzkkdq7GM?t=712 (not really about config, but a ton of great insight)

What info to include and exclude in the abstract presentation data?

I think we start simple - nav drawer items, current competition year, current competition week, etc. And then as we iron out bugs in the infra, we can ramp it up to include more.

Should the client fetch the presentation data with the content data in each API response, or fetch it much less often? If the latter, the content data reply could contain a token for its needed (immutable, versioned) presentation data so the client will know if it needs to fetch that.

In the paper above, there's a great section about how config schemas get updated, I'm a fan of doing something similar to that.

[nwalters512]

I'm all for moving more domain logic to the server; that should help a lot with some of the inconsistencies we've been seeing. However, I'm not convinced that having the server dictate presentation is the best option.

For example: take the team info screen. Let's assume for a second that we want to achieve the current layouts with server-provided presentation.

On Android, info is displayed in a series of cards split into sections, with most card sections having variously colored icons. The sponsors section is integrated into the middle of the first card, and it expands on click.

On iOS, info is split into a series of cells in a different order than on Android. There are no icons, the font sizes and weights are different, and some items are prepended with certain text that isn't necessary on Android (e.g., team location starts with "from" on iOS, whereas Android doesn't need that because of the location icon).

The obvious solution would be to converge the designs so we don't have to worry about two separate presentations. But then we could end up with one overarching design that doesn't feel particularly at home on either of the platforms. If we have two separate designs, then we have to either a) provide a separate presentation config for each platform, all of which we'd have to try to keep in sync, or b) special-case presentation logic on the client. Both of those put us roughly where we are now, if not behind.

There may be additional solutions I haven't thought of yet, and "Because it's hard" is definitely not a reason not to do something. That was just one concern I had, and I'd love to hear thoughts about that.

[phil-lopreiato]

Another concern is that there is no way to specify drawable resources in Android (and I think it's the same on iOS) via some string coming totally from the server.

The drawable ids are actually just constants, but since they can change in between builds, they need to be explicitly stated at compile time.

[nwalters512]

Android does have Resources#getIdentifier(...), but that doesn't solve the problem of a) maintaining a separate resource identifier for each platform, and b) what to do in the case where the server tells the client "hey, use this icon", but the client hasn't updated to the latest version and doesn't have said icon. Then we either have a) to maintain different presentations for different versions (ew) or b) serve icons as images from the server.

[phil-lopreiato]

Yeah. I think we should set a smaller goal for 2017, and that's just:

• Get as much domain specific logic on the server as we can. This mostly falls on the shoulders of APIv3. We should compile a list of all the places the apps do calculation work currently and make sure all of those situations are addressed.
• Build a system to handle config (app parameters and meta-data), but not presentation info. This will also allow us to have server-side rollouts for big features, so we can test things a little more easily. However, this system could be adapted to provide presentation info in the future.

If we can get those two things down and tested next season, adding in presentation logic should be a lot simpler than building it all from scratch. Plus, we'll have another year's experience of running a multi-platform service (with an iOS app and maybe more added to prod), so we'll know exactly which pain points we need to address, instead of guessing now.

[1fish2]

I like this plan.

Among these APIv3 considerations, the Android client has a lot of un-DRY code to do stuff like convert team names "frc1114" into team numbers "1114". Maybe the server should send team numbers without the "frc" prefix. (Keep it in string form to support multi-robot off-season teams.)

Is there a reason the iOS app shows the team fields in a different order? Does it show text labels instead of icons because icons haven't yet been implemented or because icons wouldn't fit the design style? There should be platform-specific styles but not gratuitous differences.

Between domain logic and presentation details is the choice of what to present. E.g. team@event could include the short team name. (Tapping a team number from a match or alliance screen gets there, and that's about the first thing the user wants to learn about a team.) E.g. match result data could include both RPs and point scores with (or in separate config data) a recommendation of how to prioritize those two when screen space is tight.

For the server to someday add fields with icons, the icons would have to be downloadable either at xxxhdpi resolution or at client-display resolution ... or SVGs if iOS can handle them. To reference icons already present, the client could look up persistent names with getIdentifier() as Nathan pointed out, and fall back to a default icon or no icon if the client is too old to have it. This is only useful with icons that are already in the library but it'd handle adding the team name to the team@event screen. We can defer this for sure.

An easy feature is to hide fields. Just omit them from the API data. Reordering fields would be a straightforward application of presentation data, but it doesn't sound useful.

[nwalters512]

Is there a reason the iOS app shows the team fields in a different order? Does it show text labels instead of icons because icons haven't yet been implemented or because icons wouldn't fit the design style? There should be platform-specific styles but not gratuitous differences.

I wasn't really advocating for keeping the designs separate. I haven't used an iOS device in years, and I've never used a Windows Mobile device, so I don't have an intuitive sense of what would or wouldn't look at home on each platform. It was more of a thought experiment: if we wanted designs to differ significantly between platforms, what would that require from some kind of externally-specified presentation?

Among these APIv3 considerations, the Android client has a lot of un-DRY code to do stuff like convert team names "frc1114" into team numbers "1114". Maybe the server should send team numbers without the "frc" prefix.

I can't speak about why the decision was made initially to use keys instead of numbers. To me, it seems like a form of database normalization. There are certainly cases where both the key and the team number are useful in an application. Choosing which one to include seems kind of arbitrary, and changing from the key to just the number would be a remarkably breaking change for any consumer of the API. I'm not sure if it's worth it.

[1fish2]

Yes, serving presentation data presumes we'd rarely want the client designs to differ significantly. It gets messier if we do.

[nwalters512]

Serving presentation data presumes we'd rarely want the client designs to differ significantly.

I don't know if that's an assumption I'd be willing to make for every feature we might build and every platform we might expand to in the future.

[1fish2]

Then maybe we won't serve presentation data.

Some variations are straightforward, like separating fields to include/exclude on a tiny screen (smart watch) or just prioritizing fields to fit in.

[nwalters512]

Absolutely. Sorry if I appear overly hostile to the idea. My goal isn't to shoot it down entirely. I'm just trying to evaluate if it's necessary, a viable solution for the long term, and worth the engineering effort to implement.