[Accepted with Revisions] SDL 0216 - Widget support

[jordynmackool]

Hello SDL community,

The review of the revised "SDL 0216 - Widget Support" begins now and runs through May 14, 2019. Previous reviews of this proposal took place March 20 - April 9, 2019 and January 23 - February 26, 2019. The proposal is available here:

https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0216-widget-support.md

Reviews are an important part of the SDL evolution process. All reviews should be sent to the associated Github issue at:

https://github.com/smartdevicelink/sdl_evolution/issues/664

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of SDL. When writing your review, here are some questions you might want to answer in your review:

• Is the problem being addressed significant enough to warrant a change to SDL?
• Does this proposal fit well with the feel and direction of SDL?
• If you have used competitors with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study? Please state explicitly whether you believe that the proposal should be accepted into SDL.

More information about the SDL evolution process is available at

https://github.com/smartdevicelink/sdl_evolution/blob/master/process.md

Thank you, Jordyn Mackool

[Jack-Byrne]

Apps can create widgets from any HMI level as allowed by policies (e.g. from HMI_NONE or BACKGROUND) and send Show or SetDisplayLayout to add content (text and soft buttons) to that widget. The RPCs sent to the widget follow policies of HMI_LIMITED.

Note that policies are OEM specific. What RPCs are allowed to be sent in certain HMI levels are defined by the oems policy table. Existing policy mechanisms could be used for the configuration of CreateScreen to be allowed in NONE or BACKGROUND.

However, if you want to define different permissions for a when a show to a widget is allowed vs show to the mains screen is allowed, this will require new policy behavior that does not exist yet.

If a user taps on a soft button in the widget and the app main screen is in HMI_NONE the app will automatically transit to HMI_BACKGROUND.

I believe an app is put into background after it has been activated and after a different app is put into the foreground.

Separately, if an app is activated but then the main screen if navigated back to the app list, then the app is put into LIMITED. I think what hmi level the app is put into after selecting the widget should depend on what state the main screen is in before widget selection.

[Jack-Byrne]

The RPC Show and SetDisplayLayout are identified as the only screen related RPCs.

Alert, ButtonSubscription, AddCommand/Submenu, PerformInteraction, Slider, Scrollable Message, and Send haptic data are additional RPC that change how the HMI presents its current view. I think these should be considered for adding the screenID parameter.

Also I think that user interaction rpcs should also be considered (onButtonPress, onAddCommand). Core/mobile will need to know the context of a selection when managing the behavior of user input.

This is especially important for the future use case of having multiple screens. These RPCs will need the ability to specify which screen the rpc is meant for / originated from.

[Jack-Byrne]

All this can be reused for widget screens. When creating a new widget screen the available widget templates are provided in CreateScreenResponse.displayCapabilities.templatesAvailable.

I think it would be worthwhile to spend some time designing new template types that are meant specifically for widgets instead of trying to reuse the old template layouts.

[Jack-Byrne]

Alternatives considered: Screen related OnHMIStatus

I like this idea. I think different views on multiple displays should definitely have their own hmi state. Adding screenID to onHMIStatus would be the first step in making that possible. However I think there should be a separate hmi level meant for widgets. A specific hmi level for widgets would help with managing rpc policies/app permissions.

[Jack-Byrne]

On a side note we have done some work in the past for allowing multiple hmi connections to core at the same time.

https://github.com/smartdevicelink/sdl_broker/blob/master/index.js

This is a simple node app that works as a router of sorts for allowing multiple hmi clients (displays) connect to core at once. Each HMI client registers certain components that they have implemented (subset of UI, vehicle info, remote control, TTS, VR, etc). When core is sending messages to the hmi broker, the hmi clients only receive the messages for components they have registered/subscribed for. This solution makes it possible to do widgets/multiple displays (minus the functionality that having screenID would provide as outlined in the proposal).

[Jack-Byrne]

Additional side note related to the App Service Proposal

The IVI system itself can be considered an app service consumer as one of the greatest advantages of this proposal is to augment IVI systems with additional services that it might not be built with or become obsolete.

According to that proposal it is possible to have the hmi act as a "AppService Consumer". I believe the main intention of this is for allowing "widgets" to display information and interact with a published service.

For example the HMI might implement a media consumer that is displayed at all times on the cluster. This would allow the display of relevant media meta data at all times no matter the active app or hmi state.

[kshala-ford]

Note that policies are OEM specific. What RPCs are allowed to be sent in certain HMI levels are defined by the oems policy table. Existing policy mechanisms could be used for the configuration of CreateScreen to be allowed in NONE or BACKGROUND.

100% Agree. As this is OEM specific it was not included in the proposal.

However, if you want to define different permissions for a when a show to a widget is allowed vs show to the mains screen is allowed, this will require new policy behavior that does not exist yet.

Alternatives considered: Screen related OnHMIStatus

I like this idea. I think different views on multiple displays should definitely have their own hmi state. Adding screenID to onHMIStatus would be the first step in making that possible. However I think there should be a separate hmi level meant for widgets. A specific hmi level for widgets would help with managing rpc policies/app permissions.

We at Ford had a long conversation about how to solve HMI levels. Adding a new HMI level may be very risky and break existing implementation. I'm not so sure if we actually need a new HMI level in the enum. The existing elements also apply to widgets. This change is mandatory once we have multi display support (subsequent proposal coming soon).

If a user taps on a soft button in the widget and the app main screen is in HMI_NONE the app will automatically transit to HMI_BACKGROUND.

I believe an app is put into background after it has been activated and after a different app is put into the foreground.

I guess SDL should send ActivateApp with hmiLevel=BACKGROUND to HMI if the app was not activated (therefore currently in HMI_NONE).

Separately, if an app is activated but then the main screen if navigated back to the app list, then the app is put into LIMITED.

This is only the case for media and navigation apps. Not for default apps.

I think what hmi level the app is put into after selecting the widget should depend on what state the main screen is in before widget selection.

The list of 5 requirements is actually trying to do that. We considered all HMI states and app activation in our requirements. It contains requirements for app (and main screen) states where SDL/HMI should react. It does not contain requirements for HMI levels where no action is required. I could add more lines explaining other levels for clarity.

Alert, ButtonSubscription, AddCommand/Submenu, PerformInteraction, Slider, Scrollable Message, and Send haptic data are additional RPC that change how the HMI presents its current view. I think these should be considered for adding the screenID parameter.

They might change the HMI but they are not screen specific. All overlays are display related. MenuParams and AddSubMenu make sense in a multi display environment with having multiple DEFAULT screens on other displays. All RPCs can be extended with the subsequent proposal about multi display support. Right now I don't see the need to modify them.

Also I think that user interaction rpcs should also be considered (onButtonPress, onAddCommand). Core/mobile will need to know the context of a selection when managing the behavior of user input. This is especially important for the future use case of having multiple screens. These RPCs will need the ability to specify which screen the rpc is meant for / originated from.

Hmm, good point. They may be covered by the unique button/command ID. Uniqueness should be throughout all screens and displays.

I think it would be worthwhile to spend some time designing new template types that are meant specifically for widgets instead of trying to reuse the old template layouts.

Well... we will design completely new templates but at the same time we try to avoid adding new names. The existing template names should be reused but the OEMs can design how the template with a certain name should look like.

On a side note we have done some work in the past for allowing multiple hmi connections to core at the same time.

I also thought to have an alternative with multiple registrations. The idea is smart but didn't continue with this direction as it comes with some downsides:

• It may require multiple app ids
• Lot of RAI params are unneeded for widgtes
• Widgets would have been another HMI type
• You don't know about multi display or widget support until you registered once (may be this is not really a downside but it adds complexity)

Additional side note related to the App Service Proposal

If I'm understanding correctly we are actually planning something like what you mention. The service proposal will definitely be used together with widgets. This proposal will not be the last one as we want to extend SDL to other displays and extend capabilities for apps to gain more control and provide better and customized user experience.

[BrettyWhite]

I know the proposal states that mobile changes aren't covered in this, however I think that design needs to be included for a proper review of this proposal.

For mobile libraries it's necessary to add the new RPCs and add support for multiple screen managers.

I think this part needs to be included as part of the proposal. It is too "hand-wavey" to just say "add support for multiple screen managers". As the screen managers themselves are fairly sizable in what they do I would really like to see this part detailed out and included.

[kshala-ford]

@BrettyWhite you're right. We should have mentioned that the proposal focuses on RPC level only. The sentence was meant to only clarify how we plan to specify additions/modifications to Android and iOS managers in another subsequent proposal once this proposal is completed. Trying to avoid building a proposal on top of another unaccepted proposal. Are you OK with this approach?

[BrettyWhite]

Trying to avoid building a proposal on top of another unaccepted proposal.

I agree, but this is backwards. If multiple screens support is a requirement of widgets, that should be agreed on and accepted first.

[joeygrover]

So there's a lot to unpack from this proposal. I think I'll start with the good.

1. Multi-screens. Yes! This is an area that has to be explored and fully featured for SDL to truly be the industry leading connectivity choice. It's something projection modes simply can't do and SDL has the opportunity to do it very well.
2. Widgets: another yes! Extending an app into more of the IVI system beyond their templated box will really help IVI systems become more custom to users and present a much deeper integration.
3. Ensuring features we need now are setting us up for how we do things in the future.

I agree 100% on these aspects of the proposal. Overall I think this proposal is extremely verbose and wide spanning which causes it to be a bit hard to follow. While I agree it's important that the proposal is setting up for future features, it seems as though too much of the "future" proposal is needed to really make this feature work and the generic nature of its pieces seem right. I don't think it's fair to vote on things like Create/Delete screen that have much larger implications and how that plays into other RPCs in a proposal solely about widgets. I imaging the intention (and I could be wrong) of these RPCs will be used to create additional screens on different monitors/displays as described here:

An app can have multiple main screens depending on the displays e.g. main screen on the central console and main screen on rear seat entertainment system. However, it should not be allowed to have multiple main screen on a single display. Main screens follow HMI levels and permissions.

To me, this is where a major leap happens. The amount of work required to create the logic to handle such a change simply can not be vaguely described. Therefore, I wouldn't want to agree on some of the groundwork for such a feature in a proposal that doesn't include those details.

Alert, ButtonSubscription, AddCommand/Submenu, PerformInteraction, Slider, Scrollable Message, and Send haptic data are additional RPC that change how the HMI presents its current view. I think these should be considered for adding the screenID parameter.

They might change the HMI but they are not screen specific. All overlays are display related. MenuParams and AddSubMenu make sense in a multi display environment with having multiple DEFAULT screens on other displays. All RPCs can be extended with the subsequent proposal about multi display support. Right now I don't see the need to modify them.

All of those RPCs and user interaction RPCs will need to contain which screen they are designated for. As it seems the multi display feature would allow users to be in different aspects of the app's UX it only makes sense that things like Alerts and PICS have specific screens they are assigned.

I think it would be worthwhile to spend some time designing new template types that are meant specifically for widgets instead of trying to reuse the old template layouts.

Well... we will design completely new templates but at the same time we try to avoid adding new names. The existing template names should be reused but the OEMs can design how the template with a certain name should look like.

I would be worried that we will continue the disparity on UX between IVI systems here. I would think widgets need much stricter control over their layouts as they are a very small space that should be even more aware of driver distraction guidelines and differences between IVI systems could cause the app to create separate widget integrations for OEMs.

Additional side note related to the App Service Proposal

If I'm understanding correctly we are actually planning something like what you mention. The service proposal will definitely be used together with widgets. This proposal will not be the last one as we want to extend SDL to other displays and extend capabilities for apps to gain more control and provide better and customized user experience.

For me this is the biggest issue I have with the proposal. That is because in my opinion app services are the perfect way for IVI systems to create widgets. They can even create widgets based on which app service is being used to provide the data. This prevents SDL from making promises on how the UX will be handled but also doesn't put that burden on SDL to try to maintain all the different OEM implementations. There are a few modifications that could really help the app services feature make widgets more feature rich, but I think they are easily added.

~If we don't use app services for widgets then I think it's important that we state that's not what app services are for. My fear is that if we don't do one of the two things we will create two different features that can accomplish the same goal but only work in a certain percentage of situations each. It's never a good idea to create multiple solutions to solve a single problem.~ EDIT: I have actually changed my thinking here. I believe using a larger more controlled proposal alongside the app services proposal is not only possible but could be harmonious.

[jordynmackool]

The Steering Committee voted to reject this proposal on 2019-01-29 based on the amount of feedback given in the comments. The author plans to rework the proposal in a new format and resubmit the updated proposal for review.

[joeljfischer]

1. One thing I think this proposal could use is use-case examples. For example, how would Accuweather and WebEx use this proposal (since you have them in your mockup near the top)? Or a music app, navigation app, or POI app like Yelp? e.g. Would WebEx create a widget when a call connects for the in-call info? Would they have another?

   This might also be something where this proposal could be better married with App Services (e.g. WebEx became the active service for VOIP and is in the middle of a call, then there's an API for WebEx to recommend this widget be active, or AccuWeather is the current weather service, so let's show their widget).

   There are lots of possibilities, but use-cases would help us ensure that the use-cases are satisfactorily useful to do all of the work required, and that all APIs needed to enable those use-cases are available.

2. The duplicateScreenId needs to be further fleshed out. Is this handled by Core? HMI? What if fewer text fields, image fields, or soft buttons are available? I assume extras are dropped? Are text fields concatenated?

3. You state in the intro:

   Providing display information to the app should be done by extending GetSystemCapability with a DISPLAY capability type. Each display should be identified with a system defined displayID so that apps can send requests for overlays to a specific display.

   However, this is never fleshed out in the Proposed Solution, and you seems to provide some of this info in the CreateScreenResponse. Could you clarify?

4. I assume that creating a screen with an ID of a PredefinedScreen should send an error response from Core?

5. You note:

   Audio streaming state is not related to screens but to system's audible state. To limit the changes in the RPC and support backward compatibility, audioStreamingState should be provided to all screens of one app. HMI and SDL core must make sure to send the same audio states to all screens. Example: If a media app has created a widget and then becomes audible, the media app should receive two OnHMIStatus notifications for both screens (main and widget) and both audio streaming states are set to AUDIBLE.

Why is this? Why can't the main screen just be the arbiter of audio? I think all the other parameters make sense for either / or, but audio is kind of screen-agnostic, is it not? I don't see why it would be necessary to be sent 2 or more times? Unless you're thinking of future-proofing for multiple "main" screens (e.g. rear seat entertainment that has different audio output than the main screen of the car)? In which case I think only FULL screens would get this change, and not WIDGET ones.

6. You note:

   The parameter system context makes sense for main screens to inform an app if the user entered the menu or started VR session. Knowing the system context of a widget may not be necessary. Therefore every widget's systemContext is always MAIN.

I don't know that this should be a hard line. For example, if a widget has a VR button on it (e.g. Alexa), and the main screen is in HMI BACKGROUND, then the widget would get the SystemContext VRSESSION, and not the main screen, right?

7. You note that a widget should have HMI_FULL. What about the main screen app? And what if the main screen app's content is being duplicated?

   a. What happens if you send a Show to a widget that's displaying duplicate content? It should be rejected, right?

   b. Should the main screen be in HMI LIMITED if the widget is displaying duplicate content?

   c. I see some comments above about this, but can't find it in the current proposal. What happens if (with non-duplicate content), a widget appears? Should the main screen still be in HMI_NONE? I would think at least HMI_BACKGROUND, right?

   d. I think it makes more sense for a widget to max out in HMI_LIMITED, and not have access to HMI_FULL because it cannot access the full range of HMI manipulation RPCs (e.g. Alerts, AddCommand, etc.).

8. You say:

   The details about the currently selected widget template are provided using the same display capabilities struct. It is expected that the HMI will return with fewer text fields (only main field 1 and 2), shorter text field width and smaller image resolutions. This solution reuses the existing API, to provide widget template details as mentioned.

   I'm assuming that this is a mere suggestion, and not enforced in any way.

9. You note:

   In order to inform the app what screen types are supported, the struct DisplayCapabilities should be extended with a single optional parameter called screenTypeCapabilities.

   Could this be done via changes to GetSystemCapability instead? I'd prefer not to continue to bloat RegisterAppInterfaceResponse.

10. You say:

    With above modification of OnHMIStatus the existing policies are compatible to widgets. The policy manager should be modified and use the HMI level of the main screen to evaluate permissions for all RPCs, except screen specific RPCs. In this case the HMI level of the targeting screen should be used for evaluation.

    @nickschwab Can you confirm this? I would think that with this feature, changes may need to be made for policies to allow different policies per ScreenType?

[kshala-ford]

1. I understand you point. As you say, there are a lot of possibilities. for conference calls the app developer can provide information about current or next conference, avoid double mute, show who's currently talking, it even synchronizes with the app's calendar in case of being invited to a meeting. VOIP is a different category and doesn't fit to conferences. The weather app could show me other locations and notify me about surf weather, calendar apps could show me the next meeting I have to attend, News apps can show me the breaking news, Navigation apps can offer shortcuts to e.g work in the morning and home in the evening but also to other locations I frequently visit. text messaging apps can show me if I receive a message (or course they do with Alert but this is a transient overlay). social apps can show me locations of friends. Other apps can show me upcoming reservations (restaurants, booked parking). ... ....

The whole point is that any kind of application can choose to provide brief information and me as a driver, I can have a look at all these apps very quickly in a widget view which is helpful instead of looping through all these apps.

2. I think the HMI has to deal with duplication. If Core would do duplication it would be faking requests and handling responses to fake requests.... it sounds unprofessional having Core dealing with this... And yes: just as HMI is doing it today it would just not show content if the field isn't available (if mainField4 is not shown on the HMI but the app provided it it would just be ignored).

3. the DISPLAY system capability is not used in this very proposal. It's part of the motivation and may be proposed with multi display support.

4. Correct. Creating a screen with predefined screen id returns in unsuccessful response. Result code is INVALID_ID.

5. Audible state is app (system) related. As we are still on single display and single (main) screen, audio playback can still be seen on system level. As the parameter is mandatory we wanted to keep the parameter value consistent throughout all screens on the display.

6. The system context enum is mixed with main screen types (MAIN, MENU, 'ALERT') and system types (VRSESSION, 'HMI_OBSCURED'). To your VR example: Today, if a driver presses PTT, every non-HMI_NONE app gets VRSESSION. Alerts or other overlays don't necessarily obscure a widget. Anyway as the main screen receives a valid system action, we have not expected the widget but be included.

7. Just as app main screens do today, widgets should also follow HMI levels. Regardless of duplication the HMI level of the targeting screen's HMI level is important.

   a) An app may create a widget which duplicates from the main screen. The user may close the app, hence the main screen transit to HMI_NONE. However the widget is still independent and may be visible and therefore have HMI_FULL. Still, the developer cannot send a show to the main screen to be duplicated to the widget because the main screen's HMI level doesn't allow it (expecting Show not being allowed in NONE). When writing this proposal we have discussed internally if it should be possible to override the duplicating by sending a Show to the widget directly. This override is invalidated as soon as the app successfully updates the content of the main screen... It was dropped as too complicated but I'm open to have this discussion regarding duplication details.

   b) I don't think we should do this. This would be a very invasive to the today's HMI state machine.

   c) We had something similar to it in the old version of the proposal. Interacting with the widget, transit the main screen to HMI_BACKGROUND but there's no HMI API, to do so. So we removed it. My suggestion would be to extend OnAppActivated not only with a screen ID but also with an ... optional silent flag. If this flag is true the app's screen won't be activated to the full but only to the background.

   d) If RPCs are not specifically addressed to a screen, they follow policy rules of the app's main screen.

8. I assume you mean the second sentence? No it's not enforcing anything.

9. Yes of course. However, this addition is adding a parameter to DisplayCapabilities, not to RAI response

10. I have to note that policies are independent of screen type. Widgets won't receive different policies. The same rules apply to any type. However, widgets and main screens may be on different HMI levels at one point of time.

[joeljfischer]

Sorry for the lengthy comments, there's just so much in here to discuss.

1. Do you understand my point about integration with app services? e.g. What if the user doesn't want a WebEx call widget on the screen at all times, but when in the middle of a call, it makes sense? Or what if the OEM just wants to display the user's preferred weather app's widget automatically? I think an integration here makes sense. @joeygrover Would that mostly work HMI side, or would there be an RPC component to this? Like for the WebEx example, I imagine they might have to decide on a "preferred widget" or something to give it that prominence? See some additional reasoning on point 11.

2. Hmm...okay. I suppose it makes the most sense that the HMI component would have to deal with this.

3. It worries me a little bit for it to be a part of a future extension of the proposal. I don't want to get stuck with two ways to do the same thing: one for widgets, one for full screens.

4. 👍 That should be part of the return / accepted revisions.

5. I'm not really following you here. The audio streaming state is guaranteed to be the same between the widget and the main app, so why must it be sent multiple times? I understand that it's a mandatory parameter and would be sent along if, for example, the HMI level changes, but I don't see why we need duplicate notifications.

   I'm a bit concerned about developers not knowing what to do with near-duplicate onHMIStatus as well. Additionally, the mobile managers would also need to be updated for their methods like func hmiLevel(_ oldLevel: SDLHMILevel, didChangeToLevel newLevel: SDLHMILevel). Either they need a new parameter, or they need to only relate to the main screen app. This should be noted in the proposal.

6. I'm not sure that I'm following you. Why would this be any different than (5) above? You state that it's assumed that it would be included, but not the audioStreamingState which is guaranteed to be the same as the main app? I don't think it's necessarily the case that it is always the same. e.g. An alert pops up on the screen over the widget, or the widget is otherwise obscured. I think it's the reverse for points 5 and 6 as you have them. The audio status is always the same between widget and app, but the system context doesn't seem to be.

7. a. This sounds like an important, currently undefined case. I would say that if a CreateScreen is sent to duplicate main screen content, sending any RPC directly to that widget should be Rejected.

   b. Is it that different from current MEDIA and NAVIGATION LIMITED widgets? I would argue that if the main screen is being duplicated by a widget that is on-screen, it should be in LIMITED, similar to how it is now. If the widget is not duplicating content, it should be in BACKGROUND.

   c. I agree with your proposal here. However, I would also say that when an app's widget is in non HMI NONE, the main screen of the app cannot go to HMI NONE, it must stay in HMI_BACKGROUND. Otherwise how policies affect the app seems iffy @JackLivio, do you agree?

   i. The app is in `NONE`. The widget is in `FULL`. The app sends a `Show`, but `Show` is disallowed in `NONE`. What happens?
   
   ii. Additional notes in (11).

8. 👍 Good. Could you reword the proposal here to clarify that?

9. displayCapabilities is a parameter of the RAI response.

10. This is concerning, see (7) and (11).

Additional Questions:

11. Is there a specific lifecycle you have in mind for the creation of the widget? On first HMI_BACKGROUND? In HMI_NONE after connection? I think this needs to be hard-defined as HMI_NONE for all OEMs. Here are the UX scenarios I'm trying to work through.

    a. An app is first installed. The user wants to add a widget for it, but the app needs to be started in the main view first so that it can create its additional screens. This is non-obvious to the user. b. The user plugs into the car. The widgets he had displayed last time are no longer available until the main app is started.

    These scenarios cannot happen or our UX is very poor. I don't know if this is defined in app services @joeygrover but I think app services would have to work the same way, right?

    Suggestions:

    a. Policies note that an app uses widgets, and every OEM is required to put those apps into HMI_BACKGROUND from the start, or to allow as many calls as needed in HMI_NONE. @JackLivio, if you can, please refresh my memory as to where we've seen the HMI_NONE RPC request limits. Was that OEM specific, or a hard-coded part of Core? If it can be removed, we could use policies to define that CreateScreen and Show can be sent in HMI_NONE, while other RPCs like AddCommand cannot. That would seem to solve this issue.

    Otherwise, if we do allow automatic moves to HMI_BACKGROUND we have to deal with the problem of numerous apps simultaneously sending AddCommand / CreateInteractionChoiceSet with VR grammars.

12. How does resuming work with widgets?

13. Can we change duplicate to mirror? It mirrors in perpetuity, not just once, which the word duplicate kind of infers.

14. If an app has multiple widgets, how would it differentiate them in the UI? e.g. Accuweather has a widget for 7 day, one for today. Does CreateScreen need a name?

15. I think we need a chart of your expected widget HMI levels and their corresponding UI and expected functionality.

16. @nickschwab and I were talking about cloud apps as well. @JackLivio or @joeygrover, can you confirm our suspicion that cloud apps only connect to the endpoint after they are selected on the HMI (i.e. put into HMI_FULL), which would break both widgets and app services for those apps.

17. OEMs will want to control which displays an app may be allowed on. For example, if Ford wanted navigation apps to be displayed on the HUD widget, but weather apps could not. Or a specific app has permissions to be there. I think policies would need to be extended for this as well.

[Jack-Byrne]

can you confirm our suspicion that cloud apps only connect to the endpoint after they are selected on the HMI

Yes. Proposal

[joeljfischer]

@JackLivio Until that changes, no cloud app would work with an app service or widget until they are activated. (Of course, it's the same for mobile apps at this point).

[Jack-Byrne]

We can give AppService Rpcs none level permissions to get around this issue, but we would also advise that core's config flag AppHMILevelNoneTimeScaleMaxRequests = 100 be increased.

The hmi levels an RPC is allowed in is totally up to the OEM's policy table configuration.

[Jack-Byrne]

@joel Here is the config flag related to requests in hmi level none..

[kshala-ford]

@joeljfischer Reading through the comment I see a major concern about HMI levels and policies. Let me try to comment to it:

As @JackLivio mentioned: Just as for any other RPC, OEMs decide on what HMI level they allow CreateScreen and DeleteScreen. Based on policies we can allow these RPCs also in HMI_NONE, therefore apps and their main screen doesn't need to be brought to HMI_BACKGROUND. This would be a risky intrusion into the existing HMI level state controller which is not needed. We as an OEM can choose to allow both RPCs in HMI_NONE using a specific functional group with consumer friendly messages. This means we can control what apps are allowed to have widgets and also provide control to users to also have control of it. We could even have two different functional groups. One allowing new screens only in BACKGROUND, LIMITED or FULL. The other group in all four levels (e.g. 1st party apps).

Should this be in the proposal? No because all this is related to HMI and OEMs implementations. Should we have additions to HMI integration guidelines? Maybe, we can discuss this further. Should we enforce rules policies specifically for the new RPCs or for widgets or main screens? I strongly disagree.

3. They will do different things. DisplayCapabilities is fully of screen related parameters and for single display purposes. The system capability we are talking about is rather BODY_INFO and is planned to provide metadata of the car including how many displays are available, their size, type, location relative to users. This new system capability will be worth a separate proposal. Widget support does not need this system capability now.

4. We need duplicate notifications because they are screen specific and audioStreamingState is mandatory. Let me try to give you an example on why it's suggested this way (the JSON represents dummy HMI status notification):

   1. App is started and receives {hmiLevel: FULL, audioStreamingState: AUDIBLE, systemContext: MAIN, screen:0}
   2. App creates a widget (1) which the HMI makes visible immediately. The app receives {hmiLevel: FULL, audioStreamingState: AUDIBLE, systemContext: MAIN, screen:1}
   3. System plays a TTS. App receives {hmiLevel: FULL, audioStreamingState: NOT_AUDIBLE, systemContext: MAIN, screen:0} and {hmiLevel: FULL, audioStreamingState: NOT_AUDIBLE, systemContext: MAIN, screen:1}
   4. System has finished playing the TTS. So app receives {hmiLevel: FULL, audioStreamingState: AUDIBLE, systemContext: MAIN, screen:0} and {hmiLevel: FULL, audioStreamingState: AUDIBLE, systemContext: MAIN, screen:1}

The reason why we choose this approach is to have the app to have consistent audible state. It increases the number of notifications though but we rated state consistency as a higher priority. Audible state is a very sensitive area where apps create lot of bugs. Having widgets always say NOT_AUDIBLE also works but it then requires a good documentation and many developers don't read documentations.

Regarding the managers method func hmiLevel(_ oldLevel.... The method signature won't need an update as it will still point to the applications main screen HMI level. However the manager should call this very method only if screen value is omitted or equals PredefinedScreens.MAIN_SCREEN. I can add it to the proposal if you prefer.

6. Simple. System context is mostly screen related but audible system is system related. Because of that please keep (5) and (6) separated. We were thinking of how important system context will be for widgets. For the sake of simplicity we have decided to keep widgets in SystemContext.MAIN. Everything else needs a revision of the HMI API if you look on UI.OnSystemContext.

7. a) It's not defined in the proposal but we can suggest that in the proposal. Do you think Core should return this or the HMI? If Core does it, it needs to remember the duplication mode.

8. I want to make all aware of SDL-0209 "Static Icon Capability" which adds imageCapabilities" toDisplayCapabilities`.

9. See my first paragraph in this comment. Making policies behave different per screen type is concerning me a lot. It has such a huge impact to the whole SDL system for no benefits. I strongly disagree to anything else than what is proposed or described above.

10. They are included; with all the content and screen IDs. I can add a comment in the proposal if you like.

11. The words to duplicate and mirror are very different. Mirror is more 1:1 full copy (content to present and how to present the content). But duplicate means to repeat the content only not necessarily (duplicate content to present but leave the destination the responsibility of how to present the content).

12. I don't think it needs a name. I am thinking of suggesting to use Show.templateTitle from your proposal SDL-0186. I believe it will fit very well in this context.

13. Understood. Let me see if I can create something for today.

14. I understand your suspicion. I expect Core to open socket connection to cloud apps in any HMI status <> NONE?

15. This is out of scope with this proposal but yeah we may want to control, how many widgets an appID can create on each individual display.

[Jack-Byrne]

I understand your suspicion. I expect Core to open socket connection to cloud apps in any HMI status <> NONE?

Per previous debates on cores web socket connection, it was agreed that core would only create the web socket connection after a cloud app is activated (None->Full). This was to prevent a large number of cloud apps that might be available on the head unit from maintaining an active connection when the user only uses a few apps.

[nickschwab]

I understand your suspicion. I expect Core to open socket connection to cloud apps in any HMI status <> NONE?

Per previous debates on cores web socket connection, it was agreed that core would only create the web socket connection after a cloud app is activated (None->Full). This was to prevent a large number of cloud apps that might be available on the head unit from maintaining an active connection when the user only uses a few apps.

I agree with @joeljfischer that it would be a poor user experience if an app couldn't register as a Service Provider or resume a Widget until the app was manually activated by the user. I think it would be worthwhile to consider an update in a future (but near-term) release cycle to create the websocket connections before activation if the app is an App Service Provider or a Widget Provider (which we can make identifiable via app_policies). App policies already contain information to designate if the app is a Service Provider, so we would only need to add an additional attribute to indicate that the app is a Widget Provider. This additional attribute would also be useful for knowing when to grant Widget-specific Functional Groups to an app.

[joeljfischer]

Regarding policies:

Should this be in the proposal? No because all this is related to HMI and OEMs implementations. ... We as an OEM can choose to allow both RPCs in HMI_NONE using a specific functional group with consumer friendly messages.

This is not all related to HMI and OEM implementations. That I have to state this is all the more concerning because we've been through this discussion many times on many proposals. We have an open-source policy server, and changes (like the addition of a new default functional group) should be added there. Additionally, the developer portal should be updated to have the developer checking a box for widgets so that they can get approved for that functional group allowing the CreateScreen RPC in HMI_NONE. This is all work that must be done by the SDLC before this feature is really ready.

Should we have additions to HMI integration guidelines? Maybe, we can discuss this further. Should we enforce rules policies specifically for the new RPCs or for widgets or main screens? I strongly disagree.

I don't think we can enforce rules, but even if we don't use "must" language in the proposal, the proposal ought to use "should" language and talk about the awful UX that will ensue if OEMs do not follow the suggestions. Otherwise, we're just leaving it up to a wild west of implementations, which is bad for everybody in these kinds of cases.

We could even have two different functional groups. One allowing new screens only in BACKGROUND, LIMITED or FULL. The other group in all four levels (e.g. 1st party apps).

That's kind of scary. First party apps are given special privileges here, while all 3rd party apps with widgets would have a broken UX (they need to be selected by the user on the apps screen before their widgets are capable of showing up and so can't be resumed or anything like that). This is why discussion around these items needs to happen, and the proposal ought to give strong recommendations and discussion of UX implications of the various choices an OEM can make.

3. Gotcha 👍🏻

Skip to make numbers work

5. I think I'm understanding what you're doing, but I suppose I'm still not understanding why. Why does a widget need a new onHMIStatus for audio state changes? What are the use-cases? If it's not needed for any solid use-cases, then why not just assume that the audio state for the app is the audio state for the app and all widgets? That would also provide a consistent audio state.

   I would feel a bit better to have the manager changes for hmiLevel, audioState, etc. delegate methods change mentioned in the proposal.

6. I'd like to get some other eyes on this @joeygrover @JackLivio. I'm not understanding your reasoning here. You claim:

System context is mostly screen related but audible system is system related.

But if that's the case, why would we send multiple status RPCs for the system related parameters and one for the screen related parameters, when the widget and the main app are on different screens? If the widget's UI is obscured for whatever reason, why would we not tell the widget? Since the audio state is guaranteed to be the same between widget and main app why would we tell the widget separately?

I don't think an HMI_API revision is super-unreasonable and it seems like a pretty small change. Maybe I'm just really not understanding here.

7. a. @jackLivio what do you think? Should Core or HMI handle REJECTing any RPC sent to a widget that's duplicating screen content and shouldn't respond to any Shows etc.

Skip to make numbers work

9. (Your 8) This is a misplaced deflection. For SDL-0209 we considered adding a new GetSystemCapability, but we decided the impact was quite small and it made the most sense to have it next to imageFields and graphicSupported. However, I don't think the same can be said of the current proposal. Screen type is an entirely new type of capability, and may expand in the future. See (18) for an additional related point.

10. (Your 9) I think we're already doing that, right? Because some RPCs only affect MAIN screens? Unless you're saying those restrictions should be hardcoded in Core? I don't see that that's noted in the proposal, I must have assumed incorrectly. I don't see any specific need for changes to policies per screen type at this time. @nickschwab, do you agree?

Skip to make numbers work

12. (Your 10) 👍🏻 Please do.

13. (Your 11) I don't think that's what duplicate and mirror mean. We're not talking about mirroring in terms of MirrorLink or anything like that. duplicate just doesn't really carry the aspect of "continuation". It may seem minor, but I think using mirrorScreenId would help make it clearer to developers that it's not just starting with the same content, but will continue to mirror it forever.

14. (Your 12) That would be good, but I don't think it will work, because the Show with the templateTitle would be shown until the widget is actually on screen, at which point it's too late for the use-case I presented.

15. (Your 13) 👍🏻

16. (Your 14) So yes, it appears that in the current iteration, socket connections are only opened after the app is activated on the HMI. However, for a cloud app to work with widgets or app services with adequate UX, they would need to have a socket connection open before then. This is not directly related to your proposal, but it is an implication of it.

17. (Your 15) True, it's just an implication for the next stage.

Additional points:

18. What if a screen's dimensions and capabilities change at runtime? For example, an HMI has two widgets on screen, each with half the screen. Then a third is added, and one of the widgets is changed from 1/2 the screen to 1/4th the screen? I think this links with (9), since in the next release GetSystemCapability has an OnSystemCapability subscription feature, it fits right in here.

[jordynmackool]

The Steering Committee voted to return this proposal for revisions. The author is to make revisions based on the points discussed in his responses below and the agreed upon responses in this comment. There will be short workshop planned to discuss the additional fundamental points that need to be defined such as functional groups, what RPCs are allowed in certain states, display capability location, and if we are going to use the OnSystemCapability subscription feature coming out in the next release.
The author responded to question number five in this comment regarding audioStreamingState, noting that it was proposed this way so that there is consistency and that the view should always be the same regardless if it is a widget or the main screen. AudioStreamingState is a sensitive notification for media app developers, hence the reason we wanted to have this consistency. The PM agreed that for consistency purposes they will defer to the author and agree to this implementation in the proposal. Additionally, the author responded to question number six in this comment regarding onsystemContext stating that the example given by the PM is correct and widgets can be obscured by an overlay depending on the HMI the widgets themselves can obscure from the overlay; this is dependent on the HMI implementation. For simplicity, we could ignore this and say that the widget is just in the main onsystemContext. It was agreed by all parties that we can extend the HMI API for onsystemContext to provide what screen is affected.

[jordynmackool]

Meeting minutes are attached from the SDL 0206- Widget Proposal Discussion that took place on 2019-03-05.

2019-03-05_SDL_0206-_Widget_Proposal_Discussion_Meeting_Minutes.pdf

Attachement A.pdf

[theresalech]

This proposal has been updated to include requested revisions. The revised proposal is now in review until 2019-03-26.

[joeygrover]

I have a few comments that I have tagged with priority. There are a few minor things that I think can be added during an accept with revisions vote, but there are a few others I think we need to have addressed first. Some of the items were carried over from the workshop that we either didn't have time for or couldn't come to an agreement on,

1. (MEDIUM) In my opinion, widget templates should have very strict guidelines. Currently the main screen templates can range drastically and that creates a lot of work for the developer. I would like to prevent that bad practice from continuing. For example the layouts should be presented exactly how they are defined in this proposal. The elements they have should not be moved around, including but not limited to changing them from landscape to portrait models.
2. (MEDIUM) TemplateColors should be applied to widget screens as well. There is nothing saying it isn't included but ensuring that it is included is important. Referring to the previous point, it would be fantastic to agree upon specific UI items that get which color param applied to them (background, buttons, text, etc).
3. (LOW) Add a description to all additions of displayID in the RPC spec that defines what the param is and what to do in the absence of it being provided.
4. (LOW) Add a description to all additions of screenID in the RPC spec that defines what the param is and what to do in the absence of it being provided. (Essentially default will be main screen if not present)
5. (LOW) The terminology of screen vs display is a little too close. I would prefer window or view instead of screen, or a combination that helps define it away from a physical display.
6. (LOW) When defining display/screen/system/ RPCs is it possible to get a restructured section of the RPC lists into tables for easier reference in the proposal? Or added as an appendix piece would be nice to have.
7. (LOW) At the start of Proposed solution can we get clarifying language ..., in the apps domain + of the IVI system?
8. (MEDIUM) duplicateScreenID param naming. This naming could be improved to mirrorScreenID instead. Since creating a screen will create a new logical screen on the Core module that has its own HMI status the naming should reflect that it is not a duplicate of the original screen, but instead the new screen is mirroring all the content that gets sent to the original screen.
9. (MEDIUM) The response RPCs for CreateScreen and DeleteScreen currently just state common response params, however, it would be good to include which Result codes are being used for them during implementation. Also I believe new Result codes could be useful for these types of responses. In general a new Result code of INVALID_DISPLAY for example.
10. (MEDIUM) The PredefinedScreen enum is referenced but never used. Is this just for documentation purposes? Meaning the screen id 0 will always be the main screen? It seems a little unnecessary in the RPC spec if that is the case. Is there a plan to include other predefined screens or use the enum itself anywhere?
11. (LOW) Is OnAppActivated the best HMI message for defining when a widget becomes visible? Maybe it would be better to create a new HMI message? @JackLivio any input?
12. (LOW) The added HMI state chart shows only HMI_NONE & HMI_FULL states for the widget screen type. Just as a clarifying question, is the intention that a widget can only be updated while it's visible? Otherwise policies will reject certain messages like Show while the widget is in HMI_NONE.
13. (MEDIUM) For the new ScreenCapability struct, are the ImageFields and imageTypeSupported params tied together 1 for 1? Or is it just an array that defines what types the system as a whole can support and each imageField has to align with those values?
14. (MEDIUM) Regarding the OnSystemCapability notification, I believe the first notification after RAI Response should contain all displays and screen capabilities as described. The other two scenarios I want to make sure only send back the updated display and screen combo. The OnSystemCapability should only contain the updated/changed information.
15. (HIGH) As discussed in the note under this section, I'm a bit leery of changing the SetDisplayLayout into a completely new RPC that does the exact same thing. I understand there will be some inherent confusion between display and screen here because of the naming, but is it worth the extra logic to handle knowing when to use which one? I would think it's easier to simply add a more in depth description of the RPC and explain its use case. Then in the client libraries simply rename the manager APIs to align with more accurate nomenclature.
16. (HIGH) As a user, I would like to be able to select the widget that is going to be used for an application just like I do on my phone. This means the app should be able to provide the user with choices on which they'd like and allow them to select the specific one. However, the proposal would need a few modifications in order for this to happen.
    • CreateScreen automatically creates an object on Core that gets assigned an HMI status. While it can be done this way, it does create overhead of having to maintain the state of each of those screens even though they may not currently be nor ever be visible to the user.
    • Ensuring a well defined best practice UX for widget selection is in place. If this type of functionality were considered, we would need to define a decent set of best practices on how the UX should flow. As long as we are confident on this proposal's feature set in support of that, it could be defined later.
    • Without the user's selection of a specific widget or if the selection happened on the mobile handset prior to connection with the module, there would need to be a designation of which widget should be used. This could be accomplished by adding either a boolean primaryWidget to the CreateScreen RPC, or an integer of widgetPriority that could take values to define an ordered list of which widgets should be used.
    • The current proposal would provide the app with a limited number of screens that could be created. Using the CreateScreen RPC, the app would be limited to the number of options it provides to the user without Core rejecting the RPC. It might be possible to include this information in either the RAI or a new RPC that defines possible widgets.

[Jack-Byrne]

11. (LOW) Is OnAppActivated the best HMI message for defining when a widget becomes visible? Maybe it would be better to create a new HMI message? @JackLivio any input?

I wouldn't recommend OnAppActivated at all for this case. This notification is the most "primitive" way to activate an app and it does not guarantee Core will allow the app to be in full right away (showing data consent prompts, or verifying the app is actually connected). We recommend that HMI integrators use SDL.ActivateApp request/response in order to put apps into full. Having a response tell the HMI the activation was successful is a safer approach. I think the proposal should be updated at least to use the request response pair instead of the notification.

However if you have a "widgets screen" that can show multiple widgets from different apps I don't think you want to activate all of the widgets at once. SDL ActivateApp usually requires some sort of user interaction (talk or touch) and that is a behavior we might want to reserve for widgets.

I think there are two better existing RPC options for notifying Core when you are displaying widgets.

First Approach: OnSystemContext

  <function name="OnSystemContext" messagetype="notification">
    <description>
      Notification must be initiated by HMI when the user changes the context of application: goes to menu (in-application menu or system menu);
      switches to VR; pop-up appears on screen etc.
    </description>
    <param name="systemContext" type="Common.SystemContext" mandatory="true">
      <description>The context the application is brought into.</description>
    </param>
    <param name="appID" type="Integer" mandatory="false">
      <description>ID of application that is related to this RPC.</description>
    </param>
    <param name="screenID" type="Integer" mandatory="false" /> <!-- new -->
  </function>

The proposal already suggests adding screenID to this notification, but for this use case we would need two new SystemContext enums.

<enum name="SystemContext">
  <description>Enumeration that describes possible contexts the application might be in on HU.</description>
  <description>Communicated to whichever app is in HMI FULL, except Alert.</description>
...
  <element name="SHOW_WIDGET" internal_name="SYSCTXT_WIDGET">
    <description>The app's widget is fully visible onscreen.</description>
  </element>
  <element name="HIDE_WIDGET" internal_name="SYSCTXT_WIDGET">
    <description>The app's widget is not visible onscreen.</description>
  </element>
</enum>

When the HMI wants to notify core it is showing the widget it would send

OnSystemContext : {
   systemContext: SHOW_WIDGET,
   screenID: X, 
   appID: true 
}

When the HMI wants to notify core it is no longer showing the widget it would send

OnSystemContext : {
   systemContext: HIDE_WIDGET,
   screenID: X, 
   appID: true 
}

Second Approach: OnEventChanged

   <function name="OnEventChanged" messagetype="notification">
     <description>Sender: HMI->SDL. When event is become active</description>
     <param name="eventName" type="Common.EventTypes" mandatory="true">
        <description>Specifies the types of active events.</description>
     </param>
     <param name="isActive" type="Boolean" mandatory="true">
        <description>Must be 'true' - when the event is started on HMI. Must be 'false' when the event is ended on HMI</description>
     </param>
+     <param name="screenID" type="Integer" mandatory="false" />
   </function>

<enum name="EventTypes">
  <description>Reflects the current active event</description>
...
  <element name="SHOW_WIDGET">
    <description>A widget is displayed by the hmi</description>
  </element>
</enum>

This approach would need to add screenID to the notification and a new EventTypes enum SHOW_WIDGET. When the HMI wants to notify core it is showing the widget it would send

OnEventChanged : {
   eventName: SHOW_WIDGET,
   screenID: X, 
   isActive: true 
}

When the HMI wants to notify core it is no longer showing the widget it would send

OnEventChanged : {
   eventName: SHOW_WIDGET,
   screenID: X, 
   isActive: false
}

[LuxoftAKutsan]

If the app registers with a resumption ID and this ID is recognized by the HMI, all screens created by the application will resume including widgets and their content (text, images and buttons).

1. Will SDL or HMI or mobile be responsible for resumption of widget contents ?
2. Resumption id is hash id in RAI, correct? https://github.com/smartdevicelink/sdl_core/blob/master/src/components/interfaces/MOBILE_API.xml#L3782

(LOW) The terminology of screen vs display is a little too close. I would prefer window or view instead of screen, or a combination that helps define it away from a physical display. :+1:

[theresalech]

The Steering Committee voted to keep this proposal in review, to allow the author time to respond to the comments left on the issue, and to allow other members to review and provide feedback on the proposal.

[kshala-ford]

Regarding 1. To a 100 %: I'm completely against defining templates, especially in this proposal. I will not start any work related to this item. If you feel need for strict templates feel free to write a new proposal. We need to stop adding more and more stuff to single proposals.

Regarding 2. Template coloring is included as you can see in the new SetScreenLayout RPC.

Regarding 3. displayID is only mentioned in the motivation and is out of scope of this proposal. Adding a description has no affect to the outcome of this proposal.

Regarding 4. Descriptions to screen IDs should not hold this proposal back from getting accepted. The parameters are described in the text. The reason for not having all the descriptions in the xml block is for a better readibility and reducing code blocks. Please consider this as an implementaion (documentation) detail which will be done during development.

Regarding 5. MacOS and iOS work this way, X Window work this way. The mobile API is mixing the names display and screen as they were in a 1:1 relation. Checking the dictionary "display" is more cloes to a physical device that contain a screen as a part. Display and screen" is more close to the namings in the mobile API than any ther word combination. I'm strongly against "view". I'm not so sure about "Window", it would require ScreenParams to be renamed as well as SetDisplayLayout. Also more documentation needs a revision.

Regarding 6. Because it's only "Motivation" it was listed as it is right now. The table increased the size more than we wanted to. With multi display support (when these three groups become important to consider) you will have a table included.

Regarding 7. The apps domain describes the area of the infotainment system related to SDL apps. There is no occurence of of the IVI system in my proposal.

Regarding 8. I don't see "mirror" to improve the naming. "Duplication" is the right word as every Show RPC content is duplicated to the other screen. "Mirror" is a 1:1 copy and will for sure cause more confusion as people will misunderstand this as screen mirroring.

Regarding 9. There are so many accepted proposals including responses but don't contain response params. Why do I have to add the obvious parameters success, resultCode and info? Isn't this an obvious implementation detail? I always considered them as common params for responses. FYI: Possible result codes are SUCCESS, REJECTED, DISALLOWED, INVALID_DATA, INVALID_ID, DUPLICATE_NAME, OUT_OF_MEMORY, TOO_MANY_PENDING_REQUESTS, APPLICATION_NOT_REGISTERED, GENERIC_ERROR. I don't think INVALID_DISPLAY is helpful. The info field should refer to the parameter that contains the invalid identifier.

Regarding 10. It's a documentation enum just as PredefinedLayout. The purpose of this enum is to avoid having magic (undocumented) numbers. I don't want to only have the description field to mention that 0 is related to the default screen.

Regarding 11. Well, we were looking at the HMI_API and saw BasicCommunication.OnAppActivated as the best candidate. BasicCommunication.ActivateApp is an SDL to HMI request. I also read the other comments and SDL.ActivateApp has only very limited documentation. To my understanding it is related to policies and doesn't make sense here because policies are already processed (otherwise widgets could not have been created). I want to let you all know about something very important regarding app and screen activation. If the user interacts with the infotainment system making an app screen visible (by tapping on the app icon or by scrolling/swiping screens to the visible area) the HMI doesn't ask Core to activate the app. The app has to be activated no matter what. A request doesn't make that much sense because HMI is not asking for activation. The app screen is visible and demands content to be shown. Again policies are already processed so SDL interface is not needed. Last I want you to read the description of OnAppActivated

(OnAppActivated) Must be sent by HU system when the user clicks on app in the list of registered apps or on soft button with 'STEAL_FOCUS' action.

The proposed solution meets the current HMI integration requirements and fits the need of activating app screens.

Regarding 12. Yes it is considered in this way to avoid content updates that will not be visible (thinking of the infotainment system performance). Personally I don't have a strong opinion. I'm also OK to a state machine that does NONE - (screen becomes visible) -> FULL - (screen becomes invisible) -> BACKGROUND - (screen becomes visible) -> FULL and so on.

Regarding 13. The parameter imageTypeSupported replaces graphicSupported which wasn't clear in the description. The old param was kind of saying "The screen supports Show.graphic hence supports dynamic image type". ScreenCapability.imageTypeSupported is supposed to be the app's conventient way to identify dynamic images to be supported.

Regarding 14. See alternative solution

Instead of only having DisplayCapability it is possible to separate display capability and screen capability in SystemCapability. This may allow some benefits avoiding repeating display data when screens have changed. Also, it may allow OnSystemCapabilityUpdated to specific screens, instead of repeating all screens data.

We also thought about your suggestion but we left it in the alternative solutions because it will cause an inconsistent API. It has to be documented very carefully that sometimes all screens capabilities are returned. sometimes only of a single screen...

Regarding 15. Again see alternative solution. I really want to understand the threshold when we admit to a clear API and when we accept legacy stuff. I know that the RPCs are more and more invisible to the end user (developer) but still... it is kind of arbitrary...

You have two options. Deprecate SetDisplayLayout only or the four parameters displayCapabilities buttonCapabilities, softButtonCapabilities, presetBankCapabilities. Which one is less expensive?

Regarding 16. I read this point multiple times and every time I think "This is the OEMs decision". Anyway Nothing of this point is required for the first steering committee acceptance. See my comment of the first point. This point is definitely something for a subsequent proposal.

[joeygrover]

1. We will have to agree to disagree here. I was well aware that this wouldn't be a popular opinion among OEMs but wanted to have my concern voiced and on the record nonetheless.
2. I apologize, I missed that. (While the second point wasn't address in your comment, based on the first item I will assume it was not something you would agree with).
3. Fair enough.
4. This wouldn't hold the proposal back from being accepted, but rather a nice documentation revision that could be made with an "Accepted with revisions" vote. I'm always for more details and specificity in these situations and it shouldn't be more than a minor effort to include. If you want I can help make the changes and avoid putting the burden on you.
5. I agree our API does tend to mix the terminology here a bit. I am not sure on the best resolution. Can you further example your strong opinion against "View", just curious for my own understanding? Maybe something like ScreenWindow, ScreenView, etc even would help in my opinion. I think I'd like to defer to the steering committee on this item if necessary, unless you'd be open to those suggestions or something of your own.
6. Would you mind if I created a table as an appendix to the proposal? I think it will still be helpful for this proposals as a quick visual aid for reference. Then when there is the phase 2 multi-display proposal it could simply be copied or referenced.
7. I think that's the same thing, right? Again this is just helpful, clarifying language. This is another thing I could help with if you wanted me to.
8. A widget can only receive content from one other screen right? I don't believe it was possible for a widget to send an array of screens that it wanted to receive content updates for and pushed to its screen. That is why I believe the mirror terminology makes the most sense. For each widget, there is one source screen to which the content would be mirrored from. While a single screen could have its content sent to multiple widgets, the proposal is setup to allow a widget to define one "source" screen and not an array of "destination" screens. (Unless this is exactly what it does and I have completely misunderstood). I see mirror greatly improving the name so I don't believe we will have a compromise and would like to defer to the Steering Committee for a decision.
9. Just an ask for best practice. However, each response does only take a subset of the entire list of ResultCodes so again it's good to be specific when possible. I wasn't sure about any custom results, so the second part was more of a probing question to see if it's something you've thought about. If you don't mind, this is also something that I will like to help you with post a committee vote if you'd like.
10. Ok, I agree magic numbers are bad practice. Is there any plan to include other enum values? Like, would there ever be a time when other screens were added by default?
11. I will default back to Jack on this one. Maybe someone from Luxoft can comment as well.
12. I was only looking for clarification here. I agree there's no reason to really send updates while the widget isn't visible. Just something we have to ensure is clear to developers (even though it will be very likely handled by the library managers).
13. :thumbsup: Sounds good to me.
14. I think it's fine to say that during a notification only the updated screens will be included, while the request/response would return everything. This is how the AppServicesCapabilities works. Also, the system capability manager will be taking care of this as well for the developer. Caching the full result and updating during a notification received.
15. I think if an RPC already exists in the spec it has more of an allowable grey area in terms of what it does, however, when we add new RPCs they should be specific and well named. We do the best we can from the start and introduction of RPCs but we also don't know what future additions/modifications we will make. Secondly, I believe deprecating the 4 params is less expensive because they are all non-mandatory parameters in the first place so they can simply be not included.
16. I agree it is always up to the OEM, but with SDL being an enabling technology for OEMs we should allow them to make that decision and not prevent them from it if possible. So the reason I bring it up is that I would like to make sure that we protect for it with the new widgets feature from the start. I believe this is a worthwhile addition to the feature so I would suggest deferring to the steering committee and they can decide if this should be included in this phase, possibly ensuring enough time to allow a proposal to be written and voted on before this proposal would be implemented, or voting for a simple rejections of the idea.

[Jack-Byrne]

11. @joeygrover @kshala-ford I am ok with leaving OnAppActivated and OnAppDeactivated as the proposal originally described.

[joeljfischer]

1. I personally agree with the author that the layouts shouldn't be rigid, as long as they fit the description in the name, allowing for flexibility like landscape / portrait.

Fix numbers

5. I agree that screen vs. display are a bit too close here, and window vs. display (or view vs. display) would be a large improvement. I have confused the two too many times to count. It is true that screen can have the definition you imply in this proposal, but it also has a common definition that's interchangeable with display (e.g. The TV screen, the movie screen, both are physical screens). I think window is perhaps moderately preferable to view.

   I don't think SetDisplayLayout would need to be revised since it's already being deprecated, right? SetScreenLayout would just have to be revised to SetWindowLayout? (Side note: This is also a bit confusing because in the section Screen manipulation you mention SetDisplayLayout as in use. This is probably a minor oversight from the last revision). ScreenParams would be weird, but the terminology was already weird due to the conflation of "Display" and "Screen" in the API. I think it would probably be okay, though I agree, certainly out of place.

Fix numbers

9. I don't believe that Joey did actually ask to include the common response params. He's simply asking for which result codes are valid and what states lead to them, as well as if any new result codes should be created. For example, if sending an RPC to an invalid display doesn't use a new result code like INVALID_DISPLAY I would assume you mean it to use INVALID_DATA and that the info code should give specific help. Many RPC responses list their valid Result codes. (e.g. RegisterAppInterfaceResponse, UnregisterAppInterfaceResponse, etc.)

Fix numbers

12. This is definitely an interesting one, because we also need to discuss how Core / HMI is going to handle the possibility of many more HMI elements being activated. e.g. if 10 apps connect and are in HMI_NONE and all have 2 widgets available, that adds up to 30 "apps" with their respective states and activations (e.g. 10 apps in HMI_NONE calling CreateScreen at the same time and Core / the HMI having to create all these object instances of the widgets). Or, should an app simply declare which widgets they support (in something like a manifest), with unique Ids, and then the HMI tells Core, which tells Mobile via a notification which they want actually created? I know this is a slightly more complicated approach, but it may improve the Core / HMI system performance and the complexity burden.

    The only downside I can see to only allowing SHOW in HMI_FULL is the possibility of stale data "flashing." e.g. User goes to home screen, app gets OnHMIStatus for FULL, app updates song that was playing 5 songs ago to the current one, but that takes 0.5 seconds so there's a weird flash instead of the data already being up to date.

Fix numbers

16. I think there's been a lot of confusion about this, so please allow me to talk about this at a bit of length in order to fully explain Joey and I's position here.

    There's are basically two possible user experiences for widgets: static placement and dynamic placement. Static placement is when the user selects a widget to be placed in a slot and covers several possible use-cases:

    a. S1 - The user selects from a menu of all possible widgets to place it into a slot. b. S2 - The user selects the app they want to place into a slot and the system determines which widget from the app to actually place into that slot.

    Dynamic placement covers several possible use-cases:

    a. D1 - The system chooses which widget to place in a slot based on an app being recent (e.g. the user went from an app screen to the main screen and the system wants to place that app's widget in a slot. Similar to iOS and Android recent app screens but with widgets. b. D2 - The system chooses which widget to place in a slot based on the active app for a given app service type. e.g. There's a slot for the active navigation / music / weather app and the system places the active app's widget into that slot.

    This proposal should support all four use-cases, including when the app has made multiple widgets available to the system. I believe the current proposal does not properly handle S2, D1, or D2 when apps have multiple widgets. There is no way in the current proposal for the system to know which widget should be used. Because of that oversight, I don't believe that the current proposal can be accepted and a later proposal used. That's a fairly significant use-case for which this proposal has undefined behavior.

    I think my comments on point 12 also bear on this point. Adding a manifest which then the system could implement would work. It could look something like this:

<function name="SendWidgetsManifest" functionID="SendWidgetsManifestID" messageType="request" since="5.x">
    <description>This RPC can be sent at any time to update the list of widgets that should be made available to the system</description>
    <param name="widgetManifests" type="WidgetManifest" mandatory="true" array="true" minSize="0" maxSize="10">
        <description>If this array contains 0 elements, remove all the app's widgets, and this app no longer makes them available to the system. The order of the widgets in this list is their priority from first to last for dynamic selection purposes by the HMI.</description>
    </param>
</function>

<struct name="WidgetManifest" since="5.x">
    <param name="name" type="String" mandatory="true">
        <description>Must be unique among the app's widgets. Will be used to identify the widget to the user.</description>
    </param>
    <param name="appServiceAssociation" type="AppServiceType" mandatory="false">
        <description>If provided, this widget should be used in dynamic situations when this app is the active app for the service provided. e.g. If the app is the active media app, this is the widget you want displayed on the system if the system only picks one. This overrides the priority given by the order of SendWidgetsManifest.widgetManifests.
    </param>

... (Other parameters as needed)

</struct>

<function name="OnWidgetActivated" functionID="OnWidgetActivatedID" type="notification" since="5.x">
    <description>When received, the app must call `CreateScreen` for the appropriate widget on the appropriate screen.
    <param name="name" type="String" mandatory="true" />
    <param name="screenID" type"Integer" mandatory="true" />
</function>

# Probably needs `OnWidgetDeactivated` as well or something.

And so forth. I wrote this in about 5 minutes since I know it probably won't be used as is and so it probably contains a number of oversights and errors. Just one alternate possible direction to get us to support all cases as well as decreasing the burden on the HMI by not creating app objects until they are needed by the system.

Alternately, the current framework can probably support multiple widgets for cases S2, D1, and (partly) D2 by adding some sort of priority flag and fully support D2 by adding an app service association flag.

I apologize for this being so late, as I have been heavily focused on this week's upcoming release.

[theresalech]

The Steering Committee voted to defer this proposal (15 votes), keeping it in review until our next meeting on 2019-04-09. More information about the discussion on this proposal that took place during the 2019-04-02 meeting can be found below.

• Ford team would like to accept proposal as is, and resolve items in comments separately in subsequent proposals, so that work on feature implementation can begin.
• Project Maintainer believes proposal should be a spec/architecture for a fully implementable feature as it stands, without undefined behavior left to define at a later date. As the earliest this feature could be implemented is the next release of Core (September 2019), we do have time to continue discussion.
• Larger Items in Discussion:
  • 12. Project Maintainer and author have agreed to add the BACKGROUND state to the widget HMI state machine. This proposal will be revised so that HMI NONE, BACKGROUND, and FULL should be in the state machine, as opposed to only NONE and FULL. The author has advised that they don’t believe there will be a resource issue on the HMI with a large number of widgets, and therefore the manifest is not necessary. The Project Maintainer and author also agreed that the HMI Guidelines should include mention about handling resources with multiple widgets.
  • 16. Project Maintainer believes we need to define the experience for multi-widget support, for example, how does the module know which widget to display as primary? The author referenced using the PredefinedScreens enum, with 1 as the magic number for primary widget (default main screen), and 0 which makes the default main screen pre-created. If you want to set the primary widget, use 1 as the ID. The Project Maintainer posed the question of what happens when an app creates a widget in the same slot where it already created a widget, and how can an app change which widget is the primary. There was discussion about including higher level abstraction RPCs to do so. It was determined that this still needs to be defined.
• The Steering Committee did not have time to discuss the additional items referenced in the comments between the Project Maintainer and author, and so these will need to be discussed in our next meeting.
• The Project Maintainer made it clear that they agree this is a very important proposal that they would like to be accepted, once feature behavior is more thoroughly defined. They noted that agreeing to and defining this behavior in the evolution process will help to avoid running into issues during implementation, which could have an impact on release timing.

[joeljfischer]

I would like to note the main reason we don't believe we can approve this proposal until issues like 16 are resolved as opposed to proposals like the High Level Interface / App Services. The HLI and App Services were designed to append on proposals and do so by expanding functionality. This proposal is different in that if we accepted it in the current state we would be doing so knowing that the proposal has undefined behavior.

If the author wishes to revise the proposal to remove the possibility of an app creating more than one widget for now, then the issue of (16) could be added in an additional proposal. However, the current proposal allows for multiple widgets, but doesn't define behavior in important related cases, therefore the maintainer believes that those cases need to be resolved before a vote for acceptance should happen.

I hope that helps to clarify our position on the difference between accepting this proposal as-is, and multi-proposal features like HLI and App Services.

[joeygrover]

Status

• Closed: 1, 2, 3, 10, 11, 13
• Handled with doc update branch: 4, 6, 7
• Author agreed to update: 12
• Open: 5, 8, 9, 14, 15, 16

I have pushed a branch to my fork of this repo to include a few of the documentation updates I was asking for to help get this proposal though. It includes changes for my previous bullet points 4, 6, and 7. See here. It can be merged post acceptance of the proposal with the author's permission.

As far as the open questions I'd like to group them into a couple of buckets. Unless the author and project maintainer can come to an agreement before the next steering committee meeting on a few items, I would like to take them to the steering committee for a vote so that we can move past the items and get the proposal through. The other items, I believe we can continue the discussion with.

Steering Committee Decisions

Item for discussion	Author's Stance (Please correct if wrong)	Project Maintainer's Stance
5. Screen terminology	Screen is acceptable, and a closely related to other RPC and parameters	Screen and display are too closely related and will result in confusion. Recommend an alternative (Window, ScreenWindow, etc)
8. DuplicateScreen vs MirrorScreen	Duplicate works because the Show content is duplicated to the screen, and mirror will cause confusion with screen mirroring	Duplication implies a single time event, where as mirror is an ongoing term. When creating a widget, a single id is given which implies that the created screen will be mirroring the single screen for its lifecycle.

Continued discussion with Author

| 9. Result Code: I'm fine with the author's response, I would just like them included in the proposal post an acceptance. (I can include these in the previously shared branch if the author wishes). @joeljfischer did you have anything else you needed from this one?

| 14. I didn't get a response for the last post I added. Does the author have any additional comments? If the author still doesn't agree, I'd like to add it to the steering committee decision group.

| 15. I didn't get a response for the last post I added. Does the author have any additional comments? If the author still doesn't agree, I'd like to add it to the steering committee decision group.

| 16. So in the last meeting we talked about adding a DEFAULT_WIDGET value to the PredefinedScreens enum set. This would act as the "primary" widget. This could work, but we would need to make sure we could handle that type of logic in the libraries for a developer switching their primary widget. I don't like the create/delete method as it would likely cause screen flickering, but I do think a manager could handle the switch possibly by caching the data that is currently displayed. @joeljfischer please expand on this if you have any additional thoughts.

[kshala-ford]

Thank you Joey for the changes. I merged them into my fork, added some changes and created a pull request..

Regarding 5. I'm OK with using window instead of screen. Whenever I speak of a screen I mean window. the PR does not contain changes to reflect this new name.

Regarding 8 I can only say it again that mirror is a constant 1:1 copy whereby duplicate means duplicating the content of the "Show" requests whereby the target screen can be of a different type (therefore can look slightly different).

Regarding 9. I have added the result codes to the PR

Regarding 14 I've changed the proposal in the PR to match the alternative solution, which is to return all screen capabilities after registration and GetSystemCapability request and to return the affected screen only on CreateScreen and SetScreenLayout.

Regarding 15 I lean towards clean API and call it SetScreenLayout or SetWindowLayout. Alternatively we could add the params to Show which would be very benefical to synchronize layout changes with content changes. Any thoughts to this?

Regarding 16 I've added a new parameter to CreateScreen which is appHMIType and is a 1:1 copy of RegisterAppInterface.

Specifying an HMI type allows an app to create a screen related to one or multiple HMI types. As an example if a MEDIA app becomes active, this app becomes audible and is allowed to play audio. Actions such as skip or play/pause will be directed to this active media app. In case of widgets, the system can provide a single "media" widget will acts as a placeholder for the active media app. The default main screen, which is pre-created during app registration, will be created based on the HMI types specified in the app registration request. It is only allowed to have one screen per screen type and per HMI type. this means that a media app can only have a single MEDIA main screen and a single MEDIA widget. If the app requests a second screen of the same screen and HMI type, the response will be rejected with the result "INVALID_DATA".

Still the app can create widgets omitting this parameter or with specifying the default HMI type. Those widgets would be available to the user independent of the HMI type.

[joeygrover]

Status

• Closed: 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14
• Open: 8, 15, 16

| 5. Sounds good! Thank you for your compromise.

| 8. I believe we are just seeing it from different sides. You mention a Show gets duplicated, which is correct, and we mention a screen is mirrored which is almost correct. I think the important piece to the mirrored screen is as you mention they can look different, ieSetDisplayLayout or equivalent, so they would use different templates. So really it's the content update that is being duplicated or mirrored but not the window itself. I will have to keep thinking on this, but adding a little more to the param name with content or even Show specifically might make things a little clearer on what is going to happen.

| 9. Thank you!

| 14. Sounds good! The system capability manager can do the heavy lifting for keeping everything up to date without the extra traffic.

| 15. I actually really like the idea of being able to send content directly with the template switch. The one issue we would incur however, is the subsequent screen/window capability changes would only be received post the initial Show message with the new layout selected. So the app would possibly have to send a second Show with the updated fields (ie, merged line1 and line2). I think the screen managers could handle this for the most part though and could always use a "best-guess" or minimum requirement per layout. I would think the screen flickering/old data between layout changes might be worse than having to send the second Show.

| 16. I have to think this one through a little more. Would the app have to register with the AppHMIType in the RAI before they could create a new window with that type? Would the main window be associated with their first AppHMIType provided in the RAI? Would the phase two with multiple displays define one window type per AppHMIType per display?

[joeljfischer]

16. I'm strongly against using AppHMIType here. I think using AppServiceType is much more useful for a few reasons. First, with the hmi type system, the head unit only knows of two possible active apps: media and navigation. This precludes the possibility of the head unit having a single "weather" widget (or when app services expands, a "voice assistant" widget, etc.) with the user's favorite service. I think using app services here gives all the same benefits, plus more (such as automatic redirection of other RPCs like SendLocation, and remote control ButtonPress).

    I'm having a bit of trouble fully parsing your latest comment on this issue, but it appears that your solution is to limit apps to essentially one widget unless they support multiple AppHMITypes. I would actually be more comfortable with simply limiting apps to a single widget period for now, until we can better figure out a way to support multiple widgets. For example, in the current proposal if an app is a MESSAGING HMI type and an INFORMATION HMI type, and provides widgets for both, how does the system know which widget to present in the situations I provided S2 or D1 above (copied below as well)?

There's are basically two possible user experiences for widgets: static placement and dynamic placement. Static placement is when the user selects a widget to be placed in a slot and covers several possible use-cases:

a. S1 - The user selects from a menu of all possible widgets to place it into a slot. b. S2 - The user selects the app they want to place into a slot and the system determines which widget from the app to actually place into that slot.

Dynamic placement covers several possible use-cases:

a. D1 - The system chooses which widget to place in a slot based on an app being recent (e.g. the user went from an app screen to the main screen and the system wants to place that app's widget in a slot. Similar to iOS and Android recent app screens but with widgets. b. D2 - The system chooses which widget to place in a slot based on the active app for a given app service type. e.g. There's a slot for the active navigation / music / weather app and the system places the active app's widget into that slot.

I think it might be simpler to only allow one widget per app (per display) for now, and expand later in a separate proposal when we can think through it in a more focused manner (we'd probably have to remove the system capability for sending back max screens / windows for now so that OEMs don't just change the value and we end up with a Wild West of OEM UX).

Alternatives:

a. Adding a new RPC to change the primary window / screen for a display per window / screen type. b. Using a DeleteScreen / CreateScreen RPC to swap out for a PrimaryWidget ID

I don't think anything else would work at all, and (b) would suck UX-wise pretty bad (e.g. the currently displayed primary widget is removed from its slot on the head unit and the new one isn't added back into that slot). I think (a) is the only viable alternative.

So, my suggestion is this:

<function name="CreateScreen" messagetype="request" since="5.x">
    <!-- Everything that's already there -->
    <param name="appServiceAssociation" type="AppServiceType" mandatory="false">
        <description>If provided, this widget should be used in dynamic situations when this app is the active app for the service provided. e.g. If the app is the active "media" app service, this is the widget you want displayed on the system if the system only picks one for a "media" widget slot. This overrides the `MakePrimaryScreen` RPC, which is used in dynamic situations in which the system wants only one widget representing your app. If the app has not provided a manifest for this service type already, this RPC will be rejected with resultCode `DISALLOWED`. If this parameter is not provided, the widget will not be associated with a service.</description>
    </param>
</function>

<function name="MakePrimaryWidget" messageType="request" since="5.x">
    <param name="screenID" type="Integer" mandatory="true">
        <description>When sent, the widget associated with the screenID provided will become the primary widget for this app. If the system wishes to show only one widget from this app, and this request doesn't involve an active app service, or the app does not provide a widget associated with an app service (see `appServiceAssociation` in `CreateScreen`, then the widget specified here will be used.</description>
    </param>
</function>

While it's not ideal to use a separate RPC for this functionality, all other considerations I could think of had large downsides. The managers could hide away this RPC and make it easy for app developers to use.

New Issue (17): It appears that a widget cannot use the SubscribeButton RPC. Does that mean that a MEDIA widget cannot provide play / pause buttons on their widget? Is that something we should include?

[theresalech]

The Steering Committee voted to return this proposal for revisions, as the author has been working on updating the proposal to include the agreed upon items (1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14) in PR #698. The Steering Committee will continue to discuss and work through the open items (8, 15, 16, 17) once the revised proposal is back in review.

[theresalech]

The author has revised this proposal to include the Steering Committee's requested revisions, as listed in this comment. The revised proposal is now in review until 2019-05-14.

[joeljfischer]

|8. I remain in favor of "mirror" terminology.

|16. I'm in favor of your serviceType solution for dynamic widgets. It permits app types like weather, and in the future voice assistants, or VOIP. It also allows deeper integration into the system (navigation, media, etc.). And it will allow us to create automatic widgets for developers if they so desire using service data.

It solves a lot of problems, though it won't solve the problem of pulling one widget per app if, for example, you wanted to use a "recent apps" screen using widgets. Some possible solutions I thought of:

1. Add a primaryWidget flag in addition to the serviceType parameter.
2. Ignore that use case for now.
3. Tell devs / core that the first widget they create will be the one used in that use case.

I think I lean to (3). If they DeleteScreen that widget, the 2nd (or next) widget created would be used.

|17. I didn't see any response on this one. What do you think about widget button subscriptions for media apps?

|18. As a note, it appears that mandatory parameters are added to existing HMI_API RPCs (such as OnAppActivated. Therefore, this change would be a major version change for Core to 6.0. I did not see this noted in the proposal.

|19. Adding dayColorScheme and nightColorScheme to Show may present some problems. Currently, the developer is only allowed to update their color scheme when they change templates using SetDisplayLayout. I would assume the same restriction would apply when they moved over to Show? e.g. The core implementors must reject a Show request that updates dayColorScheme or nightColorScheme when layout is not also set.

|20. I don't see it mentioned in the proposal under the Window capabilities section, but the DisplayCapability struct should only update with changed parameters. I'm unsure if the WindowCapability struct should do the same.

|21. I don't believe there's a way to see if soft buttons support text, and it appears that your example widget designs don't support images (and this may be common). I think this is probably a very good time to add the ability to add a new TextFieldName for soft button text (to determine length of soft button strings), and to add a new SoftButtonCapabilities parameter, textSupported. It will probably be important for widget soft buttons.

|22.

Moving window metadata will require more effort from OEMs and app consumers to implement this feature. The metadata needs to be sent twice, in the responses but also in the system capability notification.

Can you explain this more? I assume you mean the RegisterAppInterfaceResponse?

[kshala-ford]

|8 All I can offer is to rename the parameter to duplicateFromWindowID. Again: We duplicate the request for the window. Mirror implies the exact same presentation of the data and last "mirror" is too close to mirror projection features.

|16 my biggest concern with using the service enum is the lack of permission management. I could not find any reference to policies with app services. To my understanding a media app could easily publish weather service data? I believe we anyway need to keep AppHMIType aligned with AppServiceType. It would be very helpful if you or Joey could shed some light on this topic. If there is no alignment here I would rather see this proposal to be accepted without this additional feature until we have found a solution.

It also allows deeper integration into the system (navigation, media, etc.)

Can you explain this deeper integration?

And it will allow us to create automatic widgets for developers if they so desire using service data.

Same here? More explanation please. Are you planning to auto-create widgets from the libraries which get filled with service data?

It solves a lot of problems, though it won't solve the problem of pulling one widget per app if...

I don't think a primary widget is needed. Also I don't want to add a widget specific parameter to CreateWindow which in future is supposed to be used for main windows as well. We could add to PredefinedWindows new elements like PRIMARY_WIDGET. The ID could be 1 or any other ID that we specify. The primary widget can optionally be associated with a service type. If the developer creates a widget with this ID it would be selected to appear in the recent app widget area except the widget is already visible as a service based widget or because the user has pinned the widget to be constantly visible. It all depends on how the HMI implements widgets; all we should do is to give a hint to the HMI. The developer cannot create another "primary widget" as it would fail with INVALID_ID.

|17 I saw your note but I had no chance to respond to it outside of the proposal. (Hard) button subscription is a system RPC. As a media app when you subscribe to PLAY_PAUSE you will receive hard button press on > || if you're the active source. In SYNC3 (and I think sdl_hmi does so, too) we add some kind of hard coded soft button in the center of the media template. We would not change anything to SDL regarding button subscription.

|18 that's a mistake. The description mentions "if this param is not included...". Consider the parameters for OnAppActivated and OnAppDeactivated as optional.

|19 Why would they? I don't see any need to do so? It could be easily abused by setting the same template. Why add a barrier if it's so easy to work around it?

|20 I'm not sure what you mean. Right now only the window that changes should be reported. See here

|21

I don't believe there's a way to see if soft buttons support text

I think until now we expect soft buttons to always support text.

and it appears that your example widget designs don't support images (and this may be common)

Or course they do support graphics. I don't see why widgets shouldn't support graphics.

I think this is probably a very good time to add

I have focused on restructuring the existing capabilities without adding new elements. I don't see the need or importance for your proposed addition but you can feel free to create another proposal for it. The current widget proposal has a lot of content which is not in scope of widgets and I don't want to add more things to it which are not related to widgets.

|22 RAI response and SetDisplayLayout response. The OEM has to provide display capabilities and window capabilities in two different ways at least until core 6.0. I think this is more effort. Of course it's not too much work but I wanted everyone to understand that the old and new structure must be supported by OEMs.

[joeljfischer]

|16

my biggest concern with using the service enum is the lack of permission management. I could not find any reference to policies with app services. To my understanding a media app could easily publish weather service data?

I just double checked the developer portal, and there creating app services does require requesting permissions via policies.

Can you explain this deeper integration?

Sure, I just meant a more formalized way to describe the "current system" media, navigation, weather, etc. app. This is similar to what currently exists in Sync 3 with the current media app data being presented on the Home and Audio screens, but more formalized via service data, which prevents the possible case of the media app going to a different template and then the media data is no longer mainField1 / 2 / etc. (as an example).

Are you planning to auto-create widgets from the libraries which get filled with service data?

I think that would certainly be possible and even relatively easy if there were a proposal to do so in the future.

We could add to PredefinedWindows new elements like PRIMARY_WIDGET.

The main downside I see to this, as opposed to my other suggestion of using the "first" created, and if the "first" is deleted then falling back to the "second" created, is that if the PRIMARY_WIDGET is deleted, and the developer wants another current widget to take its place (let's say it has id 2), then the developer needs to delete and re-create that widget into the PRIMARY_WIDGET slot. It's not ideal, but it's also not terrible. I think I can live with it.

|17. The concern that I have here is the ability for a widget to display the "soft" subscription buttons that e.g. Sync displays (this usually happens on the main window when the subscription happens). I'm fine with not doing that and simply having the developer make soft buttons that do the same thing.

|18. 👍

|19. The barrier can't be worked around because Core prevents it from changing the color scheme if you set the same template that is already in use.

|20. I see that note now. 👍

|21.

and it appears that your example widget designs don't support images (and this may be common)

Or course they do support graphics. I don't see why widgets shouldn't support graphics.

I typed "images" when I meant "text" 🤦‍♂

I do think that this is an important thing for good widget support. If developers have no way of knowing that a soft button doesn't support text then they may end up with creating soft buttons that work on some systems but not others. These features are important for widgets to work well on a wide variety of systems, and not just one OEM's. If OEM 1 supports text in a widget soft button and OEM 2 does not, and the developer tests only on OEM 2's system, then the app's support for OEM 1's system will be poor or broken.

Because widgets makes an implicit change to have soft buttons that don't support text, there should be a corresponding capabilities change to describe whether a soft button supports text. This is a small change but I believe it is important.

|22. 👍

[Jack-Byrne]

16. Couple of things:

i) Core will validate against policies the app service type an app is allowed to publish. After this, if a published media service app sends weather data in the OnAppServiceData notification, it will be up to the consumer app to know that data is wrong.

ii)

Are you planning to auto-create widgets from the libraries which get filled with service data?

~This is already possible when using the HMI as an app service consumer and we have started implementing this as a feature into the Generic HMI.~

Edit: misunderstood the question.

iii) CreateWindow RPC has a param widgetType of type AppServiceType. Note that AppServiceType is marked as Documentation only since the app services RPCs use type String for all of the service types. Should widgetType be changed to be type String to keep consistency?

The reason for this in app services was that Core could not validate enums for future app service types that an older module does not know about. That part of app services does not really apply to widgets so if AppServiceType is kept as the type for widgetType, then the Documentation tag should be removed from AppServiceType in the spec.

[joeygrover]

Just wanted to say thanks @kshala-ford for all the hard work you've been putting into this proposal. For me at least, the latest revision has a very understandable and comprehensive description of the solution.

| 8. I think this works. The argument we had was that the param originally sounded like it was duplicating the window not the update to the window; that's why mirror made more sense with that param name. But the suggestion helps clear that up. Other suggestions (in case you like one better, I'm indifferent at this point with at least the suggested renaming) duplicateUpdatesFromWindowID, copyUpdatesFromWindowID, duplicateWindowIDUpdates, duplicateUpdatesFrom, duplicateUpdatesFromID

| 16. I think something like PRIMARY_WDIGET or a flag for primary of that window type would be the best option. It gives the developer the clear instruction that this is what will be used and not rely on module integrations that may perform differently. If a primary window is deleted, it will not be replaced until the developer recreates a primary window. (Using window in general here in case there are new window types beyond MAIN and WIDGET that would follow a similar use case as WIDGET)

| 19. I believe in the proposal for template colors we specifically stated that the updates should only occur when a template is being changed. I would have to find the specific call out, but I know it was a discussion point. Another option would be creating a smaller struct that held both params to specifically call out that it should only happen during a layout change. (Ignore the name, it's not great)

<function name="Show" functionID="ShowID" messagetype="request" since="1.0">
 :
 :
   <param name="layoutUpdate" type="LayoutUpdate" mandatory="false">
    <description>
        If included the window should update the supplied layout configurations (layout type and color scheme)
    </description>
  </param>
</function>

<struct name="LayoutUpdate">
  <param name="layout" type="String" maxlength="500" mandatory="true">
    <description>
        Predefined or dynamically created window layout.
        Currently only predefined window layouts are defined.
    </description>
  </param>

  <param name="dayColorScheme" type="TemplateColorScheme" mandatory="false" />
  <param name="nightColorScheme" type="TemplateColorScheme" mandatory="false" />
</struct>

| 21. So we have to answer are widget SoftButtons required to support text? If yes, then they follow the same guidelines as other soft buttons on the system and we could supply a proposal later to have a system wide change. However, if there is a case we believe that will only support images/graphics, then we HAVE to include a new param in the SoftButtonCapabilities struct. Forcing a deve into a trial and error situation is never a good practice. It is a simple addition that doesn't have too much of an effect if not included.

<struct name="SoftButtonCapabilities" since="2.0">
: 
: 
    <param name="imageSupported" type="Boolean" mandatory="true">
        <description>The button supports referencing a static or dynamic image.</description>
    </param>

+    <param name="textSupported" type="Boolean" mandatory="false"  since="X.X">
+        <description>
+                The button supports the use of text. 
+                If not included, the default value should be considered true that the button will support text.
+        </description>
+    </param>
<struct>

[kshala-ford]

I'm leaving out numbers we have agreed on. (17. 18. 20. 22.)

@JackLivio Regarding 16 iii) We can make it of type String, though enum would be handy for the app developer to know what to put into the parameter. Do we know about service types supported by the HMI? Is there some capabilities with a list of supported services?

|8. Can we agree on "duplicateUpdatesFromWindowID"?

|16 I think the actual use case is to have recent apps with widgets. Therefore I would suggest a new "PredefinedWindows" called RECENT_WIDGET named after the app name. The benefit of a predefined window is that the HMI can precreate it and return it in display capabilities after registration. I think this name fits better than having a primary widget. This predefined recent widget solves the problem of deleting and creating as it makes it obsolete. A new show can overwrite everything on the recent widget. What are your thoughts?

|19. In the color scheme proposal I found:

A SetDisplayLayout request with the same layout as is currently displayed will result in the current template to change color."

Either we do Joeys suggestion or we make "layoutUpdate" parameter optional.

| 21 Joeys suggestion of the parameter looks reasonable. If that's all we need we can easily accept this one for the revision.

[joeljfischer]

|19.

In the color scheme proposal I found:

A SetDisplayLayout request with the same layout as is currently displayed will result in the current template to change color."

That is true, however it then says in the next sentence:

SDL Core should track the number of attempted SetDisplayLayout requests with the current template and REJECT any beyond the first with the reason "Using SetDisplayLayout to change the color scheme may only be done once."

So there may have been a little confusion there. The last sentence I quoted is what is implemented. I think we're in agreement that we'd have to do the same thing with the color scheme in Show as mentioned.

[kshala-ford]

Yes. I'm sorry I haven't read and understand this sentence in the first place.

|19 still: I think I just found another issue with color scheme and the fact that it requires a template change. In the widget proposal there is this:

The color scheme parameters are related to the app's color scheme. Setting a new color scheme parameter will change the color scheme of all windows of the app. Color scheme parameters will ignore the window ID parameter.

saying that color scheme should be applied to the app and to all windows, but the new layout will only be applied to the specified window (not to all windows).

Here are my suggestions (in hope we can agree to a solution for today): a. Have color scheme per window. The RAI color scheme will be applied to the main window and will be the default for new windows/widgets. Show with new color scheme is always window specific and doesn't update the defaults. In this case I would accept Joey's suggestion. b. Allow color scheme change without template change. A single color scheme is applied to all windows. The rule to REJECT would be removed. In this case I would keep the params as they are ("layout" param needs to be mandatory="false" though).