[Accepted with Revisions] SDL 0273 - WebEngine Projection mode

[theresalech]

Hello SDL community,

The review of the revised "SDL 0273 - WebEngine Projection mode" proposal begins now and runs through May 19, 2020. Previous reviews of this proposal occurred January 15 - February 25, 2020, and March 24 - April 28, 2020. The proposal is available here:

https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0273-webengine-projection-mode.md

Reviews are an important part of the SDL evolution process. All reviews should be sent to the associated Github issue at:

https://github.com/smartdevicelink/sdl_evolution/issues/912

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of SDL. When writing your review, here are some questions you might want to answer in your review:

• Is the problem being addressed significant enough to warrant a change to SDL?
• Does this proposal fit well with the feel and direction of SDL?
• If you have used competitors with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study? Please state explicitly whether you believe that the proposal should be accepted into SDL.

More information about the SDL evolution process is available at

https://github.com/smartdevicelink/sdl_evolution/blob/master/process.md

Thank you, Theresa Lech

Program Manager - Livio theresa@livio.io

[Jack-Byrne]

1. This statement makes it sound like this should be the behavior for all apps that use PROJECTION should be activated into a web page?

The app HMI type PROJECTION should be enabled for in-vehicle apps. When apps with this HMI type are activated, the HMI should make the web page of this app visible on the screen. This web page will become the main window of the application.

2. I don't really understand the need for restricting Webengine projection apps to not be able to send show requests.

3. Was there an issue with creating a new app hmi type specifically for webengine projection apps instead of reusing type PROJECTION? I apologize if it has already been discussed.

4. Has there been any discussion related to what existing SDL features these webengine projection apps will actually find useful?

5. As discussed in previous issue threads, this proposal makes some pretty fundamental changes to SDL's principles. I wish there was more of a story/justification in the proposal for why the SDL project would benefit from implementing this feature.

[kshala-ford]

Hi @JackLivio. Thanks for you review.

1. Do you mean "it sounds like this should be the behavior for all apps" like apps from any platform? The proposal is related only to applications running on the IVI in the WebEngine as proposed in SDL-0240. Just like SDL-0031 apps with permission to use PROJECTION should present their own UI.

2. The restriction is only related to the app's main window which is not visible at any time and technically it wouldn't exist. Show would still work for any other window like widgets. Successfully replying to Show isn't of a correct behavior because the addressed window just doesn't exist. Instead Core could return without executing the request. If there are use cases where projection apps can still show the main window, I'm happy to revisit this item.

3. No worries. No, this wasn't already discussed. The first version of this proposal was proposing a new variant but after reviewing I realized the similarities to the PROJECTION mode which allows apps to present their content. The difference is the used technology behind the scenes. Instead of streaming video we would use the website and the html DOM.

4. When we planned and designed the feature we discussed this point and we believe there are plenty of SDL features that are useful. The most important items are app policy management and app lifecycle. We would like to streamline the implementation to manage activating apps, deactivating or closing apps or run apps in background. Besides the benefits for us implementing projection apps, it's also beneficial for the projection app developer to use all the SDL features like app services, remote control, vehicle data widgets and many more.

5. In fact the proposal doesn't make any change to the SDL principles that don't already exist. The same rules apply to webengine projection apps as for mobile projection apps as described in the accepted proposal SDL-0031. The only difference is the implementation to project (no video streaming but show the app's webpage).

[Jack-Byrne]

1.

Do you mean "it sounds like this should be the behavior for all apps" like apps from any platform?

I know you meant only for web engine apps but I just wanted to point out how someone might find it ambiguous.

2. I think when we create these restrictions we end up complicating things in the future when we do finally create a use case. Maybe an OEM wants to create a hybrid view that dedicates part of the main window to traditional templates and part of the window displays the webengine?

Also the HMI implementation could choose to respond with failures for show requests for webengine apps. I don't think its a necessary requirement for Core.

3. What if we create a new app hmi type, "WEBENGINE_PROJECTION"? Open to other naming.

FYI I believe there are specific rules/exception related to the hmi state & level for when the app is a PROJECTION application, similar to how NAVIGATION apps have special rules. Do we want those same rules to apply to webengine apps?

4. Ok thank you. I think this was a good response and should even be included in the proposal. Relating to 5, I know what content goes into the motivation section of the proposal is subjective, but I think that section could use some extra material. I find it helps tell the story and vision for new features.

[kshala-ford]

1. Got it. I can modify the proposal and state following:

The app HMI type PROJECTION should be enabled for in-vehicle apps. When in-vehicle apps with this HMI type are activated, the HMI should make the web page of this app visible on the screen. This web page will become the main window of the application.

2. I get your point and I would support Show requests when we have demand of such a use case. I also thought of a hybrid use case but we decided against it as we need to focus on the requested set of features. Also we want to stay efficient with the given time and budget. If another SDLC member would like this hybrid feature included we're more than happy for contributions. Please note that such a hybrid feature should be agnostic to other projection modes (mobile nav and mobile projection), so it could make sense to propose this feature separately. Just so much to the hybrid idea: With this feature it would be even more important to provide capability information to the app and properly reply to Show requests. Otherwise apps would not know if the template main window exist. Sending RESOURCE_NOT_AVAILABLE in projection only makes sense telling the app that the main window is not available. Also window capabilities are in place to provide the capability information.

3. I'm not sure if adding a new HMI type that behaves the same as PROJECTION is a good idea. the HMI type would be app platform specific and I would prefer a more unified variant. Core can distinguish between a mobile projection app and a non-mobile projection app at least by the (secondary) transport type but also by device info and other sources.

You're right. NAVIGATION is a special HMI type and there are special rules in the scenario of multiple nav apps. This proposal only describes PROJECTION. Another proposal for WebEngine based nav apps is in the backlog and dependent of this proposal.

4. Yes I'm more than happy to add my comment to the motivation explaining the benefits of SDL.

[joeygrover]

I apologize for my late comment here. I really wanted to get a good understanding and make meaningful comments.

6.

With the current depth of this proposal, the HMI type should be used by 1st party OEM apps only. With future proposals and workshops the SDLC could open the HMI type to 3rd party by creating and defining proper driver distraction and user interface guidelines.

I do believe OEMs should be able to use all the features of SDL, like this proposal, as they wish when it comes to their own apps. What they do for their own apps does not affect other OEMs implementations of SDL (in most cases). What I do worry about is the effect a proposal like this will have on the ecosystem for third party apps. While there's an argument that this is operating in the same way the SDL-0031 works, I believe that is not completely accurate.

With this being for locally stored applications, there is not as much of an adverse effect to circumvent the SDLC app certification and could therefore be implemented before the SDLC agrees upon the guidelines for these types of apps. This would put any OEM who is following our path to a disadvantage compared to the OEM(s) who have chosen to ignore it. We simply do not have a way to effectively monitor and enforce this. Maybe the SDLC members do not feel this is a concern though. I can imagine Ford doesn't, but would like to hear from other members specifically.

7.

To the author's knowledge there is no impact to existing code unless there are barriers implemented in SDL Core preventing an in-vehicle app from registering with the PROJECTION HMI type.

I think there could be a big impact to the screen managers that will be created for the JavaScript Suite. Regardless of whether this proposal is accepted or not, the screen manager will need to be implemented to support templated implementations so how that manager works when there is no main window will have to be addressed.

8.

There are no alternatives available that the author feels competitive to the projection mode using a WebView.

I think there are some other items that could enhance the SDL experience without adding this feature:

1. Better user interfaces from OEMs. Most seem to be designed by engineers or lawyers. App partners should be involved in designing templates to understand what they want to see.
2. Create standard HMI designs and reduce the extra work the developer has to handle at run time for different UIs for each IVI.
3. Implement all UI features including template colors.

These would help create a better SDL experience for applications from all platforms instead of just one, which is more competitive in its own right. These actions help SDL fall more in line with what competitors are doing, but SDL could do better.

9. I would like to see how this changes the previously supplied flows for web engine apps that simply start a new window but returns to the SDL HMI. This would instead keep the opened web engine instance in focus, so the flow would change. I'm assuming just using the HMI type of PROJECTION the HMI would know what to do.

[theresalech]

The Steering Committee voted to keep this proposal in review, to allow the author time to respond to the comments, and for the Project Maintainer and SDLC members to continue their review and discussion on the review issue. It was urged that all SDLC members, especially OEMs, review this proposal and understand how it will impact their current and planned future implementations of SDL.

[kshala-ford]

@joeygrover

7. At the time of writing this comment, the javascript managers are not part of any accepted proposal. The proposal that adds the screen manager to the javascript suite should consider reading the window capabilities and behave accordingly. This behavior should be considered for the screen manager independently of this proposal.

8. I think I was referring to "what other projection alternatives exist besides video streaming". With regards to improve SDL templates I agree to your list and I would append a few more items. This proposal is only about suggesting to use a different presentation technology than video streaming for projection apps addressing the concern of how to realize SDL-0031 for webengine apps.

9. The HMI would know what to do when apps are activated with ROJECTION hmi type. It should know from the device info that the application is a locally installed webengine app. Actually the the policy app id should be more than enough as the HMI installed the application before and send "AddAppProperties" to Core with the policy app id.

[joeygrover]

7. At the time this is true, and I was hoping the author could help provide their insight on how it work work even still. However, I suppose another piece of this is that the entire multiple window system and widgets manager update proposals have never been submitted. So maybe that should be a priority which will include such changes. Was this something the author or another person from their company could handle?

8. The reason I bring it up is you stated there is no alternative that supports the level of advantages as this feature. This is the piece I disagree with and why I brought up those points. The reasoning behind this type of UI is to provide a better UX and simpler SDL integration, but as stated this could be improved in other ways that are not as brute force of a change and all SDK platforms would benefit from. I do not see the author agreeing with me on this, so the point is more for the SDLC; no further discussion is necessary here.

9. I can understand what triggers the behavior, but I would like to know how it affects the previously supplied flows for web engines applications. Can you please provide such flows that are updated to reflect such an application?

[kshala-ford]

7. Let's not digress. SDL-0216 introduced the window capabilities and the default "main window" which this proposal suggest to be the projection window. The proposal states The window capabilities of this open main window will be empty except for the window ID and physical button capabilities. which has effect to the screen manager as it's already reading the capabilities. If you take a closer look to the subclasses of the screenmanager in sdl_ios and sdl_java_suite you will see that the sdl libraries are already taking window capabilities into account. There is no projection mode specific behavior required for the screen managers.

I think the second point of Jack's comment is closely related to this point as he describes the scenario if an app can be PROJECTION but also have a template main window (some sort of hybrid mode). If we are more specific, this hybrid mode would require modifications to the definition of the predefined default main window. I think the easiest approach would be to introduce a new projection window type and a predefined main projection window which will be added to the window capabilities array if the app is a projection app (including mobile navigation apps). Still, the default main window may be empty when not supporting text, buttons and graphics if the HMI isn't including a templated main window. Alternatively it could be omitted from the window list but I'm afraid the libraries could crash as the expect the default main window to always exist.

I realize that the below statement from the proposal isn't really valid as it's in the scope of the HMI. Core should not overrule the decision of the HMI.

The window capabilities of this open main window will be empty except for the window ID and physical button capabilities. Show requests that address the main window won't have any effect on the HMI. If the app sends this request, Core should return an unsuccessful response with the result code RESOURCE_NOT_AVAILABLE.

I hope my insight was helpful. If you have any further questions or feedback to the suggested window capability change please don't hesitate to send me your feedback. I'm more than happy to help.

8. I am sorry but if we are not having a discussion to your point please allow me to provide my comment for the SDLC with regards to template improvements and the projection mode.

I agree to the comment from the project maintainers that the SDL templates can be improved in different ways allowing app developers to create great applications with an awesome graphical user interface. Ideas that I would add to the list of the project maintainers are

• Enhanced text layout allowing app developers to impact the font of text fields beyond bold, italic etc.
• Template layouts allowing apps to specify the window layout using ... grids or horizontal/vertical separators where text fields, image views and buttons can be added as needed.
• Responsive layouts which can be seen as an extension to layouts as one templates can fit to many different types of screens.
• HMI theming which is different than just unify the template layout but allow apps to specify the layout and the OEMs to specify the window theme.
• Web based templates where a web view can be projected controlled by the SDL libraries using web technology allowing SDLC templates. This idea could go together with HMI theming.

Please don't forget that these are my personal ideas I have in mind that could be an awesome enhancement to the UI component of SDL. But I know all these enhancements are really difficult to realize for many different reasons. Still I would love to see enhancements to the templates become reality.

The mobile projection mode was proposed for more advanced applications. I'm not sure how far I can speak about use cases before I get into trouble but we are looking for the ability of showing complex controls like sensor meters (g-force etc.) that are going beyond text, image and buttons. This is really difficult to solve in a template approach. Other use cases are to show videos e.g. from cameras. Also customized map solutions are considered (though this is more NAVIGATION related). We are looking for the ability to implement OEM specific or non-consumer features with the new in-vehicle application platform. The web presentation allows us to move forward with delivering the demand in a realistic period of time.

I want to let everyone know that I believe that the template improvements can benefit from the web based presentation mode as a tool for any kind of application independent from where they are running (mobile, cloud, in-vehicle). Creating templates for the HMI and make them robust is a difficult task and transfer the complexity would be a gate with many opportunities for template improvements.

This vision of an SDL user interface can only be reached step by step. Starting with the projection mode in a constrained environment is a start and I believe it will be inspiring and educational for improving the design and layout of SDL templates.

9. The flow would only change minimally. If a webengine app is in HMI_FULL the HMI would not keep the app's browser view hidden but show it in the viewport on the HMI. The flow of the app's HMI level doesn't change. I can modify the flow that is included in SDL-0240 and mention where the HMI could make the web application visible on the HMI. I'll come back to you once this is done.

[theresalech]

Hi @kshala-ford - please find collective comments from the PM below. Thanks!

1. Sounds good, thank you!

2. With this feature it would be even more important to provide capability information to the app and properly reply to Show requests.

We already provide the display capabilities for an app in the RAI response. If the head unit does not support show requests for a webengine app, the display capabilities will let the app know. It is not necessary to say webengine apps cannot send shows, when that information can be propagated to the app by the head unit's declaration that those fields are not supported for the main window or the selected template.

3. What is stopping a webengine app from wanting to be a traditional PROJECTION app and stream video to the head unit? Taking that question into consideration complicates this even further and suggests we need a new app hmi type.

If we keep PROJECTION as the app hmi type for webengine apps, then in Core there must be a new application API named something like webengine_projection_enabled(). This will return true if the app hmi type is PROJECTION and some secondary form of identification that it is a webengine app.

We don't believe that the lack of a secondary transport is enough of a reason for determining if a PROJECTION app is a webengine app or not. There could be non webengine apps that use the websocket transport in the future. There needs to be an additional identifier of the app type or we should use a different app hmi type.

Also the existing Core application api mobile_projection_enabled() needs to be extended to consider if the app is webengine or not. This method is called in too many cases to list here. Currently it only checks if the app hmi type is PROJECTION.

Our suggestion is to create a new app hmi type for webengine projection apps.

4/5. Sounds good, thank you!

6. We think this may have been missed in your initial response. Please address.

7. So if we return to the point of contention on being able to address potential issues with other proposals and take a look at one of the latest comments from this proposal's author on SDL-0275, it seems we are in agreement that proposal issues can be debated and raised referencing other proposals currently in the process. The default behavior for the screen manager is to still assume there is a "main window", if the capabilities aren't available it has default values it uses for items like text. Should we update these default values? Should we enforce that the capabilities have to be supported and sent? Currently there is actually no hard requirement that the display capabilities have to be supported and a null value could be returned but the screen manager would still attempt to work through the lack of knowledge. Even if the capabilities were included but set all values to false/null, the screen manager would still be available but everything would fail. I'm wondering if there is a better way to handle this as it is not intuitive for a developer in my opinion. Also, there is no working implementation of including the widget windows into the screen managers which is one of the pointed out features that still works with projection. And while yes, it does "work", the amount of code that it would take for a developer to implement a robust, OEM agnostic implementation could be very large and complicated. I think the outcome to number 2 would have sufficient information to resolve this issue.

8. My point here is that I would much rather see the SDLC focus on these improvements instead of moving forward with this type of solution. There is nothing stopping us from creating new templates that support advanced UX designs. The author has made it clear they feel the projection mode is more important. We simply do not agree on priority and as stated previously; no further discussion is necessary on the point.

9. Waiting for the author to respond with updated flow.

10. Given that PROJECTION and NAVIGATION currently use the NAV_FULLSCREEN_MAP template, when doing "PROJECTION" in this manner, a template type should be defined. So that, for example, an app could switch to a built in template and back.

[kshala-ford]

With regards to 9. We are not planning to change anything of the app activation flow except that the app's WebEngine tab is shown on the HMI

[theresalech]

The Steering Committee voted to keep this proposal in review, to allow the author time to respond to the comments provided, and to provide additional time for other SDLC Members to review and provide feedback.

[kshala-ford]

2, I think the PM is misunderstanding my point. but this item can be closed as it becomes invalid considering the solution regarding 10.

3, I need more time to look into this item regarding the ability to identify a webengine app. This proposal is not in a hurry and I would rather try to avoid adding a very webengine specific HMI type and reuse what's already available.

What is stopping a webengine app from wanting to be a traditional PROJECTION app and stream video to the head unit?

I may be wrong but I don't believe a locally running WebEngine app should capture the DOM and stream it through SDL as a video back to the HMI to be projected. I don't even believe web applications can do video streaming without the need of browser extensions.

6, I did recognize the PM comment and do understand your concern but they are not valid. I don't understand why the PM is bringing up these concerns but silently accept the original projection mode proposal.

7, I appreciate looking into my comments and providing valid feedback. I think I can speak for all members of my organization that we always support resolving issues even across proposals currently in review. I want to note that not doing so became common practice in the past and that's what I was referring to while still providing feedback that the managers should respect window capabilities independently of this proposal.

If you agree I would like to skip (and potentially invalidate) this item and invite you to have a look at 10. I think the template variant invalidates a lot of my previous expectations and your concerns.

10, You're bringing up a very good point here and I spend some time understanding how PROJECTION and NAVIGATION apps work at the moment. Please bear with me if I still misunderstand some items but I would like to stay close to the existing implementation and suggest a proposal revision to use a projection template name instead. NAV_FULLSCREEN_MAP sounds really odd for projection apps and may be a template called WEBENGINE_PROJECTION should be added.

When activating WebEngine apps with HMI type PROJECTION the HMI should initially start the app with the new template for the main window. This new template behaves similar to the video projection template but it shows the WebEngine app's DOM instead of rendering a video. The HMI can provide specific window capabilities when the main window is set to this new template. The HMI should still compute Show requests. The HMI should allow apps to switch to regular templates using Show.templateConfiguration. The templates should show the content (text, buttons, graphics) as specified by the app in previously sent Show requests. Please let me know your thoughts.

(Still 10.) think above suggestion makes item 2 and 7 invalid as the template approach matches existing window capabilities mechanism and Show RPCs are still allowed for the main window.

EDIT: (still 10 ^^) The new template should only be allowed for the default main window.

[theresalech]

A quorum was not present at the 2020-02-04 Steering Committee meeting, so a vote on this proposal could not take place. This proposal will remain in review until our next meeting on 2020-02-11.

[theresalech]

Hi @kshala-ford - please find collective comments from the PM below. Thanks!

3. We believe the AppHMIType enum values are not supposed to apply to multiple features. All of the app hmi types that exist correspond to one specific type of application or behavior. We feel that creating a specific AppHMIType for WebEngine projection is necessary.

6. While the author might not see this is valid, the PM has brought it as a point of contention and therefore believe it to be valid. I'm sorry, but we will have to ask you to provide feedback again. Thank you! Speaking for Joey who brought up the point, he was personally not able to be available during the review of the mentioned proposal so there was no chance to voice such concerns.

9. Thank you for the flow chart. We still have a question about what happens to an app when it is deactivated into hmi level NONE. Should the WebEngine tab and connection to Core be closed?

10. We agree to creating a new template layout such as WEBENGINE_PROJECTION or WEBENGINE_FULLSCREEN (or other name). We agree that the window capabilities for this template will be used to compute what contents of show are supported.

In the effort to maintain official templates across OEM platforms, we should design a high level wireframe of what components this new WebEngine projection template will include (this is not a UI design doc; it is basic requirements for implementers of the template). We think some important components to include in the wireframe are: the "return to app list" button – this should not be app defined, the add command menu button (or a requirement for the app to implement ShowAppMenu or otherwise implement CloseApplication), and app/template title.

[theresalech]

The Steering Committee voted to keep this proposal in review to allow additional discussion on the review issue. It was again requested that all SDLC Members review and provide their feedback on this proposal.

[kshala-ford]

I'm sorry for my late reply. Due to other activities it's not easy to reply properly to every item in your comment.

3. The App HMI type defines app features. Not the technology (streaming video vs. rendering html). We shouldn't imply the underlying technology from the HMI type. SDL Core can easily distinguish web apps from any other app and therefore behave accordingly to the expected technology to be used. Core shouldn't expect a video and audio stream for local web applications in an SDL manner. Adding a new HMI type is not necessary and causes impact across many different SDL areas including policy. I encourage the PM to also consider implementation effort when suggesting changes. If the PM doesn't agree I would like to have the steering committee to vote. If the vote is to add an HMI type I want consensus on who will be working on the impacted platforms.

6. If I understand it correctly if Joey's concerns couldn't be brought up in the original mobile projection proposal he or the PM are generally against any sort of app projection (except navigation). If this is the case then please consider revisiting the original proposal and suggest to revote rejecting the proposal. WebEngine projection mode is no different to mobile projection except the technology in use is different. This is the only purpose of this very proposal to use web rendering instead of video streaming but the feature stays the same. If you bring up concerns to the feature then they also apply to mobile projection as well.

9. The connection to Core should stay open. This is independent of the app HMI type and also applies to default web applications.

10. I think we can agree on the template based approach though I don't agree to the naming. The name should be applicable to mobile projection as well and should be called PROJECTION or something else more streamlined. I would suggest to return for revision with the agreement to use the template method: When the template is used the app's web page should be presented.

[theresalech]

During the 2020-02-18 meeting, the Steering Committee voted to keep this proposal in review, to allow for further discussion on this review issue.

[kshala-ford]

Adding 11. to the notes about what should be part of the template:

We think some important components to include in the wireframe are: the "return to app list" button – this should not be app defined,

I think this is very HMI dependent. I can say that Ford HMI is not showing such button or any other button on top of the app's viewport. However a dominant "Apps" button exist that guides the user to the app list. I think we agree here as long as the HMI designer makes sure that access to the app list is possible.

the add command menu button (or a requirement for the app to implement ShowAppMenu or otherwise implement CloseApplication),

I do agree. For 3rd party this should be a requirement but in the scope of this proposal addressing 1st party I am hesitant in accepting boundaries as I can see any business case to keep the app open like a fleet app being more important than follow these requirements. Think of car rental apps or OEM apps that want to be present permanently.

and app/template title.

This is again very HMI. But I agree to that HMI designers should consider the template title being shown outside of the app viewport, may be above the app.

[theresalech]

Hi @kshala-ford - please find collective comments from the PM regarding most comments below. We'll need more time to review and respond to 11. Thanks!

3. The Project Maintainer always works to consider implementation effort when reviewing proposals, and the effort required to add an element to an enum set is minimal. Additionally, we believe reusing an existing AppHMIType would result in more implementation effort than creating a new AppHMIType. By the author’s definition that the AppHMIType enum defines features and not technology it should be noted that the proposed feature is not projecting a UI, it is in fact using a custom, brought in UI that the system renders. Since these features are not the same thing (playback of video vs render of UI), it is hopefully agreed that there is a need for a new AppHMIType that clearly defines two different features. Not adding a new AppHMIType also hems us in and doesn't allow new features. For example, imagine that we added a feature for mobile apps to push HTML to the head unit. Would this be a PROJECTION app? How would the system differentiate? We suggest an AppHMIType of CUSTOM_HMI_HTML

6. To reference the original concern:

I do believe OEMs should be able to use all the features of SDL, like this proposal, as they wish when it comes to their own apps. What they do for their own apps does not affect other OEMs implementations of SDL (in most cases). What I do worry about is the effect a proposal like this will have on the ecosystem for third party apps. While there's an argument that this is operating in the same way the SDL-0031 works, I believe that is not completely accurate. With this being for locally stored applications, there is not as much of an adverse effect to circumvent the SDLC app certification and could therefore be implemented before the SDLC agrees upon the guidelines for these types of apps. This would put any OEM who is following our path to a disadvantage compared to the OEM(s) who have chosen to ignore it. We simply do not have a way to effectively monitor and enforce this. Maybe the SDLC members do not feel this is a concern though. I can imagine Ford doesn't, but would like to hear from other members specifically.

I think the feedback provided does not address the real concern. There is no way to enforce only OEM apps to use this feature. These types of apps can be deployed directly to OEMs' head units and circumvent the SDLC entirely; this makes it possible for OEMs to create one off integrations with app partners. Therefore it is plausible that the app partners will require this type of integration (HTML rendering) for OEMs to get access to their application. This puts the ecosystem in jeopardy if it does not align with the OEMs strategy.

We would really like to get other members' feedback on this and make sure their plans align to including this feature.

9. We can agree that the connection can stay open on HMI level NONE as long as there is a way for the HMI to be able to unregister and disconnect an app in case of performance issues (eg too many webengine apps running at one time).

To reference cloud applications, the HMI has a method to disconnect cloud apps in the event the app is using too much of the users data (ApplicationExitReason::CLOSE_CLOUD_CONNECTION). We would like to suggest adding a new HMI ApplicationExitReason and a new Mobile AppInterfaceUnregisteredReason that can be used by the hmi to disconnect webengine applications. ApplicationExitReason::RESOURCE_CONSTRAINT, AppInterfaceUnregisteredReason::RESOURCE_CONSTRAINT.

10. As noted in (3), because the feature is not technically "projection" in that it's not projecting an existing UI or screen to the head unit, and because that terminology is already present in SDL to mean a different feature, we recommend avoiding that terminology here. We'd recommend "FULLSCREEN_HTML" or something similar, but are open to other suggestions!

[kshala-ford]

3. My major concern is that adding a new HMI type has impact to basically every other SDL repository for a WebEngine specific feature. Originally this proposal was suggesting a new type but it was changed for this very reason. I understand that Core complexity increases. Therefore I would like the PM to provide an estimate for the other repos. Sure thing on the SDKs it's basically just adding a new enum element but more interested into the impact to SDL Server, SHAID but also generic HMI.

The only reason why HMI type is being used is because the app's initial template should be the app's webpage. Any other solution is greatly appreciated. Possibly a new param could support the same feature and would help for other platforms. The param could be called RegisterAppInterface.templateConfiguration allowing an app to set the initial template. Not sure how to protect the web view template from apps without permissions. This change would allow to deprecate color schemes from RAI as they are part of TemplateConfiguration.

6. Again, I think the PM needs to compare the concerns with mobile projection.

There is no way to enforce only OEM apps to use this feature. [...]; this makes it possible for OEMs to create one off integrations with app partners.

Same applies for mobile projection apps. These types of apps can be deployed on Apple AppStore and Google PlayStore and circumvent the SDLC as well. But no one is considering as it would be completely against any consortium agreement.

Therefore it is plausible that the app partners will require this type of integration (HTML rendering) for OEMs to get access to their application. This puts the ecosystem in jeopardy if it does not align with the OEMs strategy.

I'm not sure if I'm understanding. Are the PM saying that app partners would require permission to render HTML instead of using templates? Again this is not in our interest to allow 3rd party applications. Our first goal is to support OEM and commercial applications and then work on addressing template issues before we start with 3rd party. There are plenty of ways to improve templates if you look at the list I already provided. But we have to proceed on this proposal in the proposed context.

9. Agree. Adding an exit reason is independent of the projection mode but makes sense for WebEngine apps generally. Add ApplicationExitReason::RESOURCE_CONTRAINT to HMI API and AppInterfaceUnregisteredReason::RESOURCE_CONSTRIAINT to MOBILE API.

If the IVI is 1. running out of RAM or 2. the application uses too much CPU or 3. caused excessive data traffic the application should be closed. The IVI would decide on exiting an application due to hardware constraints.

I am thinking of adding reasons per resource issue. If I'm not mistaken the exit reason should be tracked in the policy statistics. When reviewing the statistics it would be helpful to better understand the exit reason of an application. Any suggestions?

10. my suggestion for a template name would be WEB_VIEW.

[theresalech]

The Steering Committee voted to return this proposal for the following revisions:

• (Comment 1) Add "The app HMI type PROJECTION should be enabled for in-vehicle apps. When in-vehicle apps with this HMI type are activated, the HMI should make the web page of this app visible on the screen. This web page will become the main window of the application."
• (Comment 3) Use HMI type in addition to PROJECTION, and use PM's suggestion for renaming
• (Comment 4) Add to motivation section, including: "When we planned and designed the feature we discussed this point and we believe there are plenty of SDL features that are useful. The most important items are app policy management and app lifecycle. We would like to streamline the implementation to manage activating apps, deactivating or closing apps or run apps in background. Besides the benefits for us implementing projection apps, it's also beneficial for the projection app developer to use all the SDL features like app services, remote control, vehicle data widgets and many more."
• (Comment 9) Specify that when an app is deactivated into HMI level NONE, the connection to Core can stay open as long as there is a way for the HMI to be able to unregister and disconnect an app in case of performance issues. Additionally, add ApplicationExitReason::RESOURCE_CONTRAINT to HMI API and AppInterfaceUnregisteredReason::RESOURCE_CONSTRIAINT to MOBILE API.
• (Comment 10) Use template based approach, with name WEB_VIEW.

The following items will need to be discussed when the revised proposal is back in review: Comments 6 and 11

[theresalech]

The author has updated this proposal to incorporate the agreed upon revisions noted in this comment, and the revised proposal is now in review. The proposal will remain in review until 2020-03-31. As noted in the linked comment, Items 6 and 11 still need to be discussed and agreed upon.

[joeygrover]

For reference, the specific items that were updated from the last review can be found from this merged pull request: https://github.com/smartdevicelink/sdl_evolution/pull/952

[theresalech]

Please find below comments from the Project Maintainer regarding the previous open items, as well as a few new questions:

6. We'd like a little more time to consider next steps here.

11. It sounds like we're in agreement on what should be included here. Can you please add the high level wireframe of components to this proposal?

12. (new)

The application can control the web document object using JavaScript code to manipulate the document object model of the WebEngine app.

Is the proposal limiting the supported languages to JavaScript?

13. (new)

The following text fields mainField1, mainField2, mainField3, mainField4, statusBar, mediaClock and mediaTrack should be used practically on all the base templates. It is recommended that WindowCapability.textFields should not contain these text field names if the OEM has implemented the WEB_VIEW template without these text fields.

Can you add a little more clarification to this statement? It seems the author is stating the templates could include these fields, but that the recommendation is that users don't, is that correct?

14. (new)

Widgets that duplicate content from the main window should still be possible. Despite the window capability, the app should still be able to send Show requests with all the desired content. This content should be duplicated to these widgets.

How is this supposed to work? If the main window doesn't have text field properties, the library is still supposed to send them? Currently the manager layers will either strip/modify that content if some of those fields don't exist. For example, combining 4 lines into 2. We would recommend that widgets are not able to duplicate data from a main window if it is using a WEB_VIEW template.

[AByzhynar]

@theresalech

6. We'd like a little more time to consider next steps here.

We understand and agree.

11. It sounds like we're in agreement on what should be included here. Can you please add the high level wireframe of components to this proposal?

We will create a draft and include in a new revision

12. (new)

The application can control the web document object using JavaScript code to manipulate the document object model of the WebEngine app.

Is the proposal limiting the supported languages to JavaScript?

Yes. At the moment we don't plan to support other languages in order to avoid increasing complexity. If new language support will be desired - it should be done within a separate proposal.

13. (new)

The following text fields mainField1, mainField2, mainField3, mainField4, statusBar, mediaClock and mediaTrack should be used practically on all the base templates. It is recommended that WindowCapability.textFields should not contain these text field names if the OEM has implemented the WEB_VIEW template without these text fields.

Can you add a little more clarification to this statement? It seems the author is stating the templates could include these fields, but that the recommendation is that users don't, is that correct?

Let's just rephrase to make it more clear. If OEM implemented WEB_WIEV template without text fields - WindowCapabilities.textFields should not contain them. WindowCapabilties are window specific. See the part of the HMI API below. All of the fields specified above are used in Show, Alert and similar RPCs which will be ignored by the main app window with active template WEB_VIEW as it is rendered by WebEngine directly. So in case the app sends Show, Alert etc. RPC to the main app window while App UI is rendered on the HMI - nothing will be changed on the HMI main window. This is the reason why it does not make sense to include these fields if WEB_VIEW template does not support their representation. But if afterwards app will change its active template from WEB_VIEW to some other one which has the support of these fields - these fields will be shown on the main screen.

<struct name="DisplayCapability">
    <description>Contains information about the display capabilities.</description>
    <param name="displayName" type="String" mandatory="false" />
    <param name="windowTypeSupported" type="WindowTypeCapabilities" array="true" minsize="1" mandatory="false">
      <description>
        Informs the application how many windows the app is allowed to create per type.
      </description>
    </param>
    <param name="windowCapabilities" type="WindowCapability" array="true" minsize="1" maxsize="1000" mandatory="false">
      <description>
        Contains a list of capabilities of all windows related to the app.
        Once the app has registered the capabilities of all windows are provided.
        GetSystemCapability still allows requesting window capabilities of all windows.
        After registration, only windows with capabilities changed will be included.
        Following cases will cause only affected windows to be included:
        1. App creates a new window. After the window is created, a system capability notification will be sent
        related only to the created window.
        2. App sets a new layout to the window. The new layout changes window capabilties.
        The notification will reflect those changes to the single window.
      </description>
    </param>
  </struct>

<struct name="WindowCapability">
    <param name="windowID" type="Integer" mandatory="false">
      <description>
        The specified ID of the window. Can be set to a predefined window,
        or omitted for the main window on the main display.
      </description>
    </param>
    <param name="textFields" type="TextField" minsize="1" maxsize="100" array="true" mandatory="false">
      <description>A set of all fields that support text data. See TextField</description>
    </param>

</struct>

<struct name="TextField">
  <param name="name" type="Common.TextFieldName" mandatory="true">
    <description>The name that identifies the field. See TextFieldName.</description>
  </param>
  <param name="characterSet" type="Common.CharacterSet" mandatory="true">
    <description>The character set that is supported in this field. See CharacterSet.</description>
  </param>
  <param name="width" type="Integer" minvalue="1" maxvalue="500" mandatory="true">
    <description>The number of characters in one row of this field.</description>
  </param>
  <param name="rows" type="Integer" minvalue="1" maxvalue="8" mandatory="true">
    <description>The number of rows of this field.</description>
  </param>
</struct>

<enum name="TextFieldName">
  <element name="mainField1">
    <description>The first line of first set of main fields of the persistent display; applies to "Show"</description>
  </element>
  <element name="mainField2">
    <description>The second line of first set of main fields of the persistent display; applies to "Show"</description>
  </element>
  <element name="mainField3">
    <description>The first line of second set of main fields of persistent display; applies to "Show"</description>
  </element>
  <element name="mainField4">
    <description>The second line of second set of main fields of the persistent display; applies to "Show"</description>
  </element>
  <element name="statusBar">
    <description>The status bar on NGN; applies to "Show"</description>
  </element>
  <element name="mediaClock">
    <description>Text value for MediaClock field; applies to "Show"</description>
  </element>
  <element name="mediaTrack">
    <description>The track field of NGN and GEN1.1 MFD displays. This field is only available for media applications; applies to "Show"</description>
  </element>
  <element name="templateTitle">
    <description>The title of the new template that will be displayed; applies to "Show"</description>
  </element>

</enum>

14. (new)

Widgets that duplicate content from the main window should still be possible. Despite the window capability, the app should still be able to send Show requests with all the desired content. This content should be duplicated to these widgets.

How is this supposed to work? If the main window doesn't have text field properties, the library is still supposed to send them? Currently the manager layers will either strip/modify that content if some of those fields don't exist. For example, combining 4 lines into 2. We would recommend that widgets are not able to duplicate data from a main window if it is using a WEB_VIEW template.

You are absolutely right here. We do have the same understanding. If app has main window with active WEB_VIEW template and has created widgets with option "duplicate from main window" - Widgets CAN NOT duplicate content from the main window due to:

• the main window is rendered by WebEngine and SDL isn't aware of the main window content
• the main window has the WEB_VIEW template, but widgets can't have a WEB_VIEW template The main window content duplication to the widgets will be possible only after the MAIN window app will change its active WEB_VIEW template to any other one. The template-based content is still there. It's just not showing.

[mrapitis]

The Ford team is also in alignment with the feedback provided above.

[theresalech]

The Steering Committee voted to keep this proposal in review, to continue the discussion on the review issue regarding the open items.

[theresalech]

Thanks @AByzhynar for your feedback. Please find responses from the PM below.

6. The authors have stated that this proposal is for 1st-party apps only, however they have also stated in the proposal and elsewhere that the intention is to eventually open this up to 3rd-party apps as well. We believe that there are many issues inherent to opening up "WebEngine Projection" to 3rd-party apps, which allows apps to build their own UI for their SDL app using Web UI technologies.

The first issue is that it will create app review and certification issues. Due to individual OEM driver distraction restrictions and UI requirements, the SDLC will no longer be able to handle reviews of the app for each OEM. Every OEM will instead need to review the UI and may need to get design and driver distraction teams involved in the review. This will greatly increase review times for developers. In addition, if each OEM / region / supplier requires different changes, it will create fragmentation issues for the app developer, which SDL was designed to avoid.

Second, this will create backward compatibility issues for SDL. Because WebEngine Projection and template implementations require two rather separate development streams, developers are unlikely to want to create a WebEngine app and support an SDL template app simultaneously. That means that many, and probably most, developers will drop support for the template apps if they are creating a WebEngine Projection app. In addition, app developers will likely not want to fragment their app's UI for each OEM. This means that the first OEM to market with WebEngine Projection support will have a huge advantage. Apps that drop template support will also cease to work on older head units.

Due to the above two issues, we believe that OEMs will create 1-1 relationships with developers of larger apps in order to have the app make changes to fit their overall UI and driver distraction requirements. This will result in OEMs that are later to market with WebEngine Projection and lower-cost head units not being able to have many apps available.

The PM's recommendation is to open up a discussion on how 3rd-party apps will be handled if this proposal is accepted. Will there be enforcement steps (such as taking the matter to the SDLC Board of Directors with the power to apply repercussions) to ensure that OEMs don't use 3rd-party content through WebEngine Projection? If not, we need to discuss the viability of this proposal for 3rd-party apps, including the downsides listed, and discuss possible alternatives that support 3rd-party applications in a more sustainable way.

11. Sounds good, thanks!

12. Understood, thank you. Please update the proposal to specifically state JavaScript as the only supported language.

13.

It is recommended that WindowCapability.textFields should not contain these text field names if the OEM has implemented the WEB_VIEW template without these text fields.

To confirm, you are stating that WEB_VIEW can have text fields, but there is no recommendation to have or not have, correct? If so, we believe that "recommended" should be changed to "required".

If WEB_VIEW has the text fields that were previously mentioned, show and alert cannot be ignored, even if they're not supported. Please see this section in the proposal:

1.2. What about widgets? Widgets are not affected by this proposal. They are still available and can be controlled using Show RPC. Any overlay like Alert, ChoiceSets, Slider etc. are also available to the application.

14. If the WEB_VIEW supports the necessary text fields, then a widget CAN mirror that main window. However, for the WEB_VIEW to actually support the text fields they must be shown on the UI. As previously stated, if the WEB_VIEW main window doesn't support the necessary text fields, then a widget can't mirror that window.

[AByzhynar]

@theresalech First of all we would like to thank you for your notes & catches

Please find our answers below.

11. Sounds good, thanks!

The following information will be included in the next revision:

1) It is required from the HMI to contain “MENU” button somewhere outside of the app’s viewport. The exact location of the button relies on OEM

The following restrictions are applied to the "MENU" button:
It should be visible only for WebEngine Projection apps It should be visible only when WebEngine Projection app is in FULL HMI level. If app is deactivated - “MENU” button should disappear from the screen

This “MENU” button will have only one entry by default “USER EXIT”, which will deactivate the current application to “NONE” HMI Level. “MENU” button entries list can be extended by sending AddCommand RPC from the application side. Entries can be removed by sending DeleteCommand RPC from the application. The only unremovable entry is “USER EXIT“

2) The template title also should be visible somewhere outside of the app’s viewport. The exact location of the button relies on OEM The same visibility rules are used as for the "MENU" button

The example could look like:

12. Understood, thank you. Please update the proposal to specifically state JavaScript as the only supported language.

The next statement will be included in the new revision:

JavaScript is the only language supported within the current proposal.

13.

It is recommended that WindowCapability.textFields should not contain these text field names if the OEM has implemented the WEB_VIEW template without these text fields.

To confirm, you are stating that WEB_VIEW can have text fields, but there is no recommendation to have or not have, correct? If so, we believe that "recommended" should be changed to "required".

Agree. In the new revision "recommended" will be changed to "required"

If WEB_VIEW has the text fields that were previously mentioned, show and alert cannot be ignored, even if they're not supported. Please see this section in the proposal:

1.2. What about widgets? Widgets are not affected by this proposal. They are still available and can be controlled using Show RPC. Any overlay like Alert, ChoiceSets, Slider etc. are also available to the application.

14. If the WEB_VIEW supports the necessary text fields, then a widget CAN mirror that main window. However, for the WEB_VIEW to actually support the text fields they must be shown on the UI. As previously stated, if the WEB_VIEW main window doesn't support the necessary text fields, then a widget can't mirror that window.

We have prepared answers for 13 & 14 in the form of use cases:

Precondition 1

OEM has implemented WEB_VIEW template with text fields support

Case 1.1

The application sends Show RPC (Alert etc.) directly to the main window App -> SDL -> HMI: Show RPC request with some text fields

Expected behavior:

No text fields are shown on the main screen. Despite that, the HMI sends successful response with WARNINGS and text info: "Received text fields can not be displayed in the WEB VIEW"

HMI -> SDL -> App: Show RPC response with "WARNINGS"

Case 1.2

The application sends Show RPC (Alert etc.) directly to the widgets

Expected behavior: HMI successfully shows received text fields on widgets

Case 1.3

The application has created the widgets that duplicate the main window screen. The application sends Show RPC (Alert etc.) to the main window

App -> SDL -> HMI: Show RPC request with some text fields

Expected behavior:

No text fields are shown on the main screen. The widgets successfully show received fields.

Despite that, the HMI sends successful response with WARNINGS and text info: "Received text fields can not be displayed in the WEB VIEW" HMI -> SDL -> App: Show RPC response with "WARNINGS"

Note: Received template-based content (e.g. text fields) will be kept by the HMI in order to be shown when the app will change its active template which supports content representation.

Precondition 2

OEM has implemented WEB_VIEW template without text fields support

Case 2.1

The application sends Show RPC (Alert etc.) directly to the main window App -> SDL -> HMI: Show RPC request with some text fields

Expected behavior:

No text fields are shown on the main screen. The HMI sends erroneous response with UNSUPPORTED_RESOURCE

HMI -> SDL -> App: Show RPC erroneous response with "UNSUPPORTED_RESOURCE"

Case 2.2

The application sends Show RPC (Alert etc.) directly to the widgets

Expected behavior: HMI successfully shows received text fields on widgets

Case 2.3

The application has created the widgets that duplicate the main window screen. The application sends Show RPC (Alert etc.) to the main window

App -> SDL -> HMI: Show RPC request with some text fields

Expected behavior:

No text fields are shown on the main screen. The widgets don't show received fields. The HMI sends erroneous response with UNSUPPORTED_RESOURCE

HMI -> SDL -> App: Show RPC erroneous response with "UNSUPPORTED_RESOURCE"

Note: Received template-based content (e.g. text fields) won't be kept by the HMI.

[mrapitis]

Regarding item 6, to @kshala-ford previous point:

6. Again, I think the PM needs to compare the concerns with mobile projection.

These concerns are not unique to the introduction of webview. Projected applications have the whole canvas available at their disposal today.
Regarding the concern around 3rd party apps: As of this time the intent is 1st party only, I believe a future proposal would be required for 3rd party requests. That proposal could also contain further discussion regarding that topic.

[theresalech]

@AByzhynar @mrapitis, thanks for your responses. Please see below for collective feedback from the PM.

6. The argument is actually providing a false equivalence as has been stated multiple times. The biggest issue is that using this technology OEMs can simply start working with 3rd parties immediately as this "OEM only" feature is merged in. Those apps would then only be public after launch. After launch it would also not be visible to most SDLC members. The only way to know that a 3rd party integration was done would be getting ahold of that OEM's vehicle and going through their entire app store.

Mobile projection on the other hand, while technically similar, has other safeguards to prevent as much abuse. First mobile apps will have to go through App Certification if they wish to be used on more than one OEM's hardware. For Android apps this is actually a necessity to not break the SDL ecosystem and host the SDL Router Service. Second, those apps go to a publically available app store and would be the same applications that other OEMs use for their integration. Therefore all OEMs and the SDLC in general would be aware of such abuse much more quickly and in some cases be able to stop it before it releases. Lastly, mobile projection mode on mobile devices has many more limitations and performance issues that might deter 3rd parties from doing such an integration as the templates UX will actually work better; this is especially true on iOS devices.

We would appreciate if the authors could provide better safeguards against 3rd party integrations in the proposal, such as Board of Directors action on SDLC members, or the discussion around 3rd party integrations needs to happen directly with this proposal so that the downsides can be properly assessed before the feature is merged and it is too late.

11. We believe that the HMI should always support showing a menu button, but it would only be shown if the app sends at least one AddCommand or AddSubMenu. If the app does not send any AddCommands or AddSubmenus, then the menu button should not be shown, but in that case the app must implement the CloseApplication RPC in an exit button somewhere in their in-app UI.

Could you clarify your point 11.2? You say that the templateTitle should always be visible, and we think that's right because it's optional for the OEM to implement. But then you say "the exact location of the button relies on the OEM", but it's not a button, it's a text field. Then you say that the same rules apply to it as the "MENU" button. We're not sure what that means, could you clarify? Are you talking about templateTitle or the "return to app list" button that we referenced earlier?

12. Sounds good, thanks!

13a.

Agree. In the new revision "recommended" will be changed to "required"

Great, thank you!

13b/14. We disagree with the expected behavior for cases 1.1 and 1.3. We are proposing two options:

1. If a webview template supports textfields, that means the textfields should be rendered on screen, but outside of the webview. DuplicateUpdatesFromWindowID=0 is allowed in this case.

2. Do not allow any webview template to support textfields, softbuttons, graphics (outside of the webview). DuplicateUpdatesFromWindowID=0 is not allowed for widgets.

Option 2 would reduce the amount of complexity for developers by making clear that those textfields and imagefields will never be available and therefore the developer doesn't need to create a lot of extra code to account for them. This option would also prevent fragmentation among OEM implementations of this feature.

[AByzhynar]

Could you clarify your point 11.2? You say that the templateTitle should always be visible, and we think that's right because it's optional for the OEM to implement. But then you say "the exact location of the button relies on the OEM", but it's not a button, it's a text field. Then you say that the same rules apply to it as the "MENU" button. We're not sure what that means, could you clarify? Are you talking about templateTitle or the "return to app list" button that we referenced earlier?

@theresalech I am sorry for the confusion. My mistake with copy-paste. You are right. It's about text field with Template Title. But the location of the template title can be different and the exact location of the template title depends on OEM. It can be above or below app's viewport etc. Just somewhere outside of app's viewport. Also when the app is deactivated there is no sense to show Template Title.

[theresalech]

A quorum was not present during the 2020-04-07 Steering Committee Meeting, so this proposal will remain in review until our meeting on 2020-04-14.

[AByzhynar]

@theresalech Thank you,

Please see our answers below:

11. We believe that the HMI should always support showing a menu button, but it would only be shown if the app sends at least one AddCommand or AddSubMenu. If the app does not send any AddCommands or AddSubmenus, then the menu button should not be shown, but in that case the app must implement the CloseApplication RPC in an exit button somewhere in their in-app UI.

We agree with this option. So we are in alignment. As this information is important we would like to add it to the next proposal revision. We are also in the alignment about the location of the menu button (somewhere outside of app's viewport. Location depends on OEM)

13b/14. We disagree with the expected behavior for cases 1.1 and 1.3. We are proposing two options:

1. If a webview template supports textfields, that means the textfields should be rendered on screen, but outside of the webview. DuplicateUpdatesFromWindowID=0 is allowed in this case.

2. Do not allow any webview template to support textfields, softbuttons, graphics (outside of the webview). DuplicateUpdatesFromWindowID=0 is not allowed for widgets.

Option 2 would reduce the amount of complexity for developers by making clear that those textfields and imagefields will never be available and therefore the developer doesn't need to create a lot of extra code to account for them. This option would also prevent fragmentation among OEM implementations of this feature.

We do prefer option 2. We will add this information to the next proposal revision.

We just would like to clarify the expected behavior in the following cases:

Pre-condition for case 1:

App has an active WEB_VIEW template. DuplicateUpdatesFromWindowID=0 is not allowed for widgets. That means that the creation of the widgets with that option is disallowed for the app.

Case 1:

The app changes its current active template from the WEB_VIEW to the different one with the support of DuplicateUpdatesFromWindowID=0 for the widgets.

The question for case 1:

1) So if the app would like to duplicate its main window content on the widgets - it has to create new widgets with such allowance, right?

Note: This functionality can not be changed during the widgets' lifecycle.

Pre-condition for case 2:

App has an active template with allowed DuplicateUpdatesFromWindowID=0 for the widgets. The app has created the widgets with DuplicateUpdatesFromWindowID=0

Case 2:

The app changes its current template to the WEB_VIEW with disallowed DuplicateUpdatesFromWindowID=0 for the widgets.

The question for case 2:

1) What should happen to the widgets? Note: This functionality can not be changed during the widgets' lifecycle.

Pre-condition for case 3:

App's current template WEB_VIEW, so no textfields, softbuttons, graphics are allowed.

Case 3:

App sends Show RPC -> SDL -> HMI: The HMI sends erroneous response with UNSUPPORTED_RESOURCE

The statement for case 3:

1) If the app wants to use textfields, softbuttons, graphics - it has to change its active template to another one that supports these fields. From the moment of change - the app can successfully send Show RPC with these fields and HMI will show them on its screen.

Do you agree with the statement above?

Pre-condition for case 4:

App's current template is any with allowed textfields, softbuttons, graphics. App sends Show RPC -> SDL -> HMI: HMI successfully shows them on the main window. As well as on the widgets (if any) that duplicate main window content.

Case 4:

The app decides to change its template to the WEB_VIEW one. So from the moment of change textfields, softbuttons, graphics are not allowed. But they are still present on the widgets that were duplicating the main window.

The question for case 4:

1) What is expected content on the widgets? Should it be cleared or should it be kept?

Possible option:

As a possible option which could solve the problems described above - Show, Alert and similar RPCs can be disallowed for MAIN window while app is using WEB_VIEW template. In this case if app wants to Show something or Alert user - it can send it directly to its UI bypassing SDL core OR/AND send them directly to the widgets via SDL core.

Waiting for your feedback.

Thank you.

[theresalech]

@AByzhynar @mrapitis, please see below.

6. Please advise if you can agree to one of our suggestions (provide better safeguards against 3rd party integrations in the proposal, such as Board of Directors action on SDLC members, or discuss 3rd party integrations directly with this proposal so that the downsides can be properly assessed before the feature is merged and it is too late.)

11. Sounds good, thank you!

13b/14.

The question for case 1: So if the app would like to duplicate its main window content on the widgets - it has to create new widgets with such allowance, right? Note: This functionality can not be changed during the widgets' lifecycle.

Yes, that's correct.

The question for case 2: What should happen to the widgets? Note: This functionality can not be changed during the widgets' lifecycle.

Nothing should happen to the widget. Currently the widget would stay on screen and the developer would be required to remove manually.

Case 3: App sends Show RPC -> SDL -> HMI: The HMI sends erroneous response with UNSUPPORTED_RESOURCE

We have a question about this case. If a mainfield text is sent in a show to a template that does not support textfields (TILES_ONLY), is it expected that the HMI responds with UNSUPPORTED_RESOURCE or SUCCESS? We hope to avoid creating divergent behaviors between webengine templates and the existing templates.

The question for case 4: What is expected content on the widgets? Should it be cleared or should it be kept?

Nothing should happen to the content. The developer can choose to change manually.

Possible option: As a possible option which could solve the problems described above - Show, Alert and similar RPCs can be disallowed for MAIN window while app is using WEB_VIEW template. In this case if app wants to Show something or Alert user - it can send it directly to its UI bypassing SDL core OR/AND send them directly to the widgets via SDL core.

We disagree with this option, as it would create a corner case for the WEB_VIEW template if the OEM wants to disable Show or Alert. They should use the existing windowCapability mechanism by disabling all textFields and imageFields for the RPC.

[theresalech]

During the 2020-04-14 Steering Committee meeting, there was discussion between the Ford team and the Project Maintainer regarding item 6 from the comments. The PM reiterated that while the proposal says the feature should be for OEM applications only, there is not anything in place to regulate or prevent a 3rd party integration. The Ford team cited the motivation between WebEngine Projection and Mobile Projection as the same. The Project Maintainer noted that while the motivation might be the same, Mobile Projection has safeguards in place to prevent 3rd party integrations, and so concrete language should be added to the WebEngine proposal to prevent a situation where SDLC Members who have not implemented this feature are at a disadvantage.

Ultimately, the Steering Committee voted to keep this proposal in review until our meeting on 2020-04-21, so that the Ford team can respond to Item 6 and the Luxoft team can respond to Item 13b/14. It was stressed that the goal will be to return this proposal for Steering Committee agreed upon revisions in the next meeting, which will include a resolution to the aforementioned open items.

[theresalech]

@stefanek22 @mrapitis, as you've mentioned that the motivation for this proposal is OEM Applications only, we'd recommend the following language be added to the proposal to provide a safeguard against 3rd party integrations. Please let us know if you can agree to this language, or if you have any suggested changes, thanks!

The acceptance of this proposal means the feature is available for OEM/1st party apps and content ONLY. There is an inherent risk that OEMs could use the feature to bring in 3rd party apps and content without additional proposals as it will be impossible to enforce. If an OEM takes such an action, the SDLC will be forced to bring the issue up to the Board of Directors for possible consequences.

[AByzhynar]

@theresalech Please see our answers below:

The question for case 2: What should happen to the widgets? Note: This functionality can not be changed during the widgets' lifecycle.

Nothing should happen to the widget. Currently, the widget would stay on screen and the developer would be required to remove manually.

We would like to get rid of the necessity to oblige the developer to do it manually. SDL core is aware of all widgets data. SDL core already creates and removes widgets without the mobile app (e.g. during the widgets resumption). So we propose SDL core to remove widgets from the HMI when the app changes its active template to WEB_VIEW one. This will allow to get better UX as well as possible resumption issues.

Case 3: App sends Show RPC -> SDL -> HMI: The HMI sends erroneous response with UNSUPPORTED_RESOURCE

We have a question about this case. If a mainfield text is sent in a show to a template that does not support textfields (TILES_ONLY), is it expected that the HMI responds with UNSUPPORTED_RESOURCE or SUCCESS? We hope to avoid creating divergent behaviors between webengine templates and the existing templates.

You are right. We have checked the current behavior. If a mainfield text is sent in a show to a template that does not support textfields (TILES_ONLY), it is expected that the HMI responds with SUCCESS with no changes on the HMI screen. Is it required to specify this behavior explicitly in the proposal ?

The question for case 4: What is expected content on the widgets? Should it be cleared or should it be kept?

Nothing should happen to the content. The developer can choose to change manually.

We would like to get rid of the necessity to oblige the developer to do it manually. SDL core is aware of all widgets data. SDL core already creates and removes widgets without the mobile app (e.g. during the widgets resumption). So we propose SDL core to remove widgets from the HMI when the app changes its active template to WEB_VIEW one. This will allow to get better UX as well as possible resumption issues.

Possible option: As a possible option which could solve the problems described above - Show, Alert and similar RPCs can be disallowed for MAIN window while app is using WEB_VIEW template. In this case if app wants to Show something or Alert user - it can send it directly to its UI bypassing SDL core OR/AND send them directly to the widgets via SDL core.

We disagree with this option, as it would create a corner case for the WEB_VIEW template if the OEM wants to disable Show or Alert. They should use the existing windowCapability mechanism by disabling all textFields and imageFields for the RPC.

The windowCapability can be used to restrict Show, Alert and other similar RPCs via textFields. What do you think about a unified approach? E.g. If app sends RPCs which triggers POP-UP on the HMI screen, so overlays what WEB_VIEW shows (Alert, PerformInteraction etc.) - so in that case touch events will first go through the POP-UP area

[Jack-Byrne]

13b/14.

The question for case 2: What should happen to the widgets? Note: This functionality can not be changed during the widgets' lifecycle.

We would like to get rid of the necessity to oblige the developer to do it manually.

We do not agree. If Core removes the widget automatically, there is no method to notify the developer of the action. We propose that when a Show request that changes the layout to WEB_VIEW, and if there were previously created duplicate main window widgets, Core should respond with a warning message that the widgets are recommended to be removed. If the developer does not remove them, the duplicate widgets should continue to operate normally.

You are right. We have checked the current behavior. If a mainfield text is sent in a show to a template that does not support textfields (TILES_ONLY), it is expected that the HMI responds with SUCCESS with no changes on the HMI screen. Is it required to specify this behavior explicitly in the proposal ?

In reference to the above response, case 3 should be updated to

Case 3: App sends Show RPC -> SDL -> HMI: The HMI sends success response If the app wants to use textfields, softbuttons, graphics - it has to change its active template to another one that supports these fields. Show requests that include these fields for the WEB_VIEW template should still return a success response even though the fields won't be shown.

We believe this aligns with the previously described behavior for sending show strings to a TILES_ONLY template.

The question for case 4: What is expected content on the widgets? Should it be cleared or should it be kept? Possible Option

The outcome for case 4 should be the same as case 2. The widgets should stay with the previously populated content before the template was changed. And as described by the TILES_ONLY behavior, the HMI should not return unsupported resource in case textfields are sent to a template that does not support them.

[theresalech]

In preparation for tonight's Steering Committee meeting, we'd like to summarize the PM's requested revisions, many of which the author has already agreed to, for Steering Committee vote:

• (Item 6) Author to add the following language to the proposal:

  The acceptance of this proposal means the feature is available for OEM/1st party apps and content ONLY. There is an inherent risk that OEMs could use the feature to bring in 3rd party apps and content without additional proposals as it will be impossible to enforce. If an OEM takes such an action, the SDLC will be forced to bring the issue up to the Board of Directors for possible consequences.

• (Item 11) Author to add high level wireframe of what components this new WebEngine projection template will include: the HMI should always support showing a menu button, but it would only be shown if the app sends at least one AddCommand or AddSubMenu. If the app does not send any AddCommands or AddSubmenus, then the menu button should not be shown, but in that case the app must implement the CloseApplication RPC in an exit button somewhere in their in-app UI. Additionally, the template should include a button to access to the app list, the add command menu button (or a requirement for the app to implement ShowAppMenu or otherwise implement CloseApplication), and app/template title.
• (Item 12) Author to update proposal specifying this is for JavaScript only.
• (Item 13) Author to change "recommended" to "required" in the following:

  It is recommended that WindowCapability.textFields should not contain these text field names if the OEM has implemented the WEB_VIEW template without these text fields.

• (Item 14) Author to specify that we will not allow any webview template to support textfields, softbuttons, graphics (outside of the webview). DuplicateUpdatesFromWindowID=0 is not allowed for widgets.

If we are also able to agree on the suggestions from this comment, we'd like to vote to return this proposal for the revisions in that comment, as well as those listed above.

[theresalech]

During the 2020-04-21 Steering Committee meeting, it was acknowledged that all items in this comment were agreed upon, with the exception of the suggestions from this comment. The Steering Committee then voted to keep this proposal in review, to allow the items in this comment to be finalized.

[theresalech]

Hi @AByzhynar, were you able to review this comment from Livio: https://github.com/smartdevicelink/sdl_evolution/issues/912#issuecomment-617349168? Please let us know your thoughts! We would like to be able to resolve prior to our meeting tomorrow. Thanks!

[AByzhynar]

@theresalech Thank you.

Please check the options below:

The question for case 2: What should happen to the widgets? Note: This functionality can not be changed during the widgets' lifecycle.

We would like to get rid of the necessity to oblige the developer to do it manually.

We do not agree. If Core removes the widget automatically, there is no method to notify the developer of the action. We propose that when a Show request that changes the layout to WEB_VIEW, and if there were previously created duplicate main window widgets, Core should respond with a warning message that the widgets are recommended to be removed. If the developer does not remove them, the duplicate widgets should continue to operate normally.

What does it mean: “the duplicate widgets should continue to operate normally.” ? Should the application still be able to send Show to the main window with some fields and the duplicate widgets will change their contents as well?

Possible options: 1) “YES” –in this case, the widgets will not duplicate the MAIN window as it is rendered by WebEngine while the WEB_VIEW template is active. So the idea of MAIN window duplication is broken.

2) “NO” - in this case, widgets will be kept in the frozen state from the previously removed template.

We would like to combine the option you proposed with the previous option:

As there is no method to notify the developer of the action if Core removes the widget automatically - we propose that when the app sends a Show request that changes the layout to WEB_VIEW, and if there were previously created duplicate main window widgets, Core should respond with a warning message that the widgets will be removed, and proceed with the widgets removal. This approach will give us the following benefits.

• This will allow informing the developer of the actions
• This will allow to avoid the necessity to oblige the developer to do it manually.

IMPORTANT NOTE: The content received in Show RPC is kept by the HMI independently of the current active template. So in case, TILES_ONLY template is active and the app sends Show with some text fields, Current HMI behavior :

• HMI responds to the SDL with SUCCESS
• HMI saves the received content internally
• HMI will not change its screen (so the received fields won’t be shown)
• HMI will show previously saved content if the app will change the active template to the one with the support of those fields.

• (Item 11) Author to add high level wireframe of what components this new WebEngine projection template will include: the HMI should always support showing a menu button, but it would only be shown if the app sends at least one AddCommand or AddSubMenu. If the app does not send any AddCommands or AddSubmenus, then the menu button should not be shown, but in that case the app must implement the CloseApplication RPC in an exit button somewhere in their in-app UI. Additionally, the template should include a button to access to the app list, the add command menu button (or a requirement for the app to implement ShowAppMenu or otherwise implement CloseApplication), and app/template title.

We just would like to clarify the following: "the template should include a button to access to the app list". We consider that “App List” button should be independent of the used template. So it should be always available outside of the template.

About RPCs which trigger HMI POP-UPs:

As WebEngine Projection will use a different mechanism for rendering its UI - we would like to state explicitly the following : If the app sends RPCs which triggers POP-UP on the HMI screen, so this POP-UP overlays what WEB_VIEW area shows (Alert, PerformInteraction etc.) - in that case, touch events will first go through the POP-UP area. After POP-UP is closed - touch events processing returned back to the WEB_VIEW area.

• (Item 14) Author to specify that we will not allow any webview template to support textfields, softbuttons, graphics (outside of the webview). DuplicateUpdatesFromWindowID=0 is not allowed for widgets.

The part outside of the webview - is a bit confusing. If these fields are not allowed outside of the webview - Does it mean that WEB_VIEW template supports all these fields inside its web_view? Can you please clarify?

[theresalech]

Hi @AByzhynar, thanks for your comment. Please find our responses below. If you are able to review and respond today, we'd appreciate it! If you can agree to the terms below, this will help to put the proposal in a position to vote to return for revisions during today's call.

11.

We just would like to clarify the following: "the template should include a button to access to the app list". We consider that “App List” button should be independent of the used template. So it should be always available outside of the template.

Yes, we agree this can be updated to "the HMI should include a button to access the app list."

If the app sends RPCs which triggers POP-UP on the HMI screen, so this POP-UP overlays what WEB_VIEW area shows (Alert, PerformInteraction etc.) - in that case, touch events will first go through the POP-UP area. After POP-UP is closed - touch events processing returned back to the WEB_VIEW area.

Yes, agreed.

13b/14.

What does it mean: “the duplicate widgets should continue to operate normally.” ? Should the application still be able to send Show to the main window with some fields and the duplicate widgets will change their contents as well? In response to all of the cases that have been provided for this item, for the scope of this proposal, we would like the behavior of the WEB_VIEW template with duplicate widgets to align with the behavior of existing templates (ie switching from NON_MEDIA to a TILES_ONLY template with a duplicate widget).

In order to not complicate this proposal further, we suggest that any new behaviors affecting widgets should be in their own proposal. We think the solution you provided should apply to all templates, and not act as an adhoc solution for web engine only.

14.

Does it mean that WEB_VIEW template supports all these fields inside its web_view?

Apologies for the confusion. These shouldn't be allowed inside or outside of the WEB_VIEW template. This can be updated to say "Author to specify that we will not allow any webview template to support textfields, softbuttons, graphics (outside or inside of the webview)."

[AByzhynar]

@theresalech Thank you for your responses and proposed options. Please see our answer below:

11.

We just would like to clarify the following: "the template should include a button to access to the app list". We consider that “App List” button should be independent of the used template. So it should be always available outside of the template.

Yes, we agree this can be updated to "the HMI should include a button to access the app list."

We are in alignment here.

If the app sends RPCs which triggers POP-UP on the HMI screen, so this POP-UP overlays what WEB_VIEW area shows (Alert, PerformInteraction etc.) - in that case, touch events will first go through the POP-UP area. After POP-UP is closed - touch events processing returned back to the WEB_VIEW area.

Yes, agreed.

We are in alignment here as well.

13b/14.

What does it mean: “the duplicate widgets should continue to operate normally.” ? Should the application still be able to send Show to the main window with some fields and the duplicate widgets will change their contents as well? In response to all of the cases that have been provided for this item, for the scope of this proposal, we would like the behavior of the WEB_VIEW template with duplicate widgets to align with the behavior of existing templates (ie switching from NON_MEDIA to a TILES_ONLY template with a duplicate widget).

In order to not complicate this proposal further, we suggest that any new behaviors affecting widgets should be in their own proposal. We think the solution you provided should apply to all templates, and not act as an adhoc solution for web engine only.

Agree.

14.

Does it mean that WEB_VIEW template supports all these fields inside its web_view?

Apologies for the confusion. These shouldn't be allowed inside or outside of the WEB_VIEW template. This can be updated to say "Author to specify that we will not allow any webview template to support textfields, softbuttons, graphics (outside or inside of the webview)."

Agree. So we have an agreement in all the options above. Thank you

@theresalech Can I add the aligned changes above to the existing PR with the revision?

[theresalech]

As the PM and author have come to agreement on all revisions, I've summarized below in preparation for tonight's Steering Committee meeting. In tonight's meeting, we will ask that the Steering Committee votes to return this proposal for the following revisions:

• (Item 6) Author to add the following language to the proposal:

  The acceptance of this proposal means the feature is available for OEM/1st party apps and content ONLY. There is an inherent risk that OEMs could use the feature to bring in 3rd party apps and content without additional proposals as it will be impossible to enforce. If an OEM takes such an action, the SDLC will be forced to bring the issue up to the Board of Directors for possible consequences.

• (Item 11) Author to add high level wireframe of what components this new WebEngine projection template will include: the HMI should always support showing a menu button, but it would only be shown if the app sends at least one AddCommand or AddSubMenu. If the app does not send any AddCommands or AddSubmenus, then the menu button should not be shown, but in that case the app must implement the CloseApplication RPC in an exit button somewhere in their in-app UI. Additionally, the HMI should include a button to access the app list, the add command menu button (or a requirement for the app to implement ShowAppMenu or otherwise implement CloseApplication), and app/template title.
• (Item 12) Author to update proposal specifying this is for JavaScript only.
• (Item 13) Author to change "recommended" to "required" in the following:

  It is recommended that WindowCapability.textFields should not contain these text field names if the OEM has implemented the WEB_VIEW template without these text fields.

• (Item 13b) Author to update the proposal to reflect that the behavior of the WEB_VIEW template with duplicate widgets should align with the behavior of existing templates (ie switching from NON_MEDIA to a TILES_ONLY template with a duplicate widget).
• (Item 14) Author to specify that we will not allow any webview template to support textfields, softbuttons, graphics (inside or outside of the webview). DuplicateUpdatesFromWindowID=0 is not allowed for widgets.

[theresalech]

@AByzhynar regarding your question, you can update the revisions PR with these changes, but I will not be able to review that PR until the Steering Committee has agreed to return this proposal for said revisions. Thanks!

[AByzhynar]

@theresalech

• (Item 6) Author to add the following language to the proposal:

The acceptance of this proposal means the feature is available for OEM/1st party apps and content ONLY. There is an inherent risk that OEMs could use the feature to bring in 3rd party apps and content without additional proposals as it will be impossible to enforce. If an OEM takes such an action, the SDLC will be forced to bring the issue up to the Board of Directors for possible consequences.

We agree. We will add this information to the next proposal revision.

[theresalech]

The Steering Committee voted to return this proposal for the revisions outlined in this comment.