Closed theresalech closed 4 years ago
As the first commenter, I want to note this first: If this proposal passes, it is step 1 of the biggest change to SDL since its creation. If you have a stake in SDL, review this proposal. It changes the fundamentals of what SDL is.
I see that under 2.2.3, this proposal indicates that this should only be used for first-party apps, but as I say under general note 2, this is fairly weak language. I will therefore address several comments on the basis of the fundamental change that this makes to SDL UI.
I must say that I'm surprised by this proposal, given that it fundamentally alters against the core principles of SDL: that an app only has to do one integration and its design is (1) guaranteed to pass OEM review (every OEM will now have to do design review on every Webengine SDL app), (2) will fit with the rest system (every app will now have a distinct design that won't fit with every OEM system, or any OEM system), (3) is driver distraction safe (apps can now design whatever they want and it's not going to go through the same stringent process that the OEM's own HMI goes through).
Furthermore, this proposal seems to be the opposite of every discussion we have had regarding template customization. Passing this proposal should re-open every previous discussion we've had about template customization that has been opposed by OEMs in the past. If totally open customization is on the table, why wouldn't we first go in the direction of increased template customization? Why not have an XML file that mobile apps or web apps can transfer that declares the layout of the template? Then at least the template would retain some semblance of the OEMs UI.
1. Will SDLServer have to change to accommodate these OEM store changes?
2. Under 2.2.3, you write:
With the current depth of this chapter, the new HMI type should be used by 1st party OEM apps only. With future proposals and workshops the SDLC should open the new HMI type to 3rd party by creating and defining proper driver distraction and user interface guidelines.
I don't think this language is strong enough. This is a large amount of work for 1st party only apps. It may still be worth it to the SDLC to only build this for 1st party apps, but I personally think we should at least have some idea of what we're going to do about 3rd party apps. And I'm somewhat skeptical that there's a generic, one-size-fits-all solution to this problem that the SDLC will agree to. It took a year to decide on an english only string for backgrounding a navigation app. When we talked about template customization, after hours of discussion the most we could agree on were three optional colors that the OEM could use or not use however they wished.
Furthermore, this proposal doesn't even really mandate that it can't be used for third party apps. It only says that it should only be used for first-party apps. That leaves the door open for OEMs to go and do exclusive deals with app partners to do apps that only work on their platform, which undermines the very spirit of SDL. If this proposal were to go forward, I would strongly recommend to change should to must. Even then, how would we really enforce it?
3. I don't see the security concerns of running unknown JavaScript code on the head unit addressed. With SDL we have some amount of security due to the protocol based nature of data flow. Now we would have arbitrary code running on the head unit. What sandboxing / security features must be used to allow for this kind of arbitrary code? What about the security risks inherent in having a web browser that is bound to have security flaws running in a head unit for a decade or more? Especially with unknown SDL JavaScript code running on it?
1. There doesn't seem to be a mechanism to update an app bundle. I'm assuming that the OEM app store would need to know that its installed bundle is out of date and be able to update it. Is that up to the OEM app store backend? But I would assume this would need to be a policies issue to ensure they have the most up to date app bundle that supports the given head unit. (e.g. if v1.0 supports SDL 3.0-5.0 and v2.0 supports SDL 5.1+).
2. Ideally, a change like this would also include a way for a mobile app to transfer / update a PWA app and act as an internet relay for it. That way vehicles without built in modems (or users without modem subscriptions) can access this feature and the apps that use it.
3. Under 2.1 is written:
The default app presentation approach should be template based. With the web app becoming active on the HMI, the HMI should not present the app's HTML document object. The body of the index.html file of the web app should be empty and should not be modified with scripts as the app won't be visible on the HMI.
I'm confused by what is meant by the body if the index.html
file should be empty? Is the index file supposed to modify itself based on whether or not the template approach will be used (which it doesn't seem like they can know ahead of time)? Wouldn't it be better for the manifest to point to a different file that the HMI can start in this case? What if an app wants to only support OpenHMI?
4. Under 2.2, you write:
The window capabilities of this open main window will be empty except for the window ID and physical button capabilities.
Shouldn't at least the displayName
be made available as well (though it seems that the proposal including WindowCapabilities
is missing displayName
)? A link to the WindowCapabilities
definition would be helpful because it's not yet implemented as well.
5. Under 2.2.2, you write:
Using the new HMI type would be policy controlled. On the policy server this new HMI type can be added to the valid HMI type list per app ID. Only apps with permissions to use this new HMI type would be allowed to register.
Doesn't this conflict with 2.1? Wouldn't the app simply be given a template instead of a WebEngine page? I'm unsure how 2.1 is intended to work here. Is the app supposed to re-register without the OpenHMI
type?
6. How is the app supposed to reach out to the internet for data? Are they intended to use a specialized API? The standard JavaScript call? Will Core / HMI route them through the modem? Can they use a connected mobile device as a relay?
7. In 3.1, you write:
The app package should be a compressed zip file
zip is not a great compression format. If OTA size is the main concern, why not consider something like tar + gzip? Though zip is very widely available, and would probably work on all systems in some manner.
8. In 3.5 you write:
The entry point HTML file should refer to the manifest file ()
Where? In the head
? Providing a sample HTML
file may be helpful.
Action Items from Meeting on 2019-06-24. Additional minutes available upon request.
The Steering Committee voted to return this proposal for revisions, so that the author can update per the action items identified in the 2019-06-24 workshop, as listed in this comment.
The author has updated the proposal, and the revised proposal is now in review until 2019-07-23.
I have attached a PDF containing a lengthy response to "Chapter 2", aka OpenHMI, of this proposal. I understand that it is lengthy, but I encourage everyone, both OEMs and App Developers, to take the time to read it. I have spent a lot of time writing this because I truly believe in SDL and its future and I believe that OpenHMI is detrimental to its future.
As an aside, I think that OpenHMI can (and should) be removed from this proposal and proposed separately. It's a huge change that is separate from "Chapter 1" and "Chapter 3". Those two chapters go together, but OpenHMI is really quite separate from them. It can (and should) be discussed separately from the other items.
The Steering Committee voted to defer this proposal, keeping it in review until our next meeting on 2019-07-30, to allow the author and all SDLC Members to review the OpenHMI Response from the project maintainer and provide input. It was reiterated that all SDLC Members are encouraged to review this proposal and project maintainer's feedback.
During the 2019-07-30 Steering Committee meeting, an SDLC Member asked for the general intention of this proposal. The author answered that it provides a new way of how applications are presented on the HMI, especially in relation to in-vehicle web applications, using JavaScript. The Steering Committee then voted to defer this proposal, keeping it in review until our next meeting on 2019-08-06, to allow for more review and discussion of the proposal. It was again reiterated that the project maintainer believes this proposal to be introducing a very impactful change, which should be carefully considered by all members.
The Steering Committee voted to defer this proposal, keeping it in review until our next meeting on 2019-08-20, to allow the author and members time to respond to the feedback from the Project Maintainer. It was agreed that if there was no activity on the review issue by the next meeting, the proposal would be deferred as inactive.
With regards to the OpenHMI response the paper/response has valid complaints but also many personal opinions and also invalid points, information that is not related to the OpenHMI. The paper focuses a lot at 3rd party support which is not at all part of the proposal, therefore makes a lot of hypothesis. Overall the paper is an overflow of data and it is very difficult to figure out what parts are in focus and important for the proposed features.
This overflow is causing confusion to the audience and is dangerous because people can't (or don't want to) confirm all the hypothesis and theories. Many of the concerns are related to things that are not part of the proposal. The proposal includes the OpenHMI because it tries to avoid many of the concerns to ever happen. A common runtime environment, application package and projection mode for in-vehicle applications is what this proposal is looking for. The PM complaints mentioned in the paper to enable 3rd party support are mostly (or even completely) business related.
Future proposals can be published and negotiations with SDLC members to enable 3rd party support with OpenHMI can be made while SDLC members add this feature to vehicles. This increases the number of vehicles supporting this feature making the in-vehicle SDL apps a really attractive platform to develop once the feature is enabled for 3rd party. This can be key to attract app developers that have not accepted (or left) SDL due to the limited possibilities with templates and mobile apps.
Besides that, I want to let the project maintainers know about the possibilities of an app and library driven user interface. Custom templates or common templates and themes would all be possible with an Open HMI. While there is a business case for 1st party apps, we can use this feature to improve the user interface concept that is useful for 3rd party.
Removing it from the proposal is a significant reduce of functionalities compared to a mobile application which can project a user interface to the vehicle (see mobile navigation apps or HMI type PROJECTION
). This will cause the authors organization to rethink the strategy e.g. by defining the OpenHMI as PROJECTION
apps. This approach will for sure damage SDL and the SDLC. While the PM believes that removing OpenHMI from the proposal avoids SDLC getting damaged the opposite will happen.
For the sake of making progress with the in-vehicle applications and the web engine I moved the chapter into a new proposal which is ready in #809. A revision to this proposal is also ready in #810. Applying this revision is accepted by the author only together with moving #809 into review.
The Steering Committee voted to return this proposal for revisions, to remove the OpenHMI components and only focus on in-vehicle applications and the web engine. OpenHMI components will be discussed separately in a subsequent proposal, once in-vehicle applications and web engine are agreed upon. As the author noted revisions (removing OpenHMI) were ready (https://github.com/smartdevicelink/sdl_evolution/pull/810), the Steering Committee then voted to bring the revised proposal into review.
The revised proposal will now be in review until 2019-08-27.
It was also communicated during the Steering Committee Meeting that returning a proposal for revisions and bringing the revisions into review in the same week is not something that should happen moving forward, unless the proposal is critically time sensitive. The Project Maintainer is working to clarify the SDL Evolution Process in the repository’s process.md, and will share with the Steering Committee for review soon. Once the process is more defined, we can ensure it’s adhered to in all scenarios moving forward.
The Steering Committee voted to defer this proposal, keeping it in review until our next meeting on 2019-09-03, to allow for additional time to review.
1.
While SDL Core is operating, the server should be permanently available and listen for connections on the specified port.
*if Core is built with DWEBSOCKET_SERVER_TRANSPORT_SUPPORT
set to `true,
Should there be any other mechanism that would determine if Core should listen on this port? For example, if the policy table includes at least one entry for a local web engine app the websocket server transport should be activated? I would prefer if the websocket wasn't opened unless there was an app that was intended to use it, but this is not a blocker for me. I would imagine it would also be possible to know how many connections core should allow based on PT entries, but would likely cause more issues than it is worth.
2a.
Due to a new app platform, the hybrid app preference should be modified to track mobile, cloud and local apps.
Just as a clarification, the original decision was to treat local and cloud apps as the same type that would give preference to local apps over cloud.
2b.
App registration on the SDL Developer Portal (smartdevicelink.com) should allow a developer to specify an app as a local web app.
The developer portal currently allows for Android, iOS, Cloud, and Embedded as the registered app types. If a change is requested to that set I want to make sure it is very clearly stated and this statement alone isn't taken as the spec of what should be done.
3.
1.4 HMI API using App Properties RPCs
Should the PT entry be extended to include more information about the app and its transport in the case of needed a Websocket server vs Core being the ws client?
4. I think one aspect that could enhance this proposal would be the ability load the app as a javascript module from the HMI if the HMI were also using a webengine. Maybe it makes sense to create a new line in the manifest to include the javascript file only.
{
"entrypoint_html": "index.html",
"entrypoint_js": "js/hello_sdl.js",
...
}
This could be a separate proposal, but changing the entrypoint
key should be down now to have it qualified with the specific type of entry point if this type of feature has support.
5.
vii. Min SDL version supported
Is this intended to be RPC spec version? Currently we support versions for both the protocol layer and the RPC messaging layer and they are different. So I would imagine we should specific this version to RPC spec version, and possibly add a new key/value pair for protocol.
6. The manifest has a lot of useful information included, but the previously implemented Cloud Transport Proposal used the policy table for some of this information to display to the HMI. (eg app icon). Creating this new packaging for webengine apps would create a division between how other information from other apps (Java SE, Java EE, Node.js) is gathered currently. Maybe it's possible to create a similar structure for the other non-mobile frameworks as well?
7.
Both are difficult to sandbox. Compared to a web engine the effort to sandbox a Node.js or java application and to protect the vehicle system are very high.
I'm not sure I agree with this point when using the Java Virtual Machine (JVM) has a few different options to increase security to a sandboxed environment. This includes simply running the JVM as different user or implementing the Java Security Manager. I would also venture that if apps have created Java implementations they would be familiar with these. However, the volume of apps that would use this framework might be less than node.js/javascript so the point is likely moot.
@kshala-ford Hey Kujtim, some feedback/questions below. Keeping with the numbering that @joeljfischer started in his "technical notes" - some of which should probably still be addressed before bringing this proposal to a vote.
1. Does the SDL open-source project have any responsibility regarding app updates, or is the burden of handling app installation and updates intended to be placed entirely on the shoulders of OEMs? For example, if an app developer creates an app targeting sdlMinVersion: 5.0
but then later updates to sdlMinVersion: 6.0
, whose responsibility is it ensure that only qualifying installations of Core receive the app update? Currently, SDL Server does not alter responses based on the version of SDL Core which triggered the Policy Table Update request.
9. How is the OEM supposed to receive the compressed app package from a developer? Are developers expected to upload their compressed app package on smartdevicelink.com, which would then be attached to SHAID app records in the form of a file URL and propagated to SDL Servers? The end-to-end process from the perspective of an app developer is a bit vague to me.
10. The mobile app properties HybridAppPreference
change from BOTH
to ALL
provides more clarity around behavior, but it creates a potential backward-compatibility issue in SDL Server since ALL
would not be recognized by older versions of Core. How would older versions of Core react to receiving an unrecognized value of ALL
? If this would produce undesired results, how is an SDL Server supposed to know which ENUM value to send back in app policies when the Policy Table Update request does not indicate which version of Core it originated from? Perhaps @JackLivio or @jacobkeeler can provide insight.
11. The mobile app properties HybridAppPreference
addition of LOCAL
does not match the current platform option available to developers on smartdevicelink.com today. I suggest changing this new enum value to EMBEDDED
to be as consistent as possible. This would also help avoid any misconceptions that a "LOCAL" app cannot retrieve data from external web APIs.
12. The proposal states that these apps would not be allowed to be modified without going through another round of SDLC review for security reasons, however, there is an inherent risk of an app performing maliciously post-certification by using an external web API with well-obfuscated requests and responses. While this isn't something that can be easily addressed, but I feel it should be recognized as a weakness and carefully considered when reviewing these kinds of apps.
13. When an app isn't recognized, the proposal states that a Policy Table update will occur and the app's name will be verified with the list of approved app nicknames. Is a Policy Server required for this kind of app to work, or can these apps be used through the use of Core's default Policy Table?
14. Can you please describe your reasoning for putting so much metadata in the manifest.json
file which could be retrieved from a Policy Table's app_policies
instead?
15. In the example manifest.json
provided in the proposal, the example locales
do not follow the current Policy Table locale format of [language]-[country]
. For example, instead of de_DE
, we should use de-de
for consistency (note all-lowercased and uses a hyphen).
The Steering Committee voted to defer this proposal, keeping it in review until our next meeting on 2019-09-10, to allow for additional conversation on the review issue between the author and commenters.
@nickschwab, @joeljfischer, @joeygrover I will follow Joey's enumeration from 1 to 7 and reply to Nick's comments from 9 to 15. Technical notes from Joel will be appended with new numbers due to the revision that made a few notes obsolete.
To @joeygrover comments
|1
*if Core is built with DWEBSOCKET_SERVER_TRANSPORT_SUPPORT set to `true,
Of course, I think this can be implied as listen to a port is impossible without the code included.
Should there be any other mechanism that would determine if Core should listen on this port? For example, if the policy table includes at least one entry for a local web engine app
I like the idea but I have the same concerns around increase complexity. The query for the policy database is no rocket science but not sure if it delays Core initialization (might affect app resumption). I think the proposed ini settings to also specify an adapter address (e.g. localloop) it gives a fair amount of security to production vehicles. The benefit of this ini is that we could configure and listen to local network for test benches hence help developers to test their app from their computer.
|2a I'm not sure if I am understanding. I know cloud apps are independent of their location and could run on the local machine (though specified as cloud app). The new app preference should be independent of and should not interfere with the cloud app feature.
|2b I was reading SDL-0203 and the comments made during review but couldn't find any useful information. smartdevicelink.com doesn't say much about embedded apps. What does "Embedded" stand for in the developer portal? Isn't it what I mentioned in 2a? If you want us to spec out things properly we need the PM to do the same.
|3 Please have a look at bullet point 6
"endpoint" parameter will be omitted. Instead the HMI is responsible to launch local apps.
|4 Not sure if I can really follow what you're suggesting. I assume it is about loading an app into a web based HMI by pointing to a js file. I'm not a fan of this approach as it would require a monolithic, single process, single DOM, single domain where nothing is protecting the HMI from apps. If this feature is really something we would like to include (in future?) it needs to be fleshed out further. I believe "entrypoint" doesn't need to be changed to prepare this feature as a new key like "app_type" = [default|extension]
where default
is the app type we are proposing here and extension
(or any other name) is regarding your suggested feature. Still this new parameter could be introduce in a future proposal.
|5 Yes this is the RPC spec version. It should be optional and allow developers to specify a RPC spec version required for the app to be operational. Same can be included for min Protocol version.
|6 Can you specify what information you are referring to? The only field I can see is the app icon. While I understand why an icon_url make sense for a cloud app I see benefits for a local app to embed the icon into the application package. The app installation routine of the IVI system will have all app information available when installing the app. For the developers and for the OEMs it's beneficial in the way that it is clear to how the app appears on the system. It's not possible to change the app appearance or reachability (voice commands etc.) from a different source. Last the driver/user could be confused if the app all of a sudden appears differently (different app icon) or if the app cannot be launched with the previously known voice command, all without an actual app update.
|7 One of the main features of JVM is to provide platform independency and of course this approach naturally includes security mechanisms. There are methods of Java to increase the security (as you mentioned the security manager) and practices to restrict access using the OS user roles. Using a sandbox with a web engine doesn't only protect the system from the app but also the app from the system (or other apps). Security and privacy are main goals of the sandbox. By default features are permission controlled (see chrome://settings/content) providing a very good protection service out of the box. As mentioned, there are many web apps available and we expect a high chance to reuse existing app code for in-vehicle apps.
To @nickschwab comment
|8 (belongs to @nickschwab notes to 1.
)
Does the SDL open-source project have any responsibility regarding app updates ... whose responsibility is it ensure that only qualifying installations of Core receive the app update?
The burden is on the shoulders of the OEMs. An OEM will have many different versions of SDL Core on the road and these vehicles would request an app list from an OEM app store backend returning a list specific for the vehicle. The OEM backend would be based on data from the SDL server.
|9 This is one possible option to submit the compressed package to smartdevicelink.com. Imagining an app certification for such apps I believe it would be necessary. However, we're planning a developer to submit to our (OEM) developer platform (which can be referenced from smartdevicelink.com if such an API exist).
|10 I think this is a general question about SDL server and versioning. The first thing I have in mind to solve the problem for this proposal is to:
BOTH
and don't add ALL
(Cloud)AppProperties
named hybridAppPreferences
which is an arrayALL
is realized by the array including all enum elements.BOTH
would match the new parameter set to [
MOBILE,
CLOUD,
LOCAL]
(or embedded... terminology yet to be decided)|11 This is funny because in automotive using the word embedded is understood as "it is shipped with the vehicle" or "it's purely offline" or "it's fixed with the surrounding (software)". Local is describing a location instead. I would say "A local web application is running in an embedded WebEngine".
To the point that "EMBEDDED" already exist... I want to refer to 2b from Joey and defer the discussion to the next SC meeting. I believe there's need to discuss how EMBEDDED
got introduced by the PM.
|12 You are right. The possibility of backend logic or data changes post-certification is a challenge. It's necessary to verify that the requests an app can send are known during certification and that they are all are properly tested. The app behavior might change due to backend sending different data hence triggering the app to behave differently. I want to note that WebEngines prevent apps to post-load executable code from a backend.
|13 The plan is when an app is installed into the vehicle, the HMI sends SetAppProperties
to SDL Core with the app ID. SDL 0248 was designed for this case to allow SDL requesting a policy table update if the provided app ID is unknown.
|14 Can you please be more specific to "so much"?
|15 I want to make everyone aware of the RFC 5646 which we should follow everywhere where internationalization and localization is required. to your point: Hyphen is okay but I don't think lowercase region is a good idea. It should be uppercase.
To @joeljfischer comment (excluding OpenHMI questions)
|16 1. Will SDLServer have to change to accommodate these OEM store changes?
If this question is still valid but unanswered can you please be more specific to "these OEM Store changes"?
|17 3. I don't see the security concerns of running unknown JavaScript code on the head unit addressed...
Just to recap. With developers submitting a compressed app package to SDL/OEMs, all the code can and will be reviewed before it is made available to apps for installation. As I noted in |12 WebEngines prevent post-loading executable code therefore the app code is known with the app being installed. The question regarding the web browser/engine vulnerability (but also compatibility): It should be the OEMs goal (as it is our goal for this feature) is to keep the engine up-to-date throughout the vehicle lifespan. This means that regular software updates should be made available to the vehicle. At least the engine should really be at the latest version. I want to add that this is a general issue for any kind of runtime environment (node, JVM etc.). It's the host-owner responsibility to keep the host up-to-date For cloud apps, it's the developer's server. For local apps it's the vehicle/IVI.
|18
- There doesn't seem to be a mechanism to update an app bundle. I'm assuming that the OEM app store would need to know that its installed bundle is out of date and be able to update it. Is that up to the OEM app store backend?
Yes!
But I would assume this would need to be a policies issue to ensure they have the most up to date app bundle that supports the given head unit. (e.g. if v1.0 supports SDL 3.0-5.0 and v2.0 supports SDL 5.1+).
this is the purpose of minSDLVersion. The OEM store should know all apps and also different versions of the app. Depending on the requesting vehicle and the current version of the vehicle it may return only supported apps.
|19
- Ideally, a change like this would also include a way for a mobile app to transfer / update a PWA app and act as an internet relay for it. That way vehicles without built in modems (or users without modem subscriptions) can access this feature and the apps that use it.
We have done some analysis to how far phones can be used as a relay to the internet. Phone features like Hotspot or WiFi direct or using an app as a relay... The complexity and the risk of phone OS changes breaking the suggested feature caused us to not continue this route further.
|20
- In 3.1, you write:
The app package should be a compressed zip file
zip is not a great compression format. If OTA size is the main concern, why not consider something like tar + gzip? Though zip is very widely available, and would probably work on all systems in some manner.
I think any compression or packaging would work. With the OEM app store we will anyway need to unpackage and repackage the app compatible with the OEM specific OTA app installation procedure. This is planned to be completely transparent to the app and the partner and should not affect the behavior of the app.
|21
- In 3.5 you write:
The entry point HTML file should refer to the manifest file (<script src="manifest.json" />)
Where? In the head? Providing a sample HTML file may be helpful.
Any place is fine. Adding them in the <head>
should be no issue (except performance wise). I think the common practice these days is at the bottom of the HTML file.
<!doctype html>
<html>
<head></head>
<body></body>
<script src="manifest.json"></script>
<script src="sdl.js"></script>
</html>
@kshala-ford Thanks for your response. I appreciate you taking the time to discuss these items so we can arrive at the best outcome for SDL. Responding to my items below:
|8
The burden is on the shoulders of the OEMs. An OEM will have many different versions of SDL Core on the road and these vehicles would request an app list from an OEM app store backend returning a list specific for the vehicle. The OEM backend would be based on data from the SDL server.
Thanks for clarifying. Can we explicitly state in the proposal that managing app updates is the responsibility of OEMs and that no changes to SDL Server or SHAID are being suggested by this proposal to support such functionality?
|9
This is one possible option to submit the compressed package to smartdevicelink.com. Imagining an app certification for such apps I believe it would be necessary. However, we're planning a developer to submit to our (OEM) developer platform (which can be referenced from smartdevicelink.com if such an API exist).
I agree, having developers upload their compressed WebEngine app package to smartdevicelink.com is ideal. Can you please add additional details to the proposal to clarify that the Developer Portal is to be updated to allow developers to upload their compressed WebEngine app package (please also list the allowed file extensions), the URL of the WebEngine app package file to be added to SHAID, and that OEMs are required to download and serve copy of the package file to their fleet to spare the SDLC from the potentially significant hosting costs?
|10
I think this is a general question about SDL server and versioning. The first thing I have in mind to solve the problem for this proposal is to:
Deprecate BOTH and don't add ALL Add a new parameter to (Cloud)AppProperties named hybridAppPreferences which is an array The old parameter remains for existing clients in the wild. The new is the successor and should be used in favor or the old one. ALL is realized by the array including all enum elements. Old parameter set to BOTH would match the new parameter set to [MOBILE, CLOUD, LOCAL] (or embedded... terminology yet to be decided)
Alternatively, we could change the developer-facing definition of BOTH
to actually act as an "all", without changing the name, since the underlying behavior would not be changed. This would avoid the technical debt of adding an ALL
option or introducing the duplication of data. I'm not sure if there has been a precedent set for scenarios like this?
|11
This is funny because in automotive using the word embedded is understood as "it is shipped with the vehicle" or "it's purely offline" or "it's fixed with the surrounding (software)". Local is describing a location instead. I would say "A local web application is running in an embedded WebEngine".
To the point that "EMBEDDED" already exist... I want to refer to 2b from Joey and defer the discussion to the next SC meeting. I believe there's need to discuss how EMBEDDED got introduced by the PM.
I didn't realize @joeygrover had already made this point in a previous comment. Deferring to his original comment "2b".
|12
You are right. The possibility of backend logic or data changes post-certification is a challenge. It's necessary to verify that the requests an app can send are known during certification and that they are all are properly tested. The app behavior might change due to backend sending different data hence triggering the app to behave differently. I want to note that WebEngines prevent apps to post-load executable code from a backend.
Is this the responsibility of the SDLC or OEMs? Testing every remote call and attempting to observe what the application does with the responses may not be reasonable in every situation (e.g. heavily-obfuscated code, unintelligible API responses, etc).
|13
The plan is when an app is installed into the vehicle, the HMI sends SetAppProperties to SDL Core with the app ID. SDL 0248 was designed for this case to allow SDL requesting a policy table update if the provided app ID is unknown.
I don't think this fully addresses my question as to whether or not an OEM must use a Policy Server to support WebEngine applications. Can you please provide additional details?
|14
Can you please be more specific to "so much"?
@joeygrover also hints at this in his "number 6". Items such as app_icon, app name, vrNames, appVersion, sdlMinVersion, ttsName, etc, could all be added to and pulled from app_policies
objects within the Policy Table, allowing app developers to maintain those values on smartdevicelink.com rather than splitting their metadata into multiple locations. Is there a particular need to have these values available prior to being able to make a Policy Table Update?
|15
I want to make everyone aware of the RFC 5646 which we should follow everywhere where internationalization and localization is required. to your point: Hyphen is okay but I don't think lowercase region is a good idea. It should be uppercase.
I agree that it should be uppercase (e.g. "en-US"), but unfortunately there is a longstanding precedent of these locale strings being all-lowercase (SDL Core, SHAID, SDL Server). Changing them now would create unnecessary complications without any functional benefit, and likely deserve its own Evolution Proposal.
1. This first piece was just a clarification as the flag isn't presented until after the statement. Not a big deal.
I will always bring up concepts like this to save us from the "death by 1000 cuts" from thinking each feature isn't much of an addition. Maybe another INI setting? Again, not a blocker for me, but I wouldn't feel right not talking about it as an option. This could additionally done as a later proposal if performance issues are found.
2a. The clarification is that there is no difference between a "cloud app" running on the same machine as Core/HMI and a "local/embedded" app running on the same machine as Core/HMI. There were actually a few offline conversation with the Ford team about this as well, hopefully Stefan can chime in on those.
2b. Yes we can talk about this during the call. We will always strive for better spec out during this process as we continue through the process. If the PM or any author ever misses something that anyone finds that they need more info on or don't agree with, please let us know! Sometimes it's not easy to find all the details when writing proposals and that's what's so great about this process, is that we have other experts looking over our ideas with a fine tooth comb.
3. & 6.\ & 14. Nick explained most of this. I want to understand this statement better:
It's not possible to change the app appearance or reachability (voice commands etc.) from a different source.
The app can't change its icon, VR commands etc from their actual javascript app? I don't think this is explicitly stated in the proposal. Essentially these apps will not have access to RPCs like SetAppIcon
so new functional groups will have to be created in the policy server for apps packaged this way.
4. Yes this would be loading the javascript directly in the same html as the HMI. The idea here is for OEM apps, like app catalogs, that an OEM would always want loaded and ready without the need for opening a new HTML file in the webegnine. I don't see this as useful for 3rd party apps to be clear. Maybe adding a new param "entrypoint_type" : [html | js]
5. Can we make sure the param gets updated and add a protocol spec version?
7. Again this is just a clarification because the statement in the proposal is a bit misleading. I'm not debating if the proposed platform is better/worse than using the JVM, but the statement itself should be updated to be correct.
8. I'm concerned about this and (9) below and the lack of requirements or suggestions for implementation. While I understand that the author's OEM uses servers outside of the open-source, I fear that this leads the author and OEM to neglect the open-source and other OEM's ability to implement this feature well because their internal requirements for implementation are not considered for other SDL OEM members.
This is also fragmenting the SDL ecosystem at a time when we've been consolidating the fragmentation in the ecosystem through SDL Server as much as possible. If at all possible, we should probably try to have this work in the open rather than requiring custom OEM servers for their own app stores. After all, we have support in the SDL server for policies for all the various versions of Core that are running, so I don't see any technical reason that we couldn't serve the app packages.
9. Like with the above (8), I'm concerned about saying that developers will just submit apps to your custom OEM platform. We have the open-source system and it needs to support new features for the other OEM members as well. I believe that the details that are being worked out for your server platform should also be worked out for the open-source server and developer portal, and placed into the proposal and considered because they're all part of this feature.
I understand your resistance to do that, but I'm trying to nail you down on this a bit because it feels like the proposal is too loose on these details for the open-source server design. I understand that your OEM is building it's own server and doing this work itself, but it still must be designed and done for the open source as well. I'm glad that you're amenable to developers uploading updates to the open smartdevicelink.com portal, but I think more details need to be hammered out around how the server will either (a) serve those app packages and updates to end users or (b) serve those app packages to OEM servers.
Since you did mention it as well, do you have details on how app certification is supposed to work with these apps? What will we use to download and certify applications? Are updates to emulators required to enable them to test these app packages?
12. I think there are two few problems here: a. Certifying apps at every update is a huge burden on the SDLC. b. Certification is very difficult because even if apps can't download executable code, they can have a web-call that "flips a switch" onto a different embedded code path, or even just "turn on" an endpoint that starts uploading user data maliciously. Unless we're examining the source code of the app, it's very difficult to certify any malicious actor.
17. I agree, I'm just pointing out that it's much more difficult to update the vehicle/IVI than it is to update a developer server.
18. Could this clarification be added to the suggestions / requirements that I requested above for the OEM server / open-source server design?
19. But we do this currently for policy server updates, for example where the policy data is requested and then passed back using SDL. I think that this could be done well and would expand the feature's capabilities quite a bit. It's not necessary to have it do so via an OS-level feature but could be done like we do with policy server updates and other SystemRequest
s.
20. Ah okay, understood. I had read it as a requirement, but I understand what you're saying now. Perhaps it could be clarified in the proposal?
The Steering Committee voted to defer this proposal keeping it in review until our next meeting on 2019-09-17, to allow for additional conversation on the review issue.
The author wanted to discuss his suggestion in this comment to add LOCAL
as a hybridAppPreference
and LOCAL
as one option on the developer side compared to the equivalent embedded. The PM noted that in the proposal for the Java SE and Java EE library for the Java Suite (SDL 203 – Cloud App Library – Phase 1) there is a section called “SDL Cloud as Embedded App” (images on how this works can be viewed within the proposal). There was discussion between Ford members and the PM on how these applications could be used in an embedded setting and what the preferences would be in terms of a cloud connection and an embedded connection (after that discussion, there was not any additional feedback on using that specific nomenclature for that situation). Once SDL 203 – Cloud App Library – Phase 1 was accepted, in order for a policy table to correctly identify permissions between an embedded and a cloud application; a separate platform was created so that the two applications could maintain different app ids. The SDL 0158 - Cloud Transport Adaptor proposal was created prior to the Cloud Library, which is why the enums are Cloud
or Mobile
. There was no distinction created between the Cloud
or Embedded
for the preference enums, instead, it was discussed that they would be treated similarly on the head unit given each preference and they could be treated differently as applications through policies.
In response to the PM the author stated that when this proposal was created, he was not aware that the policy changes would be needed.
In conclusion, all parties agreed that it needs to be determined how to move forward from here, as right now there are strictly cloud and embedded platforms where theoretically an application could have a multitude of different cloud applications in their implementations as well as their embedded/ local implementations. It was noted that while thinking about this we need to keep in mind that mobile apps will need to have their own app ids.
The author also discussed the question in this comment stating if it was required to have an SDL server updated when running apps in the WebEngine, the answer is no and that the policy server is not required to run, there will be no extra entries need as it is just another appearance of the app with a different app ID.
As the next steps, the author is to review the issue comments to summarize what topics remain open and what topics need further discussion and provide additional documentation/details on how the SDL server and WebEngine will interact.
The Steering Committee voted to defer this proposal, keeping it in review until our next meeting, to allow the author time to review the issue comments and summarize what topics remain open and what topics need further discussion, as well as provide additional documentation/details on how the SDL server and WebEngine will interact.
|2a Clarifications and decisions need to be noted here in the comments. Or a reference to previsous decisions is needed or at least SC meeting minutes need to mention this. I'm sorry but as the author I cannot accept agreements made offline (even if they were made with the authors organization). The SDLC stated many times that conversation should happen in the evolution issue but the ultimate decision is made in the SC meeting which should appear in the meeting minutes.
The enum HybridAppPreference
is related to the application location (MOBILE means both Android and iOS). As we discussed in the last SC meeting this is different to |2b where the developer portal is more specific (Android and iOS are separate).
In hope that below table helps understanding and finding a solution
Platform | Mobile | Cloud | Local |
---|---|---|---|
Android | x | ||
iOS | x | ||
Java (SE/EE) | x | x | |
node.js | x | x | |
WebEngine | x |
Technically every x
would need an app ID ... but this is not feasible. On the other hand only three preferences don't make much sense either. I believe we need following App Preferences:
|4 I appreciate the idea but I believe "launching" js files into an existing web engine instance could be a big addition to the existing logic and can be really complicated to support. Not only is it required to have an entrypoint to a js file but also a list of all dependencies. Another issue is namespacing and sandboxing. The app could read everything (functions and data) that is defined within the engine instance. The feature itself could be tricky and complex but adding it as a subsequent proposal should not be that much of a problem. With other priorities it's callenging enough to make progress on the existing items.
|8 Sure. We can state that
managing app updates is the responsibility of OEMs and that no changes to SDL Server or SHAID are being suggested by this proposal to support such functionality
|9 @nickschwab Sure I can add the details for the developer portal. To be clear: The URL you mentioned must refer to the package on a trusted host by the SDLC. The package should never be made available directly to vehicles. OEMs should have their own backend where they may or may not add apps (annd app updates) from the trusted host. Whenever there is an app added or a package updated the developers or the portal can notify the OEMs to review and certify the app package before they copy and repackage the app on their own backends. This gate is required to make sure that OEMs can control the app updates to their vehicles and prevent malicious app packages to be delivered to the vehicle.
@joeljfischer I understand your point and vision to be open-source and please don't think I (I) am the resistance to this. The OTA software delivery is a very complex, complicated and security critical procedure. Ford vehicles can only accept packages in a specific format from a trusted proprietary source controlled is by Ford through a secured communication channel which is also controlled by Ford and the telecommunications companies Ford works together. A lot of effort is spend into the system to verify validity, quality and security. Always keep in mind that automakers are selling products that are potentially dangerous to the driver, passenger and pedestriants. We must make sure the vehicles are safe to use and nothing is distracting the driver. We want to utilize this system as much as possible which requires a gate for app packages to be tested, certified and repackaged. Add an external source with a different package structure and questionable trust of control and integrity is beyond debate.
|12 With an SDLC app certification I would expect only a brief test to be done that can confirm "during tests there was no suspicious data traffic other than the regular use case related traffic". OEMs may choose to perform the test I mentioned. @joeljfischer You are right with 12 a which is why OEMs should also test the app as they would do anyways (everything else would be blindly trusting an external source). You're also right with 12 b which is why code analysis and audit is required to find logic that changes the app behavior. At the end the OEM and the app partner would need to create an agreement regarding this point.
|13 No. A policy server is not required to run WebEngine applications. the policy control doesn't change with this proposal (except ability to add app properties from the HMI_API but this also doesn't require SDL server).
|14 The reason why these values are in the manifest is becaues these values are part of the OEM certification (at least for us at Ford). I'm afraid of 1. a complexity increase updating the app content with the package and app metadata with policy 2. App users are surprised by app appearance changes without an app updated (users don't understand the difference of an app update and policy update) 3. metadata changes are not OEM certified (unless we build a separate metadata certification process). Including metadata in the app package bundles "everything" together for certification.
@joeygrover
The app can't change its icon, VR commands etc from their actual javascript app? I don't think this is explicitly stated in the proposal. Essentially these apps will not have access to RPCs like SetAppIcon so new functional groups will have to be created in the policy server for apps packaged this way.
The app can change it's icon or registration because the RPCs are not blocked. Why whould we. This would be quite strict from my point of view. The purpose of the manifest content is to make the library able to read it and automatically register, change registration and set the app icon using the manifest data. This can be explicitly stated in the proposal.
|15 I have a strong opinion to use industry standard formats in favor of how other variants we have with SDL (the server uses all lower case... mobile API uses all upper case). Because we use ISO 639-1 for language codes and ISO 3166-1 alpha-2 for regions the issue is not major. If the metadata is used with the mobile API (RAI params or ChangeRegistration params) all needs to be converted to upper case anyways. For using it in SDL server (e.g. auto reading list of supported languages/regions) convert it to lower case. An evolution proposal isn't needed unless we want to alter the existing structure.
|19 And policy udpate through apps is one of the weakest and error-prone spots we are seeing in our vehicles. It is impossible to guarantee the router app can properly connect and stay connected 100% of the times. Connectivity is a fundamental feature required for apps to work and I don't think it makes sense to increase complexity and rely on an app potentially running in black box.
The Steering Committee voted to defer this proposal, keeping it in review until our next meeting, to allow more time for the author and Project Maintainer to discuss the open issues.
@kshala-ford the Project Maintainer has worked to summarize the items posted regarding this issue. Please see below, and let us know your thoughts. Thanks!
Under Discussion - Author to Respond 2. Here is a table with values we believe should be used for categorizing app platforms on the dev portal and for the HybridAppPreference enum:
Platform Name | Dev Portal Classification | Hybrid App Preference |
---|---|---|
Android | Android | MOBILE |
IOS | IOS | MOBILE |
node.js | Cloud | CLOUD |
node.js | Embedded | CLOUD |
Java SE | Embedded | CLOUD |
Java EE | Cloud | CLOUD |
Webengine | Webengine | WEBENGINE |
Just a bit of background on the HybridAppPreference enum. This enum was created to set a preference for a user when there are multiple platforms of the same app available in the vehicle. We originally grouped the enum into two different groups MOBILE and CLOUD as a way to categorize their transport types. This means Core does not know the difference between an embedded JavaSE app and an off board node.js app because it simply sees an IP and port in which to connect with. I first thought of including the Webengine platform within the CLOUD hybrid app preference, but because the websocket connection scheme and client/server roll between core and the app are reversed, I thought it should have its own hybrid app preference category.
Regarding hybrid app preferences: BOTH & ALL. To ensure backwards compatibility with older systems I believe we should not deprecate BOTH and add a description that BOTH applies to all categories for newer versions.
5. Can you please provide an answer to this question: Can we make sure the param gets updated and add a protocol spec version?
9. We agree. In order to ensure "the package should never be made available directly to vehicles," we can either add S3 support to the SDL Server, or provide a skeleton function that OEMs must customize in order to copy and store the app packages on their own file servers. Currently the SDL Server does not support enough information for an app store for WebEngine applications. If this is required with the implementation of this proposal, please provide more information on what this would entail. If this is an implementation detail and does not need to be defined in the proposal, the Project Maintainers will assume autonomy to develop app store support for WebEngine applications.
12. While the current SDLC App Certification does not include traffic monitoring, it is much more extensive than "a brief test." We recommend not including traffic monitoring as the time, cost, and effort associated with completing this is not feasible within the current scope of the SDLC. We also recommend testing only the initial submission of hte application, for the same reasons as noted above. OEMs should be responsible for testing further updates at their discretion.
15. If this information will be passed to the policy table in any way, we'd prefer to match the formatting currently used within the policy table.
Project Maintainer Requests Revision 7. Proposal to be revised to remove "java" from the following statement
Both are difficult to sandbox. Compared to a web engine the effort to sandbox a Node.js or java application and to protect the vehicle system are very high.
8. Proposal to be revised to include the following:
managing app updates is the responsibility of OEMs and that no changes to SDL Server or SHAID are being suggested by this proposal to support such functionality
18. Author to add the following clarifying language to the proposal that the OEM app store needs to be capable of updating applications, determining versions in comparison to the supported SDL version. This may also be impacted by item 9.
20. Author to add the following clarifying language to the proposal that the OEM app store needs to be capable of unzipping applications, feeding them to the WebEngine, and to be secure from attacks caused by unzipping potentially malicious information.
21. Author to specify that this script import occur in the head of the HTML
Resolved/No Action Needed 4. Okay 13. Understood, thanks for the clarification. 16. This has been covered in item 9 - no action required. 17. No action required. 19. No action required.
The Steering Committee voted to defer this proposal, keeping it in review until our next meeting on 2019-10-08, to allow the author time to respond to the comments raised by the Project Maintainer.
|2.
I first thought of including the Webengine platform within the CLOUD hybrid app preference, but because the websocket connection scheme and client/server roll between core and the app are reversed, I thought it should have its own hybrid app preference category.
I think I now understand. Dev Portal Classification
is related to the location of the app while Hybrid App Preference
is more a connection category.
Regarding hybrid app preferences: BOTH & ALL. To ensure backwards compatibility with older systems I believe we should not deprecate BOTH and add a description that BOTH applies to all categories for newer versions.
I don't think this would be a solution for a clean API. I would suggest a new parameter like hybridAppPreferenceList
which is an array and allows any combination of connection category. The SDL server could send BOTH for the old parameter if the new parameter is set to more than one entry. Any thoughts?
|3. & 6. & 14.
Is the manifest only for the JavaScript library to read into its config, or is this for Core and HMI to build out this information for the app to be displayed to the user before it is launched on the head unit
It's for the SDL developer portal and for OEMs to read app assets automatically when developers submit builds. It's for the vehicle to display the app prior to launch and it can be used for the library to automate app registration which is not part of the proposal yet but definitely can be added and documented (the idea came to me while it was in review).
|5. Sorry for being silent to this point... Yes I can add sdlMinProtocolVersion and rename the other parameter to sdlMinRPCVersion
|9. The OEM app store role is quite similar to the AppStore or PlayStore. The SDL server as is should be sufficient as all the information needed should be in the dev portal or in the builds manifest.
|12 agree
|15 The language/locale information would not be passed to the policy table. As I already mentioned it should be converted to the Language enum or to the policy table so that it matches.
|7 I disagree to the PM revision request. Java doesn't have any sandbox protecting the system hence effort to introduce a protection compared to a web engine is very high.
|8 agree
|18 agree
|20 I think the language should be the vehicle (or infotainment system) needs to be capable of unzipping applications, feeding them to the WebEngine, and to be secure from attacks caused by unzipping potentially malicious information.
|21 I disagree to this. Technically it is not required to specify the script element location as long as the manifest is mentioned before the sdl library. It'll be a hassle for developers and can cause problems with the planned projection mode.
Per the author's request, the Steering Committee voted to return this proposal for revisions. Summary of agreed upon revisions is below:
5. Add sdlMinProtocolVersion and rename the other parameter to sdlMinRPCVersion
8. Proposal to be revised to include the following:
managing app updates is the responsibility of OEMs and that no changes to SDL Server or SHAID are being suggested by this proposal to support such functionality
12. Include following clarification regarding SDLC App Certification:
We recommend not including traffic monitoring as the time, cost, and effort associated with completing this is not feasible within the current scope of the SDLC. We also recommend testing only the initial submission of the application, for the same reasons as noted above. OEMs should be responsible for testing further updates at their discretion.
18. Add clarifying language to the proposal that the OEM app store needs to be capable of updating applications, determining versions in comparison to the supported SDL version. This may also be impacted by item 9.
Note that the following item numbers will be re-reviewed and discussed when the revised proposal enters review: 2, 7, 9, 15, 20, 21.
The author has revised this proposal to include the agreed upon revisions noted in this comment. The revised proposal will now be in review until October 22, 2019. As a reminder, the following item numbers from previous reviews are still in discussion: 2, 7, 9, 15, 20, 21.
Gentle reminder. Looking for feedback to the remaining items in https://github.com/smartdevicelink/sdl_evolution/issues/767#issuecomment-539608000 . I'll try to be prepared for the next SC meeting in case of new comments during this evening.
@kshala-ford please find the Project Maintainer's feedback on the remaining items as well as a few new items based on the latest revision below, and let us know your thoughts, thanks!
2. We would like to see the proposal updated to include the table (provided in this comment) with values we believe should be used for categorizing app platforms on the Dev Portal and for the Hybrid App Preference enum, with "CLOUD" used as Hybrid App Preference for "WebEngine" based on the rationale below.
After more consideration, hybridAppPreferences can be specifically related to either mobile or non-mobile (cloud, embedded, WebEngine) connections. For non-mobile, we think the "enabled" key value pair in the policy table app entry should be used to define preferences for app connections when multiple non-mobile options are present. For example, if both "cloud" and "web engine" app entries are present, the preference for WebEngine would require the "cloud" app entry to have 'enabled="false" ', while the WebEngine entry would have it set to true. Then pending on HybridAppPreference, either the mobile version would be shown, the WebEngine would be shown, or both. We should reserve HybridAppPreference to be used to specify mobile versus non-mobile.
If we were to implement a hybridAppPreferenceList and a WEBENGINE enum value, this would require synchronization between the cloud and WebEngine variants to maintain the same hybridAppPreference list between different policy table entries. If a cloud app and a WebEngine app had different hybridAppPreferenceList, this could lead to weird race condition behaviors when the two apps register. We believe designating WebEngine apps under the CLOUD hybrid app preference is a simpler solution than requiring the burden of keeping cloud and WebEngine app policy entries synchronized.
7.
The original security model provided by the Java platform is known as the sandbox model, which existed in order to provide a very restricted environment in which to run untrusted code obtained from the open network. Source
As we see, the Oracle documentation clearly states that Java does actually have a sandbox model. The model has since been extended with the previously mentioned security models. It includes editing a permissions file that is similar to policies to enable apps to have access to certain system APIs or to deny them. Not only is Java's security model not difficult to implement, but it is more robust than that of javascript giving granular control for specific APIs to specific apps. The ask from the Project Maintainer is simply to have "Java" removed from the statement in question as it is incorrect and misleading.
9. We disagree with the author, and believe creating an API and points to support the functionality of an app store (listing supported applications) is required. The exact API call will be an implementation detail, however the return value will be in a similar structure as the Policy Table app entries, including app id, app version, localized app names, localized app descriptions, app icon, sdl min rpc version, sdl min protocol version, TTS name, and VR app names. As another reason to include this information in the policy table, we can create an API to query this information from the existing database.
15. The SDLC will need to vote between the two presented options:
20. We are okay with your suggestion: "the vehicle (or infotainment system) needs to be capable of unzipping applications, feeding them to the WebEngine, and to be secure from attacks caused by unzipping potentially malicious information."
21. Recommended location of the manifest's Githubissues.
Hello SDL community,
The review of the revised proposal "SDL 0240 - WebEngine support for SDL JavaScript" begins now and runs through December 10, 2019. Previous reviews of this proposal took place November 13 - 19, 2019, October 16 - 28, 2019, July 17 - October 8, 2019, and June 19 - 25, 2019. The proposal is available here:
https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0240-sdl-js-pwa.md
Reviews are an important part of the SDL evolution process. All reviews should be sent to the associated Github issue at:
https://github.com/smartdevicelink/sdl_evolution/issues/767
What goes into a review?
The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of SDL. When writing your review, here are some questions you might want to answer in your review:
More information about the SDL evolution process is available at
https://github.com/smartdevicelink/sdl_evolution/blob/master/process.md
Thank you, Theresa Lech
Program Manager - Livio theresa@livio.io