Write some examples about how Data Store API works with Messaging API

[zolkis]

This is more about possible configurations for different messaging data sources:

• messages coming from the device/built-in modem
• messages coming from paired Bluetooth devices via MAP (multiple devices should be supported).

So this would expose e.g.

• label = 'messaging',
• name = 'msg-sim1' | 'msg-sim2' | 'msg-[bluetooth-name-1], ...
• owner = 'service1' | 'service2' | ...

We need to give some guidelines to implementations on what names to expose (beware phishing).

Then, did we agree on that service id's are considered part of the data?

[airpingu]

A few questions:

1. label = 'messaging' makes sense to me and that's something we should define in the spec. Question: we should define it in the DataStore API or the Messaging API or both?
2. I don't think we should pre-define the name in the spec because it seems platform-dependent. The web pages can just use getDataStores('messaging') to retrieve all the valid data stores but should not be aware of what the names of data stores would be. Does that make sense?
3. The owner is defined as the manifest URL of the owner app in the spec. Since the data store now comes from the platform (built-in modem), we may define it as:
   • the manifest URL of the system app or
   • just 'system'

which one is preferred to be defined in the spec?

Btw, what do you mean by "service ids are considered part of the data"? Could you please elaborate more on that? Thanks!

[zolkis]

The label should be specified at least in the Messaging API.

I didn't mean predefining names, these were just examples. But we need to make a note for implementations for how to expose names (beware privacy).

The value of 'owner': let's go one step further and explain what apps can do with it. That will define what ownership means here. Depending on viewpoint, owner can be the user, the provider, the service, the app. What you refer to as 'system', is not clear and it's too wide.

About service id's: if the user has multiple SIM cards and wants to handle messages belonging to each SIM in different data stores, how is it possible to do? If it's part of the data, then separation is not possible. If it's part of the store metadata, then yes, but we need to standardize.

[airpingu]

Yes! We're aligned. We only need to describe some examples for name and owner, except for label which should have predefined values for build-in data stores.

About the service IDs, do you mean add a serviceId in the DataStore interface just like label? If so, i think it's a little weird from the point of view of DataSote API because it's really NOT related to the purpose of this API. That is, the concept of service shouldn't be involved in the DataStore API so that we can make this API more independent and generic.

I think it's a good point but I haven't had a good solution to it for now. How about adding a generic metadata attribute which is a JSON object and can be used in anyway the apps or the platform (build-in) wants to mark for the data store?

[airpingu]

After some thoughts, I think we still need to define both label and name, because the messaging app that want to access the messaging data stores have to claim its manifest by the data store's name:

{
  datastores-access: {
    "messaging-sim-1": {
      "access": "readwrite",
      "description": "access (read and write) the built-in messages data store at SIM1"
    }
  }
}

if the system provides a build-in messages data store like below, where name = "messaging-sim-1":

DataStore {
    label = "messaging";
    name = "messaging-sim-1";
    owner = "sim-1";
}

One issue here; how can the messaging app knows how many SIMs are there in order to claim the data store names in the manifest?

[zolkis]

Well, let's track back to the use cases what we want to support:

• multiple SIM / data sources in general
• stored together or separately (platform/device policy choice).

For a given platform/device, there are customization points, e.g. screen size/resolution, max number of supported SIM's, etc. These are variables of the device. When it comes to the store, there will be multiple devices to be supported by the same app/manifest. So we should take care what to put in the manifest vs for what things to have a dynamic capability/permission interface.

In this particular case, the permissions go along privacy zones: in general, you allow apps to access SIM cards, or other devices connected via Bluetooth. This we can do via manifest, but it would need an extra property (e.g. 'privacy-class') because neither 'label' nor 'name' fully match the meaning. The 'owner' could be enough, if these were static, but we may have a dynamic number of devices connected via Bluetooth.

So could we just say in the manifest what label and what privacy class datastores the app wants to access, and the access type (RO, RW). If it knows the name and owner, those could be specified, too. This would act as a filter for the user agent to determine which datastores are requested. Then, for instance, it could trigger a permission dialog showing these datastores with the names instantiated for that particular device, and the user could select which ones are granted for access to the app, before installing. Then the choice is stored in the user agent, and the app is served only with those datastores and access terms. If not enough for the app, it could tell errors, and the use should be able to go back and change the choice. But this is a policy, up to the device makers/stores.

{  // request access for all messaging datastores with SIM card 
   // and the accounts permitted by the user via dialog, e.g. facebook
  datastore-access:  // array of filters
  [ { 
       'label': 'messaging',
       'privacy': 'sim'  // or "bluetooth" or "account"
       'access': 'readonly',
    },
    {
       'label': 'messaging',
       'privacy': 'account',
       'access': 'readwrite',
    }]
}

Of course if the name is known, it could be used, but it's hard to define a namespace at app packaging time, unless it is meticulously specified by the store or this spec. I would try to avoid this level of hardcoding.

IMO specifying dynamic data such as the name or owner is not really needed either: the user agent will pop a dialog where the user may accept or limit the choices. There all matching datastores would be shown, including facebook, and the user will make a choice.

BTW this is the way the Presentation API also works: delegates these fine-grained and/or dynamic choices to the user agent, instead of exposing screen id's with discovery mechanism in the API. Whereas id's could be used in an API, they cannot be used in manifests.

In the case of Messaging, the only differences between stores will be the service id. The choice made by the user is what messaging services are permitted to be handled by the app. These may be exposed in separate or one datastore, depending on user agent (platform/device policy). As messaging services may appear and disappear, for newly configured ones the user agent may need to show the permission dialog again (or by default is off and the configuration could be enabled from settings) - depending on device policy.

So IMO what we can standardize for generic DataStore:

• 'label' (or rather called 'class', or 'type'): "messages", "contacts", "calls"
• 'privacy': "sim", "bluetooth", "account"
• 'owner': DOMString, matching the owner app's URI, if there is such thing known.

Then, in the Messaging spec, we will describe how user agents should implement datastores (how to wire label/type, privacy, owner, name, etc).

Also asking @marcosc, @sicking, @mounir, @anssik: what's your opinion?

[airpingu]

Looping @ehsan as well. Please see the above comment.

[airpingu]

Thanks @zolkis for the suggestion! Will come back with a proposal. Please stay tuned.

[airpingu]

I'm opening a conversation in Mozilla internal to discuss about this issue. Before a conclusion comes up, I'd suggest let's roll back to the status that didn't take label into consideration. Eventually, I found out the idea of label is not really useful because it simply does what the original name was supposed to do. To distinguish the sources of data stores which share the same name, we can take use of the owner. Summarize my thoughts as below:

• For Facebook messages provided by the Facebook app 1, the attributes of DataStore will be:

    DataStore {
        name = "facebook-messages";
        owner = "app://facebook1.gaiamobile.org/manifest.webapp";
    }

• Similarly, For Facebook messages provided by the Facebook app 2, the attributes of DataStore will be:

    DataStore {
        name = "facebook-messages";
        owner = "app://facebook2.gaiamobile.org/manifest.webapp";
    }

Since the Facebook messages must be provided by third-party apps, the owner is supposed to be the app's manifest URL. When the web content calls getDataStores("facebook-messages"), the promise's resolver will return an array of 2 data stores in this example, where the name is the same (i.e., "facebook-messages") but the owner can be different. It's up to the caller to decide which data store it wants to sync up (could be both or just either one which is more reliable).

• For built-in SMS on SIM-1, the attributes of DataStore will be:

    DataStore {
        name = "built-in-sms";
        owner = "system-sim-1";
    }

• For built-in SMS on SIM-2, the attributes of DataStore will be:

    DataStore {
        name = "built-in-sms";
        owner = "system-sim-2";
    }

When the web content calls getDataStores("built-in-sms"), the promise's resolver will return an array of 2 data stores, where the name is the same (i.e., "built-in-sms") but the owner can be different. In this way, the messaging app only needs to claim the following manifest without knowing how many SIMs are actually installed in the device:

    {
        datastores-access: {
            "built-in-sms": {
                "access": "readwrite",
                "description": "access the built-in SMS data stores"
            }
        }
    }

Note that "built-in-sms", "system-sim-1", "system-sim-2"... have to be something formal defined in the Messaging API spec.

• For built-in MMS on SIM-1, the attributes of DataStore will be:

    DataStore {
        name = "built-in-mms";
        owner = "system-sim-1";
    }

• For built-in MMS on SIM-2, the attributes of DataStore will be:

    DataStore {
        name = "built-in-mms";
        owner = "system-sim-2";
    }

The same. The web content can call getDataStores("built-in-mms") to retrieve all the MMS data stores which are currently available in the device, as long as it claims the following manifest:

    {
        datastores-access: {
            "built-in-mms": {
                "access": "readwrite",
                "description": "access the built-in MMS data stores"
            }
        }
    }

where the "built-in-mms" has to be a keyword pre-defined in the Messaging API spec.

[zolkis]

This could work, except that what you use as 'name' is essentially a type, not a name. A name tends to be unique. In the case above, the combination of 'name' and 'owner' is unique.

From all examples above, the general type is 'messages', irrespective whether the provider is Facebook, or e.g. Vodafone, and the account is 'user1' or 'sim1'. The 'owner' property is probably enough to capture the specialization, if it is unique (the user agent can do an internal mapping if needed). Similarly, 'contacts' and 'calls' are standardizable values for other data stores.

About standardizing the namespace for 'owner': when the data is provided by an app, using the app manifest is a possibility, but it has issues:

• the permission and access level mechanism is not clear
• when you write code, you need to know the app manifest, but multiple apps could provide that data; therefore it is not wise to expose the owner.
• when the owner is the system, we want to make distinction what part of the system owns the data. So the original plain 'system' string value is not enough for our use cases.

The original use case is that the app could find the data it is able to and wants to handle. If we don't want to go into the hell of standardizing providers/accounts etc, likely ending up in SNMP depths, then we need to delegate that part to a user choice. My suggestion is that the app requests a data store of certain type or label(s), and the matching choices are presented to the user, who will make multiple selection. The result (array of datastores) is given to the app, and can be identified by the 'owner' property.

Examples for those choices. If the device has a SIM card provided by Vodafone, with support for SMS and MMS over LTE and CS, respectively, some devices sound present 4 choices to the user, whatever string meaning either:

• Vodafone SIM1 SMS over CS
• Vodafone SIM1 MMS over CS
• Vodafone SIM1 SMS over LTE
• Vodafone SIM1 MMS over LTE.

Others will present only two choices:

• Vodafone SIM1 SMS
• Vodafone SIM2 MMS, not caring about how the message is transported. On how these are determined, we cannot really standardize, and it is better to leave for the device/user agent.

I think we should not standardize the 'owner' property - it may not be even important to know for the app. The important bits are:

• the app can handle the datastore (and that is based on type/label, e.g. 'messages')
• the user has made a choice (if device policy is such, this could be skipped)
• the user agent can handle permissions and access rights (guaranteed by the implementation), with input from the app. The app could specify in the getDataStores API (second, optional parameter for 'access') whether it wants read-only, or read-write access. If it fails for read-write, it may still retry for read-only.

In summary, I propose the following:

• rename 'name' to 'type', which values as 'messages', 'contacts', 'calls'
• don't standardize the 'owner' string;
• if the app needs to specify the id of the app URI which is the data owner, let's define a separate mechanism for that via an API (third, optional DOMString parameter to getDataStores), rather than via manifest -by default let the user agent handle the selection via presenting a multiple selection dialog to the user
• actually there is nothing left which should be specified in the app manifest. All that would be handled by the API and exposed to the user via a dialog.
• this way we don't actually need to standardize anything in the Messaging, Contacts and Telephony API's, except defining the data structures which will be handled under the given data store type. Every information for further specialization, e.g. messaging/call service id, or address book id etc, should be part of the data.

I think this may be so far the simplest and most generic solution.

[airpingu]

I'd like to clarify my understanding about your suggestions. You mean we shouldn't distinguish the messages data stores by different type. In the above-mentioned cases, the type of all the data stores should be the same (i.e. "messages"). Also, we shouldn't standardize the owner in any specs (but we may still expose that for the displaying purpose in the prompting dialogue so that users can know which apps to allow). Eventually, it's users' decision to allow the accessibility through the prompting dialogue, instead of being matched by developers' codes.

So, all the data stores will be:

• For Facebook messages provided by the Facebook app 1, the attributes of DataStore will be:

    DataStore {
        name = "messages";
        owner = "app://facebook1.gaiamobile.org/manifest.webapp";
    }

• For Facebook messages provided by the Facebook app 2, the attributes of DataStore will be:

    DataStore {
        name = "messages";
        owner = "app://facebook2.gaiamobile.org/manifest.webapp";
    }

• For built-in SMS on SIM-1, the attributes of DataStore will be:

    DataStore {
        name = "messages";
        owner = "system";
    }

where each data record in this data store can have special properties:

record.type = "sms";
record.serviceId = 0;

• For built-in SMS on SIM-2, the attributes of DataStore will be:

    DataStore {
        name = "messages";
        owner = "system";
    }

where each data record in this data store can have special properties:

record.type = "sms";
record.serviceId = 1;

• For built-in MMS on SIM-1, the attributes of DataStore will be:

    DataStore {
        name = "messages";
        owner = "system";
    }

where each data record in this data store can have special properties:

record.type = "mms";
record.serviceId = 0;

• For built-in MMS on SIM-2, the attributes of DataStore will be:

    DataStore {
        name = "messages";
        owner = "system";
    }

where each data record in this data store can have special properties:

record.type = "mms";
record.serviceId = 1;

Calling getDataStores("messages") will return all the above data stores to the user. User has to determine which data stores can be accessed by the caller app through the prompting dialogue.

Is that accurate? :)

[airpingu]

What you mentioned about the prompting dialogue is almost the same as the Mozilla's current UX design https://bug942641.bugzilla.mozilla.org/attachment.cgi?id=8410050. Please see the page Duplicate datastore names. Regarding this part, we are almost aligned. ;)

[zolkis]

Yes, the above is one possible implementation in what is concerning the owners.

Though it comes to my mind that in the special case of messaging, there may be apps which want to handle only SMS, or MMS alone - that would require another label, or separate types for 'sms-only' , 'mms-only', 'sms-or-mms' and 'messages'. The latter would retrieve all messaging data stores including sms, mms, facebook, instant messages from other accounts, etc. It is up to the Messaging spec to define these extra types and how do they filter the message stores via the getDataStores() API.

The DataStore spec still needs to define 'messages' since it seems that it wants also to handle e.g. Facebook messages (in your examples), which are not related to the Messaging spec.

As far as I can tell, there is no need to specialize Contacts and Calls at the moment, and the values 'contacts' and 'calls' defined in the DataStore spec would be enough.

To complete your examples above, a call to getDataStores() would be e.g.

• getDataStores('sms-or-mms', 'readonly'): will pop up a dialog with all stores containing SMS or MMS data, but no other messaging type (facebook, IM, etc)
• getDataStores('messages', 'readwrite', 'app://facebook1.gaiamobile.org/manifest.webapp'): if the app can obtain ownership over the data store provided by the app indicated in the call, then return the store; if there are multiple such stores, then pop up a dialog (in practice it won't likely happen); and fail if permissions are not sufficient
• getDataStores('messages'): it would pop up a dialog with all messaging stores read-accessible on the device.

[airpingu]

Just had some internal discussions with Mozillians. Eventually, we concluded that in order to make the security model more consistent, we want to force the owner to be an app (the app's manifest URL). So, in the case of built-in messages data store, the owner has to be the manifest URL of the system app instead of the RIL or modem.

To do this, the system app can register for the existing event handlers in the Messaging API, like onreceived or onsent events, and the system app will be in charge of owning and managing the built-in messages data store through the Data Store API.

So, in the above examples:

• For Facebook messages provided by the Facebook app 1, the attributes of DataStore will be:

    DataStore {
        type = "messages";
        owner = "app://facebook1.gaiamobile.org/manifest.webapp";
    }

• For Facebook messages provided by the Facebook app 2, the attributes of DataStore will be:

    DataStore {
        type = "messages";
        owner = "app://facebook2.gaiamobile.org/manifest.webapp";
    }

• For built-in SMS on SIM-1, the attributes of DataStore will be:

    DataStore {
        type = "messages";
        owner = "app://system.gaiamobile.org/manifest.webapp";
    }

where each data record in this data store can have special properties:

record.type = "sms";
record.serviceId = 0;

• For built-in SMS on SIM-2, the attributes of DataStore will be:

    DataStore {
        type = "messages";
        owner = "app://system.gaiamobile.org/manifest.webapp";
    }

where each data record in this data store can have special properties:

record.type = "sms";
record.serviceId = 1;

I will go for this solution because if the owner can just be the app's manifest URL, the prompting dialogue can know how to display the app names in a more consistent way (please see the on-going UX design at https://bug942641.bugzilla.mozilla.org/attachment.cgi?id=8410050), which shows the list of app names that are requested to open their data stores to the caller app.

So, supposing a Foo App calls getDataStores("messages"), a prompting model will pop up at run-time to ask users the following question:

"Foo App would like to access messages data from:"

• Facebook App 1
• Facebook App 2
• System App (note: this can be just "System")

Then, users will easily check/uncheck the listed apps to decide whether or not to open their data stores' accessibility to the Foo App.

Regarding moving the datastores-access declarations from the manifest to the optional parameters of getDataStores(...), I'll tend to open separate issues to address that later. It seems to me manifests are still needed and useful because users can thus know the app's capabilities in advance to decide whether or not to install the app during the installation process. That is, if a unreliable app wants to access my sensitive data stores, I will definitely not to have it installed in my device.

[zolkis]

So we mandate the existence of system apps. Should work, except if the system app is native in our case - but we can work that around.

However, I am not convinced about your arguments about the manifest :). The user can uninstall the app at any time. At install time, the user can hardly tell whether the app is unreliable or not. Hard-coding these accesses in the manifest sounds bad to me. As I said, when you write an app, you should be aware of what other apps are installed in all devices you are developing for, and deploy a different manifest if some other app is replaced... This is non-deterministic, and sounds like a maintenance nightmare to me.

[airpingu]

Some actions based on the above-mentioned agreement:

1. Recover the use of name but change it to be type. Resolved this at: https://github.com/airpingu/data-store-api/commit/0425bd8f9ceb4ceb7e31418ed4596169672bc3bf and https://github.com/airpingu/data-store-api/commit/524bf6857348794e7fd2a2a377b9ebfc20886148.
2. Fire a separate issue for moving datastores-access manifest settings into getDataStores(...) as optional parameters at https://github.com/airpingu/data-store-api/issues/36. Let's keep track of this issue there. I'll discuss with Mozilla internal to see if it's worth it.
3. Leave this issue open to define some pre-defined type like messages or contacts and to describe the behaviour of the prompting dialogue in details.