Closed alinaserg closed 4 years ago
sophokles73
commented
5 years ago Not sure what you mean by IoT platform. I assume you are talking about Eclipse Hono and the device connecting to e.g. the MQTT protocol adapter, presenting an X.509 client certificate for authentication, right?
alinaserg
commented
5 years ago Yes, you are right.
alinaserg
commented
5 years ago Under just-in-time provisioning we understand the following: not provisioned in Hono devices attempt to establish the first connection (in our case devices connect via MQTT protocol Adapter) using a certificate. As the devices are not provisioned, the automatically registration of the devices should be started (Device Registry API should be called), if the devices have valid certificates. No manual provisioning of the devices is not necessary any more. After that devices are provisioned and registered in Hono and can establish connections.
We propose to take device credentials from the trusted CA certificate, which must be deposited in Hono before the device tries to establish the connection. It should also guarantee that only the devices with valid certificates can be registered.
sophokles73
commented
5 years ago So, in any case, the corresponding tenant(s) that the devices belong to would need to have been created upfront, along with the trusted CAs that should be used for authenticating the (unregistered) devices.
That said, trying to use the Device Registration API for the purpose of adding an unknown device would not work anymore because we decided to remove the optional operations from all of Hono's device registry related AMQP based APIs in favor of the upcoming HTTP based Device Registry API.
So, based on that new API, the protocol adapter would theoretically be able to add a device for a tenant (and credentials) based on information contained in the client's certificate. However, I am not sure if that is what we really want to do because it would introduce a new dependency on the device registry's HTTP API to the protocol adapters (which already have several dependencies).
Another option might be to allow an implementation of the Device Registration and Credentials APIs to support auto provisioning based on the information that the adapters provide when invoking the existing operations. However, some of them would probably need to be extended with some optional information.
alinaserg
commented
5 years ago The second option sounds interesting for us. Do you need some more information from us or it is already enough for you to accept it as a request for a new feature?
b-abel
commented
5 years ago I would like to work on this topic.
sophokles73
commented
4 years ago @b-abel any suggestions yet as to how you would like to approach this?
b-abel
commented
4 years ago I currently see 2 main options:
b-abel
commented
4 years ago Just-in-Time-Provisioning (JiT provisioning) is a two-step process: in the first step, the tenant is configured so that the feature can be used there. In the second step, the device connects for which there are no credentials and no registration in the device registry. Both are created. On subsequent connections of the device, the data is stored in the Device Registry and the authentication process is the same as always.
The first step is pretty straight forward because you only need to add a CA certificate as trust anchor. One thing is IMHO still to do here:
The tenant configuration should be extended by a property that allows you to grant and deny permission to each trust anchor to use it for JiT provisioning.
For triggering the device provisioning by the Protocol Adapter (PA) in the second step I see several options that I would like to discuss:
An advantage of Option 1 is that this operation could easily be offered by other components and systems, requiring only a configuration change on the PA from Hono's point of view. This would certainly be the most flexible option, but we would have another API with only one operation.
Option 2 would be fully transparent for the PA. The advantage that we don't have to change anything there, comes with the disadvantage that the PA doesn't "know" about it and can't react with a larger timeout to a longer duration of the provisioning process. In addition, the registration information is implicitly generated when another API is called. This is not obvious and makes it difficult to understand Hono. This would also be a practical problem if (in the future) a PA would call the Registration API before the Credentials API.
Option 3 has the advantage over option 2 of being clearer in the communication with the PA, since the PA must explicitly initiate the process. It shares the problem with option 2 that the credentials API also generates the registration information.
If we use JiT Provisioning only for client certificate-based authentication, the device does not need to know either Tenant-ID or Device-ID. Therefore, the Device-ID can easily be generated during JiT Provisioning.
We should assume that in productive deployments external systems have to be addressed during the provisioning process. Option 1 offers the most flexible support for such a use case. There, any system could offer the API instead of the support having to be built into and delegated from the Device Registry.
If you think of the Eclipse Packages project and look for example at the joint use of Hono and Ditto an additional feature would be required. To configure a "Thing" in Ditto you need a set of information like Policies, Features, Attributes. For this it would make sense to save a "Template" for the creation of the "Thing" together with the CA certificate during the Tenant configuration. The template should contain variables which are then filled with data from the client certificate during device provisioning. In addition to Ditto, other components might also benefit from this feature (possibly it can also be used in Hono for the registration information). The question is whether we want to address this in this issue ( it would need to be supported by the Hono Device Registry) or whether the problem should be delegated to the Packages project.
sophokles73
commented
4 years ago @b-abel thanks for laying out the options. I have some comments/questions/proposals
FMPOV there is no need to explicitly rule out Option 2 because, as you stated, this approach is fully transparent to the adapters. An adapter will invoke the Credentials service's get operation to determine the device ID based on the auth ID (the client certificate's subject DN in this case). If the registry supports JiT provisioning in its getoperation, then everything is fine and from the adapter's point of view this looks just like the device had already been provisioned.
So let's assume that the adapters always try to get credentials first. If the get fails (404) then the adapter could still invoke a separate API operation, either using a dedicated Provisioning API or another operation on the Credentials API. This way we could gradually introduce the feature in custom device registry implementations first and later add support for configuring protocol adapters with an (optional) provisioning endpoint which they can invoke to register devices on the fly if the get operation on Credentials failed and the Tenant is configured to support JiT provisioning.
As an alternative to Option 1 and 3 we could even simply define the payload of a Hono provisioning message and support configuration of an HTTP endpoint per tenant which the adapters POST the provisioning message to.
To configure a "Thing" in Ditto you need a set of information like Policies, Features, Attributes. For this it would make sense to save a "Template" for the creation of the "Thing" together with the CA certificate during the Tenant configuration. The template should contain variables which are then filled with data from the client certificate during device provisioning.
Why not store the template for JiT provisioning of devices in Ditto itself? FMPOV we only need to notify Ditto about the fact that we have provisioned a device for a particular tenant on the fly. The creation of the policy, features etc for the device is then the responsibility of Ditto. There is no need to store this information in Hono, or is there?
alinaserg
commented
4 years ago Hello @thjaeckle, I think we should involve you in the discussion regarding creation of Things during Just-in-Time Device Provisioing. What is your opinion to this proposal?
Why not store the template for JiT provisioning of devices in Ditto itself? FMPOV we only need to notify Ditto about the fact that we have provisioned a device for a particular tenant on the fly. The creation of the policy, features etc for the device is then the responsibility of Ditto. There is no need to store this information in Hono, or is there?
b-abel
commented
4 years ago @sophokles73 I like the idea of gradually building support for the new feature. Practically, I see some points where I wonder if we can really support this:
This way we could gradually introduce the feature in custom device registry implementations first and later add support for configuring protocol adapters with an (optional) provisioning endpoint which they can invoke to register devices on the fly if the get operation on Credentials failed and the Tenant is configured to support JiT provisioning.
If I understand you correctly, we are already in phase 1, in the sense that a custom device registry can do that today. There is nothing for us to do. Or would you like to add support for Option 2 to the example Device Registry? At least the external systems have a further limitation with option 2: in the "Get" operation of the Credentials API, only the Subject-DN is sent. Further information from the certificate (serial number, validity dates, issuer DN, ...) can therefore not be used in the provisioning process.
As an alternative to Option 1 and 3 we could even simply define the payload of a Hono provisioning message and support configuration of an HTTP endpoint per tenant which the adapters POST the provisioning message to.
In other words, this is option 1 with the difference that HTTP is used instead of AMQP. Right?
FMPOV we only need to notify Ditto about the fact that we have provisioned a device for a particular tenant on the fly.
I wouldn't save templates in Hono either. To be able to fill variables in the template with values from the certificate, they need to be sent to the corresponding component (Ditto). This must be supported by the PA (Option 1) or by the (custom) Device Registry (Options 2 and 3).
sophokles73
commented
4 years ago From the Credentials API spec for the get operation's request message:
Additionally, the body MAY contain arbitrary properties that service implementations can use to determine a device’s identity.
We already can include arbitrary additional information (or even the client certificate itself) in the request message. It is just a matter of defining a specific well-known property to hold that information in a future version of the Credentials API.
sophokles73
commented
4 years ago In other words, this is option 1 with the difference that HTTP is used instead of AMQP. Right?
To some extent, yes. the specific URI to POST to remains undefined, though, providing for more flexibility on the implementor's side.
thjaeckle
commented
4 years ago Hello @thjaeckle, I think we should involve you in the discussion regarding creation of Things during Just-in-Time Device Provisioing. What is your opinion to this proposal?
Why not store the template for JiT provisioning of devices in Ditto itself? FMPOV we only need to notify Ditto about the fact that we have provisioned a device for a particular tenant on the fly. The creation of the policy, features etc for the device is then the responsibility of Ditto. There is no need to store this information in Hono, or is there?
As Ditto I would assume to handle a special Hono message (e.g. via another defined content-type) whenever a device was just-in-time-provisioned.
I see three options where to "put" more metadata about the device creation:
No matter where it is done, which data would be "applied" could be based on a "device-id" pattern
My gut feeling is that when specifying such data only in Ditto, some important information of the newly connected device may be lost in the "JiT message". For example when not only the device-id is relevant for determining which JiT message data to send but other meta-data which is only available at a device-connectivity layer.
Don't you see that other systems than Ditto would benefit from more JiT provisioning information?
b-abel
commented
4 years ago @sophokles73
Additionally, the body MAY contain arbitrary properties that service implementations can use to determine a device’s identity.
We already can include arbitrary additional information (or even the client certificate itself) in the request message. It is just a matter of defining a specific well-known property to hold that information in a future version of the Credentials API.
That would work. Tradeoff would be that for every "Get Credentials" call much more data would be transmitted (when using Client certs).
b-abel
commented
4 years ago Thank you, @thjaeckle, for describing the metadata options. If I understand you correctly, your idea (at least for the first two options you outlined) is that Ditto should receive a full provisioning message from Hono that Ditto can use more or less directly to create a device presentation.
I imagine that there are two kinds of information:
The second type can be stored in a template in which the configuration of the "logical entities" is defined. The template contains (to be defined) variables which correspond to individual data fields in the certificate and which are used to apply values from the certificate during provisioning. A "JiT message" would only contain the values from the certificate. What should be done with them has to be stored in the template before.
I also think that other systems besides Ditto potentially want to use this mechanism. But I'm not sure that this means that the templates should be saved in Hono. The "schema" of the template for Ditto is defined by Ditto and should perhaps also be managed by it. If other systems also need templates, they would have to define their own schema for them.
No matter where it is done, which data would be "applied" could be based on a "device-id" pattern
The point here is that you want to have several templates configured for each tenant and then have to find a mechanism for selecting the appropriate template. Right?
For example when not only the device-id is relevant for determining which JiT message data to send but other meta-data which is only available at a device-connectivity layer.
The set of data available in Hono during device provisioning is limited by scheme X.509. The maximum Hono can do is to add its own metadata such as the client ID and a generated device ID to the "JiT message".
The situation would be different if Ditto had requirements for the structure of the Hono device ID. But FMPOV, the ID is only internally relevant to Hono and other systems can map it to their own IDs as needed.
As Ditto I would assume to handle a special Hono message (e.g. via another defined content-type) whenever a device was just-in-time-provisioned.
This is a fourth option in addition to the ones I described above. I assumed that the Protocol Adapter (PA) would send the data from the Cert to the Device Registry (DR) and the DR would inform Ditto (options 2+3) or an additional API would be available to send the data (option 1) directly from the PA. Of course, it is appealing to use the Northbound Event API instead of the new APIs. This, of course, requires a consumer to do the provisioning for Ditto.
b-abel
commented
4 years ago Because I find it difficult to keep track of longer discussions on Github, especially when many different questions have to be answered, I try to give a summary.
Sticking to the template idea we have the following:
b-abel
commented
4 years ago This approach allow for two strategies for the integration with external systems like Ditto: Either it is done by the (custom) DR or the "Device Provisioning" event is used.
b-abel
commented
4 years ago We still have to decide which data of the client certificate should be transferred to the Credentials API during JiT provisioning. We could start with the following:
FMPOV Hono should not make too many assumptions about which data from the client certificate is of interest to third party systems. Trying to get any possible value from a certificate, on the other hand, quickly becomes cumbersome and requires defining the formats. An alternative would be to send the entire client certificate to the Credentials API (either base64 encoded DER or PEM). I hesitate to add this overhead to any "Get Credentials" request (if JiT provisioning is configured for the trust anchor used). One option here would be for the PA to add it to a retry request only if it received a response of 404.
What do you think?
sophokles73
commented
4 years ago I would rather include the whole certificate instead of a subset of the information contained. That way, a registry implementation may be able to take advantage of extensions in the certificate that all devices it deals with contain etc.
I am not sure about the usefulness of using a re-try to save the extra bytes in the get Credentials request, though. Instead, we could use a flag contained in the protocol adapter configuration of the Tenant in order to determine whether a protocol adapter should include the certificate or not ...
b-abel
commented
4 years ago I would rather include the whole certificate instead of a subset of the information contained. That way, a registry implementation may be able to take advantage of extensions in the certificate that all devices it deals with contain etc.
Good point.
Instead, we could use a flag contained in the protocol adapter configuration of the Tenant in order to determine whether a protocol adapter should include the certificate or not ...
I would add a flag "allow-auto-provision" to the trust anchor configuration of the tenant. I' m not sure if we additionally need to configure it per protocol adapter. In any case, it may not be a lot of data and it only concerns special cases, but this data is potentially transferred unnecessarily for the affected devices throughout their whole lifespan for every single message, although it is only needed for the very first one.
sophokles73
commented
4 years ago I would add a flag "allow-auto-provision" to the trust anchor configuration of the tenant. I' m not sure if we additionally need to configure it per protocol adapter.
I do not think that it should go into the trust anchor configuration because, strictly speaking, it is not necessarily linked to a client using a certificate for authentication but instead should indicate whether auto-provisioning (or JiP?) should be allowed for that Tenant or not, regardless of the mechanism used for conveying the necessary information.
b-abel
commented
4 years ago I do not think that it should go into the trust anchor configuration because, strictly speaking, it is not necessarily linked to a client using a certificate for authentication but instead should indicate whether auto-provisioning (or JiP?) should be allowed for that Tenant or not, regardless of the mechanism used for conveying the necessary information.
I see that the configuration on the trust anchor couples it to the use of client certificates, which is theoretically not the only option. But it would make it more flexible since it allows JiT provisioning only for certain device classes. This would give IMHO a real benefit to the tenant.
By the way, @sophokles73, do you have a preference in which format the certificate should be included in the "Get Credentials" request?
sophokles73
commented
4 years ago But it would make it more flexible since it allows JiT provisioning only for certain device classes. This would give IMHO a real benefit to the tenant.
I do not understand your point/argument here. Can you be a little more elaborate?
do you have a preference in which format the certificate should be included in the "Get Credentials" request
I think there is only one obvious candidate, isn't there? What about the Base64 encoded byte array returned by X509Certificate.getEncoded()?
b-abel
commented
4 years ago Let us assume that a tenant manages different types of devices (1. washing machines and 2. battery-powered sensors). If we configure the JiT Provisioning permission on the tenant, the tenant admin must allow it for all types (if we set the flag per protocol adapter, the result will be the same if both device types use the same protocol). If, on the other hand, the flag is configured for each CA certificate of the tenant, tenant admins have a mechanism at hand with which they can freely control which devices are allowed to do JiT provisioning and which are not, by configuring different CA certificates.
As far as the templates discussed above are concerned, it makes sense to assign each template to one CA certificate. Tenant admins can then decide which template should be used for which device based on the used CA certificate. Even if the templates do not belong to Hono, this is an important use case for which the JiT support in Hono is needed.
sophokles73
commented
4 years ago If, on the other hand, the flag is configured for each CA certificate of the tenant, tenant admins have a mechanism at hand with which they can freely control which devices are allowed to do JiT provisioning and which are not, by configuring different CA certificates.
I do not think that tenants will use this approach to partition their set of devices. However, I can see the value of enabling/disabling a particular trust anchor for being used for auto-provisioning (I like this term much better than JiP). In a similar way we would need to mark a trusted authority that we use for validating tokens, assuming that in the future we want to also support auto-provisioning based on tokens ...
Not sure what your are referring to by templates, though.
b-abel
commented
4 years ago However, I can see the value of enabling/disabling a particular trust anchor for being used for auto-provisioning (I like this term much better than JiP). In a similar way we would need to mark a trusted authority that we use for validating tokens, assuming that in the future we want to also support auto-provisioning based on tokens ...
Does that mean you're for having the flag on the trust anchor?
It could look like this:
{
"tenant-id" : "TEST_TENANT",
"enabled" : true,
"trusted-ca": [{
"subject-dn": "CN=ca,OU=Hono,O=Eclipse",
"public-key": "PublicKey==",
"algorithm": "RSA",
"allow-auto-provision": false
}, {
"subject-dn": "CN=ca,OU=Hono,O=ACME Inc.",
"public-key": "ECKey==",
"algorithm": "EC",
"allow-auto-provision": true
}]
}Does that mean you're for having the flag on the trust anchor?
Yes, but can we please rename it to auto-provisioning-enabled in order to be more consistent with other boolean flags?
b-abel
commented
4 years ago I agreed with @thjaeckle that the events discussed here would not be very beneficial and wouldn't be used in Ditto because they don't allow consistent error handling. If the provisioning in Ditto fails after the Hono Device Registry has been run, external mechanisms would be necessary to resolve the inconsistency. Therefore I do not want to implement the events in Hono for the time being.
sophokles73
commented
4 years ago closed via #1706
We have got a new requirement to register devices when they attempt to connect to the IoT platform the first time. The provisioning setting should be done based on X.509 client certificate. What do you think regarding this feature?