Adding new core with own certificates

[abroekhuis]

Perhaps this is possible, and am I looking in the wrong places. But I would like to be able to setup multiple cores in a same group, and so that each core has an own certificate.

As far as I can tell the created scripts creates a core with a name similar to the group and with one fixed certificate. Is it possible to easily setup multiple cores?

[timmattison]

Sorry for the delay. Currently Greengrass groups can only contain a single core. I’m not sure if/when that will change.

In the meantime would you want to have more control over the naming? On Jan 14, 2020, 3:47 PM -0500, Alexander Broekhuis notifications@github.com, wrote:

Perhaps this is possible, and am I looking in the wrong places. But I would like to be able to setup multiple cores in a same group, and so that each core has an own certificate. As far as I can tell the created scripts creates a core with a name similar to the group and with one fixed certificate. Is it possible to easily setup multiple cores? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[abroekhuis]

Well, naming itself is not really the biggest issue, although it would be nice to have the possibility to specify a specific name. I guess the underlying issue here is the lack of individually controlling a core. In our case I would like to be able to control (eg) the shadow of one specific core. This could be a core specific name, but also other configurations. For example, currently I am playing with a setup where I use a Greengrass core with an attached Pi Camera. This core streams the video to a Kinesis Video Stream. The idea is to have a setup with multiple devices, where each stream needs to be processed separately. If there is only one core for a group, I'm either limited to a configuration for all devices using the same setup (core), or forced to create a new group for each specific instance. Since all instances are (code-wise) the same, this does not feel right to me. All together I am a bit confused about how this could be done with AWS IoT and Greengrass. Perhaps I have a lack of understanding how I should look at cores/groups and greengrass wrt these concepts. But I always assumed, based on prior similar (non-aws) systems, that a group can have multiple cores. And each core can have a deployment, either via the group (same deployment for all cores), or individually. And on top of that the possibility to control the core's shadow for core specific setup (configuration etc).

This issue also describes the problem I have: https://forums.aws.amazon.com/thread.jspa?messageID=862606&tstart=0

Note: when I mention a core, I mean an actual running instance, and not necessarily a core in AWS IoT.

[timmattison]

I’ve looked into creating something like a “group template” that might help. Create one config that is the template and then deploy multiple groups with the same configuration.

This still means that you need one group per device but you could share the setup across them all, each would have a distinct name and shadow, and you’d be able to update them all at once.

This would require three new features. 1) Create a group template mechanism, 2) allow a new or existing group to reuse the group template, and 3) allow updates to the group template to propagate to all dependent groups.

Would that address your needs? It sounds like you may want to do some individual group customization which may require some additional options though (but still possible). On Jan 23, 2020, 2:51 AM -0500, Alexander Broekhuis notifications@github.com, wrote:

Well, naming itself is not really the biggest issue, although it would be nice to have the possibility to specify a specific name. I guess the underlying issue here is the lack of individually controlling a core. In our case I would like to be able to control (eg) the shadow of one specific core. This could be a core specific name, but also other configurations. For example, currently I am playing with a setup where I use a Greengrass core with an attached Pi Camera. This core streams the video to a Kinesis Video Stream. The idea is to have a setup with multiple devices, where each stream needs to be processed separately. If there is only one core for a group, I'm either limited to a configuration for all devices using the same setup (core), or forced to create a new group for each specific instance. Since all instances are (code-wise) the same, this does not feel right to me. All together I am a bit confused about how this could be done with AWS IoT and Greengrass. Perhaps I have a lack of understanding how I should look at cores/groups and greengrass wrt these concepts. But I always assumed, based on prior similar (non-aws) systems, that a group can have multiple cores. And each core can have a deployment, either via the group (same deployment for all cores), or individually. And on top of that the possibility to control the core's shadow for core specific setup (configuration etc). — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[abroekhuis]

This already sounds like a great start! If this would be possible, I can probably already do a lot of what I would like to do. Extra customisations are not yet fully clear, so solving those in a possibly second (or third ;)) step makes sense to me.

Note: does this provide some useful input for a solution? https://github.com/awslabs/aws-greengrass-group-setup

[timmattison]

OK, this is not the first time I've gotten this request. :D I need to think about it a bit more. Reading the repo you linked made me think that I might want to also look into doing it with CloudFormation.

Essentially GGP could run in a mode where you create a "template", the template becomes a CloudFormation stack, for new groups you can reference the CloudFormation stack and it'll use everything defined there.

The challenge is that the CloudFormation stack won't update the dependent groups automatically by default. It's possible though that I could put something together with Lambda that would do that. Then you could operate on the CloudFormation stack and updates would roll down to the groups that reused it.

It's a lot of work but could be well worth it.

Related, do you have any code for the Pi camera + KVS that could go in the example functions repo? Many people ask for a sample of that and I just don't have one. I can help get it in the GGP format if that helps.

[abroekhuis]

Interesting to see what comes out of this :). I know about the group setup and cloudformation, but have not yet used it in any kind of way.

Regarding the Pi camera, I have a working example based on the CDDBaseline which uses the kinesis gstreamer plugin and gstreamer-java. If a CDD example is good enough, I can create a PR I think. I need to clean up the code and take out specific (own) parts that I cannot share.

[timmattison]

Based on CDD? Even better! Please submit a PR and include any notes about things you'd like to clean up. I'd love to take a look and help out. Then I can finally test my own Pi camera...

[abroekhuis]

Already updated the provisioner as well. I've created a PR with an example for Kinesis. I have not tested the code in the PR, since I don't have a Pi with me at the moment, so any feedback is welcome!

-

Will do, just a few question. My current setup uses an older commit of the provisioner and aws-lambda functions. WIth my new clone (and as such head) of the lambda functions I now can't run ggp, it is missing REGION, and if set by hand to a region, it complains about "conf.core.roleName". Is something changed so that I need to specify those myself, or should I update the provisioner as well? And if so, any risk of breaking something of my current (working) setup?

[timmattison]

I haven’t seen this before. Can you attach some logs and open a new issue? Generally I try to avoid breaking variables like that. Often it is just that you need the latest function and deployment defaults but if that’s not the case I’ll help you sort it out. On Jan 23, 2020, 9:19 AM -0500, Alexander Broekhuis notifications@github.com, wrote:

Will do, just a few question. My current setup uses an older commit of the provisioner and aws-lambda functions. WIth my new clone (and as such head) of the lambda functions I now can't run ggp, it is missing REGION, and if set by hand to a region, it complains about "conf.core.roleName". Is something changed so that I need to specify those myself, or should I update the provisioner as well? And if so, any risk of breaking something of my current (working) setup? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[abroekhuis]

I was using the head of the examples, with an older checkout of the provisioner. Updated that one as well to head, and it worked without any problems. I don't think an issue is needed.

[timmattison]

👍 thanks for following up. I’m always trying to make sure I don’t break old installs or if I’m forced to I try to make the error messages indicate what’s wrong. If it comes up again and there’s a clearer way to convey how to fix it I’ll update the error messages with your input. On Jan 25, 2020, 2:30 PM -0500, Alexander Broekhuis notifications@github.com, wrote:

I was using the head of the examples, with an older checkout of the provisioner. Updated that one as well to head, and it worked without any problems. I don't think an issue is needed. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[timmattison]

Back to your original question regarding communicating with multiple cores. You can set up Lambda functions to listen on topics that contain the IoT thing name of each core. Here is one example:

https://github.com/aws-samples/aws-greengrass-lambda-functions/blob/8fb0291ec1192f17f9a0d27cfc50c59d93aefdd9/functions/CDDDMIJava/function.conf#L10

You can either use MQTT messages alone to send commands to a function or you can use the core's shadow and route the shadow messages to the functions in a similar way. Would that achieve what you were looking for?

[timmattison]

Should we keep this open or create new issues to discuss the specifics of the different items?