Organization for user applications and source

[apullin]

Is your feature request related to a problem? Please describe.
There appears to be no guide for how user applications should be integrated into the existing source structure. The same issue for libraries. (Note: I may just be missing existing content, but I have dug around and haven't found anything)

Describe the solution you would like.
Ideally, there would be or template or checklist or guide to follow for how to remove the "demos" from the source tree, replace them with a single user application or bundle of applications, and how to reconcile this with the build system configuration.

For my application, I only care about the CMake system, as I am using CLion + existing CMake configs. Users of other IDE's may want similar guides for their various environments, especially since some of them require far more manual tinkering that others.

The starting point here would be either this github repo as-cloned, or the ZIPed packages that are delivered with the "FreeRTOS Device Software" package generator via the AWS console.

Additional context
The intent and function of the current source organization are not entirely clear, e.g. multiple directories per board, boards associated with a "vendor" that is really an MCU/silicon vendor, etc. Unclear if library ports and support packages are application-specific, board-specific, vendor-specific, etc.

See the WICED SDK for an example: there, the build system there provides something like a "namespace" (by path) and a clear structure for how end users can put their application code into place and build it with/on the SDK package.

[sarenameas]

Hello @apullin ,

I agree it is a bit unclear, but please allow me to try and clear things up a bit.

For the current version in amazon-freertos, I would reccommend consumsuming by taking the files you need into your project. We have our own demo running system under the demos/ folder in order for us to choose between demos without repeating some code, but we do not expect customers to take this into their projects. They are merely examples of how to use the API.

For the comment "CLion + existing CMake configs" do you mean to say you are using our CMake configs or your own?

Let's take the MQTT library for example:

1. Take all of the files in libraries/csdk/mqtt
2. Take all of the dependencies listed here: https://docs.aws.amazon.com/freertos/latest/lib-ref/c-sdk/mqtt/index.html#mqtt_dependencies these files are found under libraries\c_sdk\standard\common
3. All the current CSDK libraries in amazon-freertos have a dependency on headers in libraries\abstractions\platform\include\platform and you will to take these headers and writes imlemention for your system
4. Create an iot_config.h and put the directory in your build's include path.
5. The doxygen docs hosted publicly will let you know about particular MQTT configurations: https://docs.aws.amazon.com/freertos/latest/lib-ref/c-sdk/mqtt/mqtt_config.html
6. All dependencies will need to be initialized so be sure to take libraries\c_sdk\standard\common\include\iot_init.h and and libraries\c_sdk\standard\common\iot_init.c. The demos show you when to use this function, in demos\demo_runner\iot_demo_freertos.c. All your networking libraries are initialized before using the library.

I know there are a lot of steps and dependencies for the libraries in amazon-freetos and we have been working towards a simpler model with zero dependencies for the libraries that you are interested in. Our latest release of the MQTT library does not have any dependencies other than C headers: https://github.com/FreeRTOS/coreMQTT/releases/tag/v1.0.0 . This library has it's demos in https://github.com/aws/aws-iot-device-sdk-embedded-C/releases/tag/202009.00 .

Please let me know if this helps.

I will bring up documentation and steps improvements with our team. Thank you for bringing this issue up.

[apullin]

CLion + existing CMake configs

I am just directly using the CMake configs provided in the repo. CLion uses Cmake as it's build system under the hood, and has that ability where the root CMakeLists.txt can be opened "as a project", and it will parse out all the enumerated targets, aws_demos, afr_crypto, etc. It's pretty great!

I would reccommend consumsuming by taking the files you need into your project.

It is still a little unclear if there is a prescribed or expected pattern to be followed to adopt this whole set of libraries.

Should an embedded application have its own abrtirary structure and build system, and then keep a whole copy of this repo as a submodule or wholly contained in a larger source structure? Should an application live "inside" this repo, adopting the given source layout, and containing all the applicaiton-specific code?

It does seem like the latter, given that core pieces like the BSP and board/mission specific implementation of library support is already positioned inside the repo and on a path that follows namespace-like path, e.g. vendors/st/stm32l475_discovery.

If someone was building a new IoT device and wanted to build on stop of this amazon-freetos repo/SDK and all the pieces therein (TLS, OTA agents, MQTT lib, etc), but has a large source set to drive the device-specific implementation (e.g. displays, motors, etc), is there a best-practices to be followed? Should users start from an empty IDE project and manually duplicate all the pertinent settings from the demo IDE projects? Or should users start from the bundled demo IDE project, and extend as needed? The idea of manually recreating an whole Eclipse project setting-by-setting is ... 😬 worrying.

For contrast, we can compare to the output of another project-generating tool, like STM32CubeMX. For that tool, FreeRTOS (kernel only) is bundled in as a "Middlewares" component alongside a number of other libraries, a template for the entry point to the user application is generated, and an IDE project file or Makefile for the whole output is made. From there, there is a prescribed pattern to follow for adding the user's code, modules, libraries, etc.

A big part of the intent of this question is that I would like to adopt some organizational layout whereby I can upgrade to new releases of the amazon-freertos source set (this repo, kernel+libs+agents+etc) in one shot, by dropping it in place around or next to the other pieces of my project's source.

Currently, I have a large-ish project that uses a much older version (1.2.3) of the amazon-freertos libraries, and thus a totally different layout and a custom-make build setup (one big eclipse project).

[leegeth]

@apullin

Thanks for your patience and apologize for the delay in getting back to you. Please find my responses below.

Should an embedded application have its own abrtirary structure and build system, and then keep a whole copy of this repo as a submodule or wholly contained in a larger source structure? Should an application live "inside" this repo, adopting the given source layout, and containing all the applicaiton-specific code? It does seem like the latter, given that core pieces like the BSP and board/mission specific implementation of library support is already positioned inside the repo and on a path that follows namespace-like path, e.g. vendors/st/stm32l475_discovery.

The existing demos in the repository follow the latter style as you have correctly pointed out. However, unlike the demos in the repository, we have a documentation for ESP32 application to consume this repository as a submodule. Please check it out and let us know if it helps.

If someone was building a new IoT device and wanted to build on stop of this amazon-freetos repo/SDK and all the pieces therein (TLS, OTA agents, MQTT lib, etc), but has a large source set to drive the device-specific implementation (e.g. displays, motors, etc), is there a best-practices to be followed? Should users start from an empty IDE project and manually duplicate all the pertinent settings from the demo IDE projects? Or should users start from the bundled demo IDE project, and extend as needed?

Unfortunately, currently there is no best practices documented about the structure of an application and project files. This is a good suggestion and we will consider addressing it. Thanks for bringing this up.

A big part of the intent of this question is that I would like to adopt some organizational layout whereby I can upgrade to new releases of the amazon-freertos source set (this repo, kernel+libs+agents+etc) in one shot, by dropping it in place around or next to the other pieces of my project's source.

Currently, I have a large-ish project that uses a much older version (1.2.3) of the amazon-freertos libraries, and thus a totally different layout and a custom-make build setup (one big eclipse project).

Sorry to hear about the issue you are facing for updating the project to the latest version of FreeRTOS. There is a plan for a Long Term Support(LTS) release for FreeRTOS and the details can be found in this link. Please check it out and see if updating to LTS release may help to solve your problem of keeping up with further updates. Please pay attention to our announcements for further updates on the LTS release.

Hope these information helps with the problems that you are facing.

[dan4thewin]

We have a backlog item to revisit the documentation and organization, so I will close this issue. Feel free to reopen this or create a new issue. Thank you.