Suggestion: allow use of git submodules for libraries

[drewcassidy]

Supporting the use of git submodules could help with the use of mbed with libraries, since it would show an easy link to the submodule repository at the referenced commit, and would remove the clutter of the .lib files that mbed uses currently. .lib files would be kept for backwards compatability and use with other SCMs.

currently, attempting to use git submodules in addition to the .lib files causes mbed to throw errors about not being able to find a valid repository

[theotherjimmy]

@drewcassidy Are you just suggesting that we pass the --recursive option to clone? If so, I think that would be a reasonable change.

[theotherjimmy]

@screamerbg Do you have thoughts on this?

[drewcassidy]

@drewcassidy Are you just suggesting that we pass the --recursive option to clone? If so, I think that would be a reasonable change.

that, plus allowing submodules in place of the folder + .lib files. currently mbed throws errors when both are present, and without the .lib files it wont update. running commands like mbed update should also call the appropriate git submodule commands

[0Grit]

@screamerbg I believe using submodules was another reason our projects were exploding

[screamerbg]

@drewcassidy @loverdeg-ep A core Mbed CLI requirement is support Mercurial and Git, and potentially other SCMs. Mbed CLI has to handle these seamlessly, regardless of how they are used, including use cases where SCMs are mixed in a code tree - e.g. a program on os.mbed.com uses Mercurial, pulls in Mercurial libraries, but also uses Mbed OS from github.

Git does support submodules, and it comes with some advantages and disadvantages:

Pros:

• It's Git's own feature
• Git submodules are well integrated with Github and other Git hosting websites

Cons:

• Git submodules behavior changed over time (git 1.9.5 vs 2.2 vs latest) about how submobiles are being handled, what's handled automatically, including errors (see below).
• Git submodule hash references are living inside Git's index, making it rather cumbersome to remove/add/change/update submodules on the fly, including in program trees with multiple dependencies, e.g. https://github.com/ARMmbed/mbed-os-example-client/
• Git handles as critical errors cases when a submodule (or a submodule of submodule) is missing or has been rewritten, and therefore the hash reference is no longer valid. This makes it rather (again) cumbersome to modify submodules on the fly and recover from any error in submodules (and sub-submodules).

For the reasons above we took the design choice to use unified handling sub-repositories between Mercurial and Git (and other repos in the future) via .lib files that contain all the needed information to reproduce a code tree, and also add certain options and help/tips to recover from a scenario when a sub-repository has been modified or destroyed.

Is it possible to use Mbed CLI .lib references and git submodules? Absolutely. You have ensure the following:

• The Mbed CLI .lib references/files are synchronized before you make a commit, using mbed sync
• Always use mbed sync after you use git submodule update as well
• Let Mbed CLI handle cloning/import and then apply git submodule, followed with mbed sync

Basically, it's all about synchronizing both git's submodule handling and Mbed CLI dependency handling (via .lib files) before introducing new changes.

Let me know if this doesn't cover your use case.

[drewcassidy]

Ah, thank you for the explanation! It would be nice if that was a bit more automated in the CLI tools, but thats probably a problem for another issue

[theotherjimmy]

@drewcassidy I take it that this issue can be closed then? Is there something else that needs to be done here?

[screamerbg]

@drewcassidy I think that we should explain this better in the documentation. Including working with submodules as library references in parallel