How will collaboration work?

[matthijskooijman]

Great to see work is done to bring an LMIC-based stack back into development. I'm curious how you are planning to structure the development of this stack?

IBM used a fairly closed model, where development happened on an internal repository and only releases were published as zip files, without much interaction with or opportunity for contributions from the user community. Here, I see you're using github and have a public forum, which is great. However, I do see that the commit history is completely squashed, with just one commit per release (implying that there is an internal working repository, or that only one or two people are working on the code).

What are the plans for this codebase? Are you expecting to work with the community, accepting pullrequests, publishing development code in separate commits when they are done (rather than only releases), maybe even share the maintainance with community members?

For some background on my question, I created an Arduino port of LMIC years ago, which has become the defacto standard stack to use on (small) Arduino-based platforms. After the initial port, I have been hesitant to do a lot of further development on it, since I did not want the arduino port to deviate too much from the original ARM-based port. I had plans to unify both again (by doing development in the main version and allow automatically creating the Arduino version from that using a script), but never really gotten around to it (and I also held off on it recently when the lorawan minimouse stack and this basicmac stack were expected to be released).

Now that basicmac is available, it looks like a great base to port to Arduino and I'd gladly see it become the new center of development for small LoRaWAN stacks, replacing LMIC at some point. However, I do not want to do a one-off port to Arduino (forking the repo and fragmenting development). But if this repository is not intended to facilitate community supported development, then it might not be so suitable to serve as such a center of development. Or if you're not planning to support (or even accept pullrequests to support) smaller (8-bit) platforms and project structures (e.g. Arduino does not use Makefiles but requires a specific library format).

[VojislavM]

@matthijskooijman you make a really good point. I think you will get a more prompt response if you post this in their development forum. I already got some answers there and it seems this is still in early stage and not relay user-friendly. P.S. If you want to give it a try have in mind answer to my question in this forum post

[mkuyper]

Back in the IBM days, taking in contributions was simply not possible due to various reasons. Going forward, we would like to take a more open approach. We are thus actively working on defining a contribution policy and process. Once that is complete and has all the necessary approvals, we will announce it here.

As you have correctly guessed, stack development currently happens in a separate repository. The code for Basic MAC is extracted from there on every release. This makes accepting and merging contributions difficult at this time (at least in the "git way"), but was the fastest course to get the project out into the open without disrupting internal projects too much. I expect this approach to change once we have defined an official contribution process.

Adding support for additional platforms is a great example for an area that we will be looking for contributions. As for 8-bit platforms, it's entirely possible that we have have neglected conceptual support for those over the last few years due to the pervasiveness of ARM and our focus on the STM32L0. But the portability of the LMiC core is still a main design principle, and if improvements are necessary to achieve this, we will certainly take those into consideration.

Not least, I am glad to see enthusiasm around this project! It has always been fun to see what people have been doing with LMiC over the years, and it's great to be back out there!

[matthijskooijman]

@mkuyper, great to hear you're planning to open up the development, nice :-) I have quite some experience with running open-source projects (both from the community as well as the company perspective), so if you want some feedback on your policy and process plans, feel free to drop me a line.

In the meanwhile, I had a first stab at porting basicmac to (AVR-based) Arduino. I have something that compiles and runs. It's still full of quick hacks (especially wrt to configuration and debug printing) and TX does not seem to work yet (more investigation needed on my end), but overall it seems an Arduino port should be feasible. I did notice that BasicMAC has become quite a bit bigger than LMIC, so it might be a challenge to get it running on the 32K flash 328p again.

Along the way I noticed quite some small improvements to the core and HAL APIs for problems that I have been meaning to fix for a long time (such as distinguishing between exact and approximate sleeping, or letting the IRQ handler do the timestamping of events, etc.), which is great.

I know this is a tricky question, but I'm going to ask it anyway: Do you have any ETA on the contribution policy? I already have some patches and bunch of ideas that I could start contributing, whenever you're ready for them :-)

[lucasmaziero]

@matthijskooijman @mkuyper The porting basicmac to (AVR-based) Arduino would be great, I'm waiting to contribute and test. :smile:

[FlorianLudwig]

@mkuyper Hi, any updates on opening up the development process? or maybe a timeline for it?

looking forward to it.

[matthijskooijman]

Any updates here?

I've found @lacunaspace willing to sponsor work on a BasicMac port for Arduino. I've got a basic version working somewhat, so I might be producing proper patches in the near future to contribute back to BasicMAC if possible (since I'm not so much interested in keeping a modified fork and dealing with the conflict fallout of merging upstream changes developed internally and published as single-commit-releases...).

[lnlp]

I would prefer that an Arduino port will not focus mainly on AVR but also target Arduino on SAM, ESP32, STM32 and nRF52.

[matthijskooijman]

I would prefer that an Arduino port will not focus mainly on AVR but also target Arduino on SAM, ESP32, STM32 and nRF52.

The target I'm planning to use this on is SAMD. Ideally, I'd also like things to work on AVR, but it seems that the smaller ones (e.g. 328p) might be too small to fit properly. Also, I hope most of the port can just be architecture-neutral and use portable Arduino API functions, though some target-specific optimizations likely make sense.

[matthijskooijman]

I just saw that Semtech has deprecated BasicMAC and will no longer develop it: 02ed42578a320445da4957c652e78145482112a6

So I guess that answers this question: No collaboration with Semtech anymore.

I'm still planning to work on an Arduino port, since Lacuna needs 1262 support, maybe also some more generic development (but between Semtech deprecating BasicMac, and the LMIC development happening at https://github.com/mcci-catena/arduino-lmic/ I am not sure if much development should happen on BasicMac still).

[matthijskooijman]

I've just cleaned up some of my work on the Arduino port. It is still a bit rough and prone to change, but it is usable as it is now, so suitable to share with others. See the README for some more info: https://github.com/LacunaSpace/basicmac/tree/work-in-progress

[nortismo]

@mkuyper Is this going to be continued or not? The whole thing is quite confusing now. If you observe the commits of the README.md in February, a message about cancellation of the development was added, then changed and finally deleted again. https://github.com/lorabasics/basicmac/commits/master/README.md

Should we now use the LoRaMac-node stack or will there be another release of basicmac with a proper collaboration strategy?

[lnlp]

LMIC and Basic MAC will be superseded by Soft Modem, on which Semtech is currently working. But it may take quite some time before it will be available.

And it will probably not be a viable option for Arduino any soon.

[nestorayuso]

but Soft Modem won't be open source.

[lnlp]

If I understood correctly the client will be open sourced but not the (backend) services and IIRC the service specifications will be open (not the implementation) and for the client implementation ARM will be used as reference platform.

[matthijskooijman]

I've published a more complete version of the Arduino port here, along with a number of other improvements: https://github.com/LacunaSpace/basicmac/ (though unlike with LMIC, the code is not limited to Arduino, the original build system should still work). I suggest anyone still interested in BasicMAC to have a look there.

[mkuyper]

A big thanks to the LMiC community! For those that are interested, I'm also continuing development of the stack here: https://github.com/mkuyper/basicmac I am putting a large focus on the simulation and test environment, but also on continued compliance, improving the run-time and portable peripherals layer, and hopefully adding more target platforms.

[lnlp]

@mkuyper "I'm also continuing development of the stack" How does your development relate or compare to https://github.com/LacunaSpace/basicmac ?

[matthijskooijman]

I'd have to doublecheck with Lacuna, but I'm pretty sure that we will be happy to collaborate further, in the form of giving @mkuyper access to our repo at Lacuna, or maybe moving development to some other shared organisation.

In any case, I would strongly suggest to base any further development on the Lacuna version, since it has seen some significant improvements already, but looking at @mkuyper's repo, it seems he's already done significant work (at least significant number of commits, haven't looked inside yet), so maybe this is already too late.

In any case, I'd very much want to avoid having multiple independently developed forks of this library. That happened with LMIC and didn't really help anyone, I'm afraid (Admittedly, I have been a large factor in letting that happen, ironically because I didn't want to develop my Arduino-specific port too much in fear of deviating from the non-Arduino-specific "upstream" version, but that backfired and caused more forks, unfortunately).

[mkuyper]

@lnlp My continued work on LMiC/Basic MAC is independent from the work that Matthijs has been doing over the years, including the Arduino fork over at Lacuna. I'm sure that if the political circumstances at the organizations that own the IP had been different, we would have been able to collaborate a lot more closely over the entire LMiC history.

@matthijskooijman I am not really considering my ongoing work to be a fork, but rather a continuation of LMiC and Basic MAC. As such, I am not planning on actively upstreaming my changes to other repositories. That said, I am of course aware of the work you have been putting in over at the Lacuna repo -- and I'm glad to see that happen! I am very much open to collaboration in any form, and unlike before, I actually can and will accept PRs submitted to my repo; the issue tracker is also enabled.

As I said in my previous post, I have specific areas that I am currently planning on directing my focus to. On top of that, there is a rather large number of production devices deployed out in the field that run on some version or other of a stack from the "core lineage" (LMiC/TrackMAC/Basic MAC). The ability to easily port firmware for those devices to an up-to-date Basic MAC stack is also very high on my list at the moment.

[smtc-bot]

Thank you for your inquiry.

Customers are encouraged to submit technical questions via our dedicated support portal at https://semtech.force.com/ldp/ldp_support.

We invite all users to visit the LoRa Developer Portal Forum at https://forum.lora-developers.semtech.com and to join the thriving LoRa development community!