🏗 Leshan 2.0 : LWM2M 1.1 🚧

[sbernard31]

Leshan v2.0.0 (master branch) is currently in development and aims to support LWM2M 1.1 (or at least some part of it).

• Core spec
• Transport spec

You can have a look to currently supported or in development features or current roadmap.

If you are interested in LWM2M 1.1 support in Leshan, we strongly advice to :

• use/test this milestone release,
• report us any bug or feebacks,
• share what part of LWM2M 1.1 missing you more here :point_down: ,
• contribute code.

This is the best way to get a final 2.0.0 release which fit your needs.

(If you need stable API and are fine with LWM2M v1.0, you should use Leshan v1.x)

[wenzheng]

Hi Bernard

I would be very interested in the newly introduced Non-IP binding for NB-IOT, where we can use 3GPP SCEF to carry LwM2M traffic, though the spec is released, however a reference design regarding how to integrate with T8 interface would be a good start for us to contribute

BR/Vincent

[rikard-sics]

As a part of a joint project with Ericsson about contributing OSCORE to Californium we at RISE have also been working on using OSCORE in Leshan, as OSCORE is part of LWM2M 1.1.

Currently we have code produced for enabling communication over OSCORE between the Leshan server, bootstrap server and client. In addition we have added support to bootstrap a client so it receives information to generate an OSCORE context for communication with the Leshan server. We would be interested in contributing this code to Leshan.

[sbernard31]

@wenzheng, sry for the delay response, I have no experience about NB-iot but I suppose the first step must be to have a way to make CoAP and DTLS working over NB-IOT in Java. In a californium context that means create new Connector(s) for NB-IOT.

@rikard-sics, That's a good news to know we could get a new contribution from RISE. :)

But I ask myself how we could handle this because as explained we plan to firstly release a 1.0.0 of Leshan which supports LWM2M v1.0.x. (so without 1.1 feature)

Maybe we could integrate this in another branch of Leshan which would aim to support LWM2M v1.1. I suppose the contribution is pretty big, right? Did you plan to continue to actively maintain this part of the code after your contribution?

[sbernard31]

It seems there is some interests about Observe-Composite and Cancel Observation-Composite at https://github.com/eclipse/californium/issues/881.

[rikard-sics]

@rikard-sics, That's a good news to know we could get a new contribution from RISE. :)

But I ask myself how we could handle this because as explained we plan to firstly release a 1.0.0 of Leshan which supports LWM2M v1.0.x. (so without 1.1 feature)

Maybe we could integrate this in another branch of Leshan which would aim to support LWM2M v1.1.

Yes it could make sense to put it in a separate LWM2M 1.1 branch since OSCORE is not part of LWM2M 1.0.x

I suppose the contribution is pretty big, right?

Actually the contribution is not that big since we made it very targeted on providing OSCORE support. Since the actual code for OSCORE communication is in Californium, for Leshan it was more about adding code to use it.

Did you plan to continue to actively maintain this part of the code after your contribution?

This work is part of a joint project with Ericsson and for the future we have the ambition to continue to use and support this code.

[yemkay]

We at HPE are interested in developing 1.1 features like CBOR, Composite Read,Write,Observe., Bootstrap trigger and Non-IP transport. We will start a new branch and contribute in coming days.

[sgoyaldel]

@yemkay : i would like to request you, please share further information once 1.1 feature available to use

[sbernard31]

@yemkay, good to know :) ! There is already some contribution about CBOR :#656 . If you are interested by this subject. (It's the very beginning of the contribution)

[rikard-sics]

@rikard-sics, That's a good news to know we could get a new contribution from RISE. :)

Just to come back to this regarding OSCORE in Leshan. Would it be fine if we made a pull request with the code we currently have putting it into a separate branch of Leshan? I suppose such a branch would have to be created first?

[sbernard31]

@rikard-sics I created a 2.0.x branch and its jenkins job.

I suppose you need OSCORE feature from Californium, so the first step will be to wait for a Californium v2.0.0M14, right ? Please if you can try to make several simple, little, self-sufficient PRs. This could help review and integration. :pray:

[rikard-sics]

@rikard-sics I created a 2.0.x branch and its jenkins job.

I suppose you need OSCORE feature from Californium, so the first step will be to wait for a Californium v2.0.0M14, right ? Please if you can try to make several simple, little, self-sufficient PRs. This could help review and integration.

Thanks a lot for creating the branch.

I see that Californium 2.0.0-M14 has now been released. I will proceed with making smaller pull requests with parts of our code as you requested.

[sbernard31]

@rikard-sics , just to let you know I plan to create a PR to integrate Californium 2.0.0-M14 this afternoon.

[sbernard31]

@rikard-sics branch 2.0.x is ready with cf 2.0.0-M14.

[sbernard31]

@rikard-sics oops not so ready, build failed because of new legal bundle in californium. I will try to fix it.

[sbernard31]

@rikard-sics this time it's really OK.

[rikard-sics]

@rikard-sics this time it's really OK.

Thanks for setting up the branch. I will proceed with the pull requests in steps as you mentioned.

[tuve]

Is there any estimated time frame as to when lwm2m 1.1 is supported?

[sbernard31]

Is there any estimated time frame as to when lwm2m 1.1 is supported?

Question : what means "LWM2M 1.1 is supported" ? Do you mean full support of all defined features.. Even for 1.0 we didn't cover 100% of the specification. So you should ask yourself which part of the specification you need and share this with us. This could help to define priorities !

Anyway, I will not be able to give you any date. This is just the beginning for 1.1 implementations. Some contributors just started to work on this. Personally, I'm focus on the version 1.0.x. and maybe I will be able to start to develop some 1.1 features at the end of the year.

All the work about LWM2M 1.1 is available in branch 2.0.x.

[Kenny-SZ]

Is OSCORE available in branch 2.0 now ? Thanks.

[rikard-sics]

Is OSCORE available in branch 2.0 now ? Thanks.

There is an open pull request to add support for OSCORE communication between a Leshan client and server to the 2.0.x branch here: https://github.com/eclipse/leshan/pull/718 More information can be found in that PR.

[Kenny-SZ]

@sbernard31 , @rikard-sics , do you know the progress to implement "Send" Operation defined in LwM2M 1.1? Has it been implemented, or is someone working on it now?

Thanks. Kenny

[sbernard31]

@Kenny-SZ, as far as I know nobody is working on this.

[Kenny-SZ]

@sbernard31, thanks for your response. In my opinion, Send operation is a very important functionality, for example: when there is a fault in a LwM2M client, it should send it out immediately in order for operators to be aware of this fault, instead of waitting for obervation from the LwM2M server. Please prioritize it if possible, thanks a lot.

B.R. Kenny

[sbernard31]

@Kenny-SZ we agree that send is really good feature because observation has too many drawback. (https://github.com/OpenMobileAlliance/OMA_LwM2M_for_Developers/issues/171)

[Kenny-SZ]

Hi @sbernard31 & All, does anyone know the progress to support LwM2M 1.1? Thanks.

[sbernard31]

Personnally, I'm still focus on the 1.0.0 release. We hope we will be able to release it soon (maybe during spring 2020) and then we would be able to start to work on LWM2M 1.1 support.

There is already some contribution from community :

• @rikard-sics is working on OSCORE support.
• @cavenaghi9 started to work on senML json & cbor content format.

[henris42]

Hi, just curious on how 1.1 topic is proceeding? I mean standard is already moving to 1.2, so I think some kind of 1.1 subset should be prioritized. For me the most important feature would be SEND, as that would enable use cases leveraging LwM2M for real-time control and alarms, like what is currently doable with MQTT (more or less successfully)

[sbernard31]

@henris42, I agree with you (see https://github.com/eclipse/leshan/issues/563#issuecomment-523350573)

The only missing part is time to do it :-)

[dachaac]

@henris42, I agree with you (see #563 (comment))

The only missing part is time to do it :-)

@sbernard31: One problems is to have some kind of incentive to contribute.

Observation: most activity seem to happen in master branch.

You have also a policy not to take any non 1.0.2 changes to master.

As an example you have 2.0.x branch which is over year old and doesn't really invite contributing. There are several other work branches that has "expired" too.

Where as in my view only LWM2M 1.1 makes LWM2M actually usable for deployments as that has better EST support than 1.0.2. There are other features like send that was mentioned also in here.

Sticking to LWM2M 1.0.2 is making current Leshan releases uninteresting.

Currently there does not seem to be good path to contribute changes with target of having LWM2M 1.1 features supported in Leshan releases.

Would you be willing to start taking 1.1 (or later) changes to your master branch? If master branch would be following latest released standard then it could bring incentive for others to contribute.

Some changes also require a bit larger changes making this API protection mechanism a bit harder to tackle. Used quite a lot of time to fight that when I was testing some certificate usage improvements with Leshan code base.

[sbernard31]

most activity seem to happen in master branch You have also a policy not to take any non 1.0.2 changes to master.You have also a policy not to take any non 1.0.2 changes to master.

This is correct because main contributor was interested in a 1.x release of Leshan which supports LWM2M 1.0.x We started to develop Leshan long time ago and we didn't want to shift from LWM2M 1.0.x to LWM2M 1.1 without providing a polished release with stable API. Now Leshan 1.0.x is released and we plan a last 1.1 version for LWM2M 1.0.x, then we will mainly focus on LWM2M 1.1 (so Leshan 2.0)

As an example you have 2.0.x branch which is over year old and doesn't really invite contributing. There are several other work branches that has "expired" too.

We created this branch to help some contributor to start to work in Leshan 2.0 which will supports LWM2M 1.1 waiting we finish the work on LWM2M 1.0.x but contributors disappear without saying a word. Currently the only active one (LWM2M 1.1) works on OSCORE in its own branch and I take time to review it works. (as I tried to do for SenML before contributor vanished)

Sticking to LWM2M 1.0.2 is making current Leshan releases uninteresting.

I hope you mean "uninteresting for me" or this sounds a bit peremptory.

Currently there does not seem to be good path to contribute changes with target of having LWM2M 1.1 features supported in Leshan releases.

Because as explained, there was no more contributors who need that. But if needed I can create it again. I mean an up to date 2.0.0 branch.

Some changes also require a bit larger changes making this API protection mechanism a bit harder to tackle. Used quite a lot of time to fight that when I was testing some certificate usage improvements with Leshan code base.

We are using semantic versioning and as explained above we aim a version 2.0 of Leshan for LWM2M 1.1 so API can be break so don't worry about that.

[henris42]

+1 for EST. I think that would be a key component, if there is any willingness to make this a platform that can be used on real production installations. I recommend only real certificate-based provisioning processes to my consulting clients. But, I'm not really talking about CoAP EST, I'm not aware of any real implementations of that. But forwarding certificate-based bootstrap to a normal EST service (like OpenXPKI) would be really good concept.

I do agree with @dachaac, Leshan needs a true roadmap to LwM2M 1.2 (via 1.1 naturally)! You cannot attract contributors with 1.0 stuff, at least in my point of view it simply does not fulfill any of my current use cases.

[dachaac]

most activity seem to happen in master branch You have also a policy not to take any non 1.0.2 changes to master.You have also a policy not to take any non 1.0.2 changes to master.

This is correct because main contributor was interested in a 1.x release of Leshan which supports LWM2M 1.0.x We started to develop Leshan long time ago and we didn't want to shift from LWM2M 1.0.x to LWM2M 1.1 without providing a polished release with stable API. Now Leshan 1.0.x is released and we plan a last 1.1 version for LWM2M 1.0.x, then we will mainly focus on LWM2M 1.1 (so Leshan 2.0)

Ok.

As an example you have 2.0.x branch which is over year old and doesn't really invite contributing. There are several other work branches that has "expired" too.

We created this branch to help some contributor to start to work in Leshan 2.0 which will supports LWM2M 1.1 waiting we finish the work on LWM2M 1.0.x but contributors disappear without saying a word. Currently the only active one (LWM2M 1.1) works on OSCORE in its own branch and I take time to review it works. (as I tried to do for SenML before contributor vanished)

Problem with these temporary branches is that they do not guarantee that the work gets integrated or maintained. Eg. it is more or less same thing as having a personal fork. Having a single feature development in branch is perfectly fine until one can do PR.

I kinda understand the vanishing part here.

Let me bring some views on this -- feel free to have a different view.

If you are working in a company and get paid to do stuff that brings value for the company then contributing in open source project that is evolving and you get added maintenance bonus as there are others also contributing and that project brings releases then it is much easier to get time (and money) allocation for such activity.

Now if one sees that this might be a good idea to contribute and you do that for in example this 2.0.x branch and it takes a year and there is no release from that branch -- it does not anymore bring added value for business. There is not even guarantee that the contributed time (and money) would bring in future added value for any entity. -> less chance to get contributions from business developers.

Now if you are developing just for fun or to learn new tech then you are more or less willing to use part of your (free) time to develop something. Then in this scenario if you use your time then seeing your name in list of committers for particular release of the software gives nice warm feeling that you did nice thing for the project. If it is in temporary branch then you do not get this nice feeling as it never gets released. These kind of developers often want to target leading edge of things.

Then we of course have die-hard developers category where you own the project and its your baby and want it to excel. :)

Sticking to LWM2M 1.0.2 is making current Leshan releases uninteresting.

I hope you mean "uninteresting for me" or this sounds a bit peremptory.

For us but yes. After using some time with Leshan we finally decided to go for commercial option as that already provided LWM2M 1.1 features that we felt that our deployment model needed. And at the time Leshan seemed to be slowly moving and was not ready to accept the changes in newer specifications that we were looking for.

Currently there does not seem to be good path to contribute changes with target of having LWM2M 1.1 features supported in Leshan releases.

Because as explained, there was no more contributors who need that. But if needed I can create it again. I mean an up to date 2.0.0 branch.

You are free to use development model you see best fit -- but how would you see following model:

"Master" branch would always be the main development focus trying to match latest released LWM2M standard. Eg. being leading edge of things.

When master has good level of features then one would make a periodical release. Even one commit could be enough to do the release if that brings something new or bugfix or something like that. This gives a feeling that project is maintained as there are new releases coming. This factor is very important when one is choosing whether to use some library or software.

When one feels that now certain milestone has been completed then master branch could be branched for maintenance/long term support release branch (should there be a need for one). Master branch would continue the original goal of keeping up to date and welcoming contributions. Let's assume one would have created "1.x" branch to help in providing businesses that need stable ground a more "long term support release" option. As those long term release branches only benefit users that need such feature they needs sponsorship (as form of time allocation from developers) -- so if you have one good for you.

Then later when time goes on it might be a time to make a "2.x" LTS branch from "master".

This kind of model gives a project forth going feeling that stuff is happening and welcomes a bit more external contributors to contribute. And if someone sticks to LTS branches then those entities are more likely to give support for those branches (wouldn't hold my breath thou...). Trick is then to get LTS contributions also back to "master" branch to keep that it in "leading edge". (many projects do require commit first to master and then the LTS branches)

I am willing to work and provide improvements in areas of certificates and EST support but I don't want that my code ends up in non-maintained temporary branch (or personal fork). Might still do that for other reasons but then I acknowledge that it is likely that I don't even try to contribute back and just do it in quick and dirty model.

I don't want to cause you extra work on re-creating 2.0.x branch so that you might receive some contributions for that -- I don't personally see that model viable contributing method. I feel better to have volatile working branch for a feature and then just rebasing periodically with master to receive new changes from there.

But in any case I would like to know expectancy if those kinda of changes would be accepted and when that would be.

Which ever model you go for -- please do tell your expectancy/model for possible contributors as those might determine whether to spend time on project or not.

Some changes also require a bit larger changes making this API protection mechanism a bit harder to tackle. Used quite a lot of time to fight that when I was testing some certificate usage improvements with Leshan code base.

We are using semantic versioning and as explained above we aim a version 2.0 of Leshan for LWM2M 1.1 so API can be break so don't worry about that.

Ok.

[sbernard31]

I'm not sure where you try to go with this long post. My feeling is that :

1. you really really want that Leshan implements LWM2M 1.1.
2. you sounds not satisfied by the way the project is managed.
3. you hesitate to contribute because you are afraid that Leshan will never release a version with LWM2M 1.1 features
4. you consider master as the only durable branch

1) 2) We also want to implement LWM2M 1.1 but until now we have higher priority. We started to work on Leshan when LWM2M 1.0 was only a draft and visibly this is faster for OMA to write specification than writing working piece of code for us. We wanted to release a stable version of Leshan which supports LWM2M 1.0 before to target LWM2M 1.1. At Sierra we are using LWM2M 1.0 with hundreds of thousands devices in production.

Now we are focusing on Leshan 1.1 which is currently the master branch. After that we could probably be able to work on Leshan 2.0 focusing LWM2M 1.1 and this will probably be the master branch. @rikard-sics from RISE is working on OSCORE feature since several months and this will be integrated in Leshan 2.0 when this will be ready.
@cavenaghi9 was working on senML but vanished without any words so PR are pending but could be used as base when we will work again on this.
If somebody want to work on LWM2M 1.1 feature, he is welcome. Again if it helps I can update a 2.0.0 branch now or I will do that after the leshan 1.1 release.
If somebody want a release with LWM2M 1.1 features, we can deliver a milestone release (e.g. 2.0.0-M1).
We will probably go to a lot of milestone release and when we will consider the 2.0.0 API stable we will deliver the stable version, then 2.x version without API break if needed. (like we do for 1.x)

By the way the development model you proposed is pretty much the one we use. We try to do the same except that we need to deal with several version of the standard which make this more difficult. Until now we focus on 1 LWM2M version at the time. This could maybe change in the future if this was finally consider as a mistake.

3) I understand your doubt. Unfortunately I can not see the future but AFIAK my company still want to make Leshan forward and LWM2M 1.1 is the next step. FMPOV, the best way to consolidate this is to contribute.

4) master is not the only one development branch. master is just the most active branch.

Problem with these temporary branches is that they do not guarantee that the work gets integrated or maintained.

Why are you considering all other branches than master as temporary ? 1.0.x : is not a temporary branch 1.x : will not be a temporary branch (current master) 2.0.x : is not intended to be a temporary branch (future master) Other branches are just feature branch, they are deleted once there are integrated in master. (sometime this is abandoned one which could still be useful, so we keep it)

I understand that 2.0.x branch afraid you because this branch is currently abandoned. This is just because it is not used anymore, but if this is needed again we can revived it. (we talked about this here)

So, If you want to contribute you are welcomed. If you have concrete issue when contributing, just tell me and we will try to find a solution. (please try to be concise) If you need a release you can ask also. You can base your work on master if you prefer or I update you the 2.0.0 branch. I could create you a dedicate branch if needed. To get a rid of revapi in master, use -Drevapi.skip=true (as for 2.0 API break will be allowed)

[henris42]

@sbernard31 All this makes me ask why there are multiple active branches? Community contributors do not want to do backports to multiple branches. Are you also saying that you are going to rebase the project with the 2.0.x? All this seems very complicated to me. IMHO the plain master should just track the standard, and if you need maintained branches for product purposes, you fork those. Main master has only requirement that it builds and "about works" with no guarantees.

[sbernard31]

All this makes me ask why there are multiple active branches?

Leshan v1.x.x is for LWM2M 1.0 Leshan v2.x.x should be for LWM2M 1.1

1.x branche is for minor release of v1 (new feature no API break) 1.0.x is for patch release of v1.0 (no API break, only bug fix)

Very simple and common way to manage software verison : https://semver.org/

Community contributors do not want to do backports to multiple branches.

What makes you think you need to do that ? If you are interested by LWM2M 1.1 you don't care about other branches ...

Are you also saying that you are going to rebase the project with the 2.0.x?

In the future the v2.0.x branch will start from the v1.1 release and then v1.x and v2.x branches will live their own lives. (no more rebase for 2.0.x) The current 2.0.x branch was just here waiting main maintainers was able to finish their work on 1.x and let contributors begin to work on LWM2M 1.1.

IMHO the plain master should just track the standard, and if you need maintained branches for product purposes, you fork those. Main master has only requirement that it builds and "about works" with no guarantees.

I have strong doubt this could have help in the previous situation.

@henris42, @dachaac, I don't know if you are really aware of that but your current approach could be considered as inappropriate. It seems you are or you were using Leshan and I never see you before (no issues, no comments, no PR, no thanks you) and first time you interact with us, this is about organization criticism, missing features and our incapacity to follow the standard. You never tried to work with us but you want to remodel the organization.

I understand your fear about contributing for nothing and that Leshan doesn't move fast enough on LWM2M 1.1, I hope I answer to this fear. Now If you are interested in contributing concretely, welcome on board ! If you face any concrete issue, we will try to resolve this together.

[dachaac]

@sbernard31 I have been following the progress in background for couple of years.

My worry was that if there is only one or two developers working on master you could use a bit help in there. But as I mentioned already feel free to use whatever model you see fit best.

I have been rebasing my work from time to time and there seems to be some conflicts from time to time so this is kinda causing some hesitation to work.

In order to fix some revapi issues I had to increase already version to 1.1.0 -- now that you have done that also in master perhaps the best approach here would be to just propose changes in smaller PR's.

Perhaps I should not try to finalize the feature right away to be as complete as it can but instead slice it to smaller integradeable parts.

I can write issue tickets describing the issues and then propose a change for problems I have tried to solve. Perhaps that kind of approach will bring us to solution faster. I will however like to target master branch as it feels better target -- I let you judge if something is too early for inclusion.

I start with certificates and EST parts and lets see how that evolves.

[dachaac]

+1 for EST. I think that would be a key component, if there is any willingness to make this a platform that can be used on real production installations. I recommend only real certificate-based provisioning processes to my consulting clients. But, I'm not really talking about CoAP EST, I'm not aware of any real implementations of that. But forwarding certificate-based bootstrap to a normal EST service (like OpenXPKI) would be really good concept.

Our current approach is to make CoAP-EST to act as Registration Authority and then have separate EST service that does the actual PKI work -- with this approach there is no need to implement CA functionality in LWM2M service side -- only verify that request is good to be passed forward.

I started writing things out in public today to wiki in here: https://github.com/dachaac/leshan/wiki/Improving-support-for-LWM2M-with-Certificates

[dachaac]

I created a ticket for tracking EST support: https://github.com/eclipse/leshan/issues/859

I hope that this gets this feature forward.

[sbernard31]

Leshan 1.1.0 is out and so master will now be used to develop Leshan 2.0.0. So old 2.0.x branch is no more needed anymore (I will delete it). Code from 2.0.x was cherry-picked/adapted in master.

[zakharenkodmytro]

Hi guys, currently Im trying to find resource instance R/W operation feature. Is current master branch supports it? I know that it is part of lwm2m 1.1 but maybe it is already implemented?

[sbernard31]

@zakharenkodmytro, R/W resource instance was not available in v1.0.x of LWM2M and not yet implemented in leshan master.

[zakharenkodmytro]

@zakharenkodmytro, R/W resource instance was not available in v1.0.x of LWM2M and not yet implemented in leshan master.

Thank you for prompt response. Do you know if 1.1 of Lwm2m is implemented in other branches OR some road map for implementation in the nearest time?

[sbernard31]

master branch now targets lwm2m 1.1 (I probably need to update the README about that :thinking:) Looking at issue and PR you could see trending topic (OSCORE, Certificate, v1.1 model support, unsigned integer support) I could have a look at how this is impacting to support it and contribution are welcomed too.

[zakharenkodmytro]

Thank you for update, it is good to know that now master targets v1.1. I will try to contribute , sure. Just let me know if someone else work on resource instance feature already.

[sbernard31]

@zakharenkodmytro, I prepared the ground to make it easier to introduce resource instance operation. You should have a look at #877. I don't plan to work more on this at short term, so If you want to move this feature forward, please base your work on read_resource_instance branch. (If you want to discuss more about this, please use #877)

HTH

[zakharenkodmytro]

thanks, let's communicate through #877 then.

[sbernard31]

The first milestone release containing some LWM2M 1.1 feature is out : 2.0.0-M1

:exclamation::skull_and_crossbones::exclamation: Affected by security issue:exclamation::skull_and_crossbones::exclamation:

[sbernard31]

A new milestone release containing new LWM2M 1.1 features is out : 2.0.0-M2.

:exclamation::skull_and_crossbones::exclamation: Affected by security issue:exclamation::skull_and_crossbones::exclamation:

[sbernard31]

Just to let you know that Read Composite(#958) and Write Composite(#959) operation support are in development.

[zakharenkodmytro]

Thanks for update, i will check it out, i have no idea for now what it is)