search

Hypfer / Valetudo

Cloud replacement for vacuum robots enabling local-only operation
https://valetudo.cloud
Apache License 2.0
6.38k stars 388 forks source link

RFC: The future of Valetudo 2020+ #422

Closed Hypfer closed 4 years ago

Hypfer commented 4 years ago

Moving forward, I guess that it is safe to say that there will be more vacuums at some point. There's a Viomi one in #420 but that will certainly not be the last one.

This seems to be a good time to emphasize the mission of Valetudo.

What Valetudo is

Valetudo aims to be a generic thing that you can put on your vacuum to remove cloud connetivity, provide a local MQTT Interface and maybe the Web interface as well, although I'm unsure about that. It might make sense to decouple it completely.

Valetudo should completely remove any vendor-specific details such as Map formats, Command formats etc. Basically you should be able to buy a robot, install Valetudo and it works exactly the same as your existing Valetudo-enabled vacuum even if it is made by another vendor.

What Valetudo isn't

Valetudo shall not become a swiss army knife of some sort. That means that everything that can be done elsewhere shall be done elsewhere. (e.g. connectivity to other services such as Telegram)

How do we get there?

I think that Valetudo requires a major rewrite. The current state is basically just a prototype.

The main ToDos that come to my mind are the following:

Capabilities

This should feature some kind of capability system similar to the supported_features of an mqtt.vacuum in home assistant. Especially with the new room detection features, there's just no point in hacking those into the current codebase.

We'd need to figure out what those might be (preferably future-proof) and how to incorporate limits to it such as the 5-Zone Zoned Cleanup limit of the roborock vacuums which often caused confusion e.g. #421

Map Data Format

There needs to be a proper Valetudo Map format which is future-proof and may contain new capabilities we don't even know yet. It will most likely be some binary blob since the current state using JSON just isn't feasible for larger setups as well as mobile access (#356)

Something like Protobuf could probably be used here.

Deployment

Valetudo binaries are huge which worked out okay-ish at first but since vendors have now started trimming down hardware specs of their vacuum, I think it's safe to say that the current state isn't future-proof.

We're also using an old and deprecated version of the pkg tool used to package valetudo into a single binary since newer pkg versions require a newer libc. That will also certainly break completely at some point.

Challenges

As you might have noticed, development of Valetudo slowed down quite a bit. There are multiple reasons for that but for the most part it's the fact that it is mostly just me who worked on Valetudo.

Looking at Heise, private Messages, the comments in #400 and many more things, It became clear that the way I mentioned contributers apparently was misleading. There is almost no community.

There only thing there is plenty of are entitled end-users who don't understand FOSS at all and just demand that things shall be done for them because reasons.

This ultimately led me to pretty much abandon the project because it just isn't fun.

There are also still unsolved issues with randomly resetting robots (#206) which constantly leads to a lot of frustration on all sides.

Also also there seem to be some interpersonal conflicts which I don't really understand but am unable to clarify since there's just silence.

Moving forward

There however is a solution to the mentioned challenges:

If you are unhappy with Valetudo but unable to contribute something to change that just use the non-fork fork Valetudo-RE and get the fuck off my lawn.

I am so done with you people.

Closing remarks

So anyways these are my plans for Valetudo. As you may have noticed, this Issue is labeled RFC so feel free to add your thoughts

legomind commented 4 years ago

@Hypfer I totally get your frustrations regarding the lack of contributors & community around the project. I happen to be one of the users dealing with the reset issue, and I regret that I am unable to add to the discussion regarding a solution. Thank you for being up front about the future of this project. Hopefully, the possibility of extending this project to other vacuums will entice more contributors. Your work on this has been amazing, I am sorry that there are many who do not seem to appreciate this project is FREE.

rumpeltux commented 4 years ago

Thanks for this overview! A few drive-by comments:

alexkn commented 4 years ago

First of all, thank you for your work.

In my opinion Valetudo is in a "works for me" status. I just don't need any new features. So I don't really know what to contribute. And I think so do many Valetudo users who could also code.

Bigger wishes, like the support of other vacuum cleaners and new firmware versions are not really tasks for beginners. New vacuum cleaners are also associated with spending money without knowing whether you can get it working. In my opinion this is a task for a main developer who may be able to buy the hardware from donations.

Some ideas for moving forward: I think the project should be splitted: 1) Code which really have to run on the vacuum which exposes a MQTT and/or REST API. 2) Webinterface 3) Supporting libraries, like the map parsing stuff, which could also be used in ICantBelieveItsNotValetudo or in my node-red-contrib-valetudo project.

With such a structure, it should also be easier to support new vacuum cleaners. If you then have the choice of what is actually installed on the vacuum cleaner, this could also solve the deployment problem.

matthiasharrer commented 4 years ago

I really appreciate your work so far and I like that you are thinking about the future of the project and proposing some new ideas.

I can totally understand the frustration regarding users who just expect things to be done, opening duplicate issues, asking for help without reading the existing issues / wiki etc. However I also think it is quite an achievement to have so many people interested in the project.

Maybe it would be an option to find someone who wants to help (and does not need to code) to manage the support issues etc. This would free up the time for you and other contributors to work on things that they enjoy doing in their free time.

Although I am not sure how much time I could put into the project right now I would totally enjoy some additional communication channel which is more focused on the technical details going forward than the existing telegram channel. Hopefully some potential contributors would come together starting to discuss the technical bits, like splitting the project, rewriting bits etc in more detail.

alexkn commented 4 years ago

@matthiasharrer Technical development questions can be discussed in github issues. For helping normal users (Support requests) it would me nice to have a forum.

I don't like telegram for discussing anything.

JPT77 commented 4 years ago

Hi, I am a new Roborock user. Thanks for this project! It's great that we can make this robot work without the "cloud". But the current state seems to be far from stable. I currently am frustrated and now considering just using manual control without any wifi connection. I would like to help, but I don't know how much time I can invest. I am used to u-boot and building for arm. Fiddling with linux... and I am Java developer. So I would be OK with writing JS I think...or with any debugging tasks... So yeah, probably we should develop a new architecture first and then distribute separate jobs to people.

kschuerholt commented 4 years ago

Let me also second that and thank you for the great work you've put in already. I absolutely understand your frustration, any kind of demand for support or features is ludicrous.

The roadmap you've laid out seems sound. If there's a way you see to split further development into smaller, digestible chunks, people will consider developing it? I for once don't know if I could contribute on a large scale for lack of experience, but if a sub-project has a clear task and interfaces, I might be able to help.

matthiasharrer commented 4 years ago

@alexkn You are probably right. We should setup a support forum and find some people willing to contribute as form mods .. this should hopefully also move support questions from github issues over to the forum.

Maybe we should also introduce a new label for marking issues for the rewrite/rearchitecture, so that they are easy to spot.

rumpeltux commented 4 years ago

@alexkn You are probably right. We should setup a support forum and find some people willing to contribute as form mods .. this should hopefully also move support questions from github issues over to the forum.

Maybe it makes sense to ask the folks from roboter-forum.com to create a sub-forum for valetudo. It’s already being discussed there quite extensively.

matthiasharrer commented 4 years ago

@rumpeltux I also thought of this and like the idea, however roboter-forum.com seems to be mostly in german. It's a good idea nonetheless as I think most of the users are from germany currently.

balu- commented 4 years ago

hi @Hypfer thanks for making this project happen. Also thanks for laying out your thoughs so clearly as you did. I totaly get your frustration, about the "one man community" and the "demanding" attitude of some users. Further I'd totaly secound your view on the rewrite step that needs to be done to get to a "bright valetudo future". But at least in my opinion a very big part of the current situation isn't pure codequality. For me the Gordian knot is #206 (the random firmware resets) which i think needs a solution first.

why do i think that is such a big deal

I know those are my very personal reasons, but I imagine that those are reasons that are at least in part holding the community back.

I really hope we find a solution to this issue. [If there is anything i can do for that please let me know]

deg0nz commented 4 years ago

I have to say, I can understand your frustration and feel with you. This is not easy. I know this from 1-2 repos I have publicly available myself. So I want to thank you for this and say to keep up the work. It is really appreciated here. You saved me a lot of work with this... For me, Valetudo is also in a "works for me" status, as @alexkn said. So no complains.

I have a question about the rewrite: Are you planning to leave it in plain Javascript or are you considering switching to TypeScript? Or even a completely different language (as Rust for example)? (This could solve your space-problem)

When I have finished my current project, and I find time, I'm happy to contribute to your rewrite. I'll watch this issue to check the progress now and then.

Hypfer commented 4 years ago

@deg0nz Thank you. Oh and thanks to everyone else who commented as well ❤️

See #426 for the rewrite.

Personally, I don't really like Typescript since for me it adds additional complexity without any meaningful reward. That might have changed, but so has ES6.

We definitely need to solve the deployment issues (outdated and broken zeit/pkg, less storage & memory on newer vacuums) in the foreseeable future so using something like rust could definitely make sense. Only problem being that I couldn't contribute much to that codebase :(

I guess it should be doable using JS. We'd just need to figure out how to get the runtime on the robot. I haven't looked into that yet. zeit/pkg was choosen at the time because it was easy and self-contained

t-animal commented 4 years ago

Honestly the decision not to switch to typescript has stopped me personally from contributing. I asked whether this would be wanted in the telegram group some time ago and it was shunned upon. Having developed javascript some time and now only typescript in my professional life I never wanna go back, certainly not in my free time. This interaction even led me to giving talks about typescript now, because I always thought the switch was a no-brainer.

This having been said, I think you're absolutely on track with the design decisions mentioned above. Decouple the frontend and the software on the robot. Let others do frontends how they like, be it react or vue or plain HTML and some JS (modern, hopefully), be it typescript or javascript. Be it a website, PWA or native app. Offer a set of APIs. This will not only make your life easier but also hopefully relief you of the burden to have to answer users' complaints in issues. A prerequisite would probably be to find someone to develop at least one frontend project.

Lastly I just wanna say thank you. My robot resets every couple of weeks and I have thought multiple times about going back to only using flolevac. But still, I flash valetudo every time, because it just adds so much usefulness. Just did it again today and probably will continue. Thanks.

HHenning0815 commented 4 years ago

I see your frustration, but...

Just close a support issue without notice why ists closed (without chance that someone can answer) is also frustrating.

I know that I have no clue about programming or so, but all that I hoped for is a little help to know what I can do to fix my problems.

rumpeltux commented 4 years ago

I see your frustration, but...

Just close a support issue without notice why ists closed (without chance that someone can answer) is also frustrating.

I know that I have no clue about programming or so, but all that I hoped for is a little help to know what I can do to fix my problems.

@HHenning0815: I think you’re referring to #473. I hope you can understand that we cannot provide individual support for issues not directly related to the Valetudo software. That said, I do agree, that bugs (even support requests) shouldn’t be closed without comments. We should setup a canned response that points people to a better place to ask such questions (e.g. for now, in lieu of a better alternative this could be roboter-forum.com).

docBliny commented 4 years ago

@Hypfer While this isn't the correct forum for this, I just wanted to drop by and say thanks for the work on this project! I just updated our two gen1s (and hopefully the reset fix sticks this time) and it reminded me to post.

Hypfer commented 4 years ago

Regarding the map data format:

It might make sense to rethink this one. The whole concept is heavily biased by how roborock saw things but that's not necessarily the right approach for a generic abstraction like Valetudo.

Currently, it contains both "static" as well as "dynamic" state data.

"Dynamic" state data includes things such as:

while "static" Map data includes these:

Basically there are these two very different questions one might ask the robot:

Which data source is used in the background to answer these questions should be irrelevant.

steven-dyson commented 4 years ago

Wasn't really sure where to reach out so I figured I would try here. I have an E35 vacuum. The readme was specific that only the older models are supported (Mine is May2019). Will you be adding the ability / readme / video how-to to root these newer vacuums? Even it it requires opening the vacuum (as long as I dont have to solder anything).

Hypfer commented 4 years ago

You cannot run Valetudo on an E35 since it doesn't even run linux at all.

You could certainly try dumping whatever storage there might be to acquire the CloudKey as well as the DID, but there hasn't been any research on those budget models yet.

Furthermore, I don't own one and in fact I'm not even the one who did and does all the root magic.

And, lastly, this issue isn't even remotely the right place for this question.

fokcuk commented 3 years ago

The only real feature I want is to be able to save/recover maps from my Gen1. But it cannot. I am not a programmer, so cannot really contribute. But I saw a post from 2019 that someone just copies the whole roboroc folder and restores is back every time the vac is used and it works great for them. But 2 years on and I think Gen 1 is long forgotten