search

commaai / openpilot

openpilot is an operating system for robotics. Currently, it upgrades the driver assistance system in 275+ supported cars.
https://comma.ai/openpilot
MIT License
49.82k stars 9.08k forks source link

Make visiond more HW portable - open source it all except DNN part #96

Closed kozuch closed 7 years ago

kozuch commented 7 years ago

Hi there,

this thread is aimed primarily at comma.ai staff (@geohot etc. - others wecome to comment too though!). You guys want openpilot to be successful and have as many people as possible using it, right? I see a big obstacle here towards that path in the visiond binary being locked down a single high-end $700 smartphone model. Let's make the whole project more accessible by changing this. Seems like the project is still in experimental stage and you need loads of field testing to validate your code and to push it closer to production. You may be getting many more testers when the HW needed costs (just guessing) $100 instead of $700.

I see your business model in keeping your deep neural network closed source and I am perfectly fine with it. But would you want to open-source the most of visiond except your DNN blob so that you keep your intellectual property while allowing for a much wider variety of HW that can be used? Let's open source all visiond (the camera stuff etc.) except your DNN blob and also make that blob as portable as possible (I guess there are ways to making DNN portable?). You will have much better reach and could test performance on various HW and the project may gain more popularity like this. I know this may hurt by having to support various CPU/GPU architectures but I think the benefits may outweigh the drawbacks.

Hope you will be hearing to my post! :)

geohot commented 7 years ago

This is a duplicate issue, see #4

One issue is even if you ported it, performance of the network won't be good with other cameras. And you really wouldn't be able to bring costs down much, since you need an 820. Currently we have no plans to open source visiond. If I saw substantial pull requests and contributions to other parts of the code I'd be more open to it.

kozuch commented 7 years ago

You are killing the community by insisting on (rather) expensive and limited set of hardware. This is not the way community projects work, but I guess you know this and you do not need to grow probably.

geohot commented 7 years ago

It won't work on (much) cheaper hardware. Show me a $100 phone with an 820 equivalent GPU, and we'll happily port it to that. We didn't choose the OnePlus 3 because we want people to have to spend needless money, it's the cheapest open 820 phone available.

If you are talking about a PC port, the ACC portion will run on a PC without visiond. Show me that you have that working first. Then you can connect your own camera to a PC and use whatever DL library you'd like.

kozuch commented 7 years ago

I am not accusing you of pushing people to spend endless money and I dont have anything working but I am trying to say that a successful community project needs to have the barriers to entry as low as possible...

Even though I do not have experience with DNN performance I do have some with classical computer vision algos - often they perform well on low resolution input images. I did some research with stereo vision and found that performance did not really scale with resolution (16x more pixels had only 4x less vision error). In other words one was able to downscale the image to speed things up without hurting the performance (in terms of errors) much. Maybe something similar could be done with DNNs.

What input resolution do you use for your DNN? Have you tried to lower the resolution to speed things up and see how it performs (in terms of both speed and errors)?

johnnwvs commented 7 years ago

@kozuch

I was just at a symposium where similar topics were discussed. I did hear some experts saying that higher resolution tended to have diminishing returns with respect to neural nets, pattern recognition, etc.

Also, if there were an interest in opening up more of the code, or just a general need for a port, I will offer my time to help.

I would like to have this running to test on my car (which has LKAS, etc) but I do not want to spend $700 to do the testing up front. I might spend that much once I am convinced it will work though.

geohot commented 7 years ago

Just out of curiosity, what would you spend?

johnnwvs commented 7 years ago

In order to differentiate, I would be willing to spend anywhere from $750-1000 for a fully working system that is guaranteed to work with my vehicle. IE: a production level device.

For testing a system I am thinking around $150-$300.

kozuch commented 5 years ago

Bump here. The visiond was open sourced lately. I created #562 for new discussion.