Apollo and real time

[ubercool2]

Dear all, I have been looking at Apollo for the last couple of days and I have really lots of questions. I will start posting them one by one in the issues. Let me know if you want them otherwise.

In http://apollo.auto/ it is stated that Apollo is open, reliable and secure software. It is also stated that the by 2017-12 Apollo was gonna be ready for deployment in "Simple Urban Road Conditions". Now one of the most elementary things to have the system act reliably (which you want on public roads) is to have it act in real-time and deterministically. For the definition of the latter 2 please see this excellent talk: https://vimeo.com/236186712.

Now I look through Apollo code and:

1. I see that you apparently use kernel with RT PREEMPT but I see no where in the code call to set the scheduler (e.g. FIFO) or to set the real time priorities. Does it mean that you just run on the normal desktop ubuntu where your programs run and share resources with wifi, bluetooth and such drivers?
2. I see logging to files or file manipulation being used everywhere: https://github.com/ApolloAuto/apollo/search?utf8=%E2%9C%93&q=fopen&type=. All of these calls are blocking and not safe at all
3. I see new and delete operator being used everwhere: https://github.com/ApolloAuto/apollo/search?l=C%2B%2B&q=delete&type=. Also on the runtime which means that you fragment the memory all the time. Which in turns means that after hours of operation you will not be able to allocate even a simple string anymore.
4. I see std::strings being used everywhere (they can not be made static).
5. I see mutexes being used everywhere: https://github.com/ApolloAuto/apollo/search?utf8=%E2%9C%93&q=mutex&type=. They can not be pre-empted or controlled with priority and hence can not be used in real time code
6. I see lots of variadic functions (e.g. printf, fprintf, ...). See why this is a dangerous flaw: https://stackoverflow.com/questions/3555583/passing-variable-number-of-arguments-with-different-type-c.
7. I do not see any memory locking (mlockall) or heap trimming (mallopt) operation that would allow memory allocation on the start only and not yield any memory back to the OS.
8. I do not see any memory supervision tools being used
9. I do not see any tracing tools being used (e.g. LTT)
10. ...

These are just some really, really obvious things which makes me wonder - what is Apollo? A collection of research libraries and tools? Because with above flaws and many more that I haven't even mentioned I do not know how there will be a path to having cars drive autonomously on the road.

[slivingston]

I am not a core developer of Apollo, nor am I affiliated with Baidu, but you wrote "Dear all", which includes me :heart:

I parsed your OP as having two parts, where you tell us:

1. what you expect if Apollo is to be called reliable, and
2. that you think cars driving autonomously with Apollo is hopeless.

First and foremost, I agree that having the system act in real-time and deterministically is critical. As you show in the OP, Apollo is currently neither. As such, it is not ready for production.

However, it is ready for use in prototyping, as a starting point from which to build new capabilities, and as a framework for research. That is to say, it is sufficiently reliable to be used in highly monitored settings, with a safety driver, etc., or in restricted road networks, etc.

From my experience in the space of startups that are making autonomous urban cars, that is all that many companies can say in terms of what they have developed internally at this time. Engineers at these startups are thinking about the future and plan to eventually refactor their code to be reliable as you define it. But for now, there are many open problems that we need to solve first. Solving these without devoting attention to real-time and deterministic behavior is more efficient, given that once good solutions are invented, we can shift to achieving reliability as you define it etc. in preparation for production.

As for your final assertion of doubt ("I do not know how there..."), I think the most useful perspective to have is that Apollo is an open source project that is relatively new. It is not a polished product. It also has a wider audience, including research scientists, educators, and for-profit companies. For myself, I am using it both for research in so-called formal methods for robotics, and also as a platform for rapid-prototyping by startups.

Anyway, as an open source project, you can contribute toward creating "a path to having cars drive autonomously"! You can also fork it and start a rival project.

[ubercool2]

@slivingston thanks a lot for your reply. I read my post again and I definitely sounded more critical than I wanted it to come along.

If Apollo is for figuring out of open problems and prototyping then this is fine with me. :+1:

I was just confused if Baidu really thinks that this code is safe, robust and secure and can get us to L5 in 5 years (as e.g. claimed here http://autonews.gasgoo.com/70014368.html).

Regarding "given that once good solutions are invented, we can shift to achieving reliability as you define it etc. in preparation for production" - there are some algorithms in Apollo, such as e.g. lat/lon controllers in this repo that have been invented and proven decades ago. There is absolutely no reason not to implement them as per practices I listed above.

[Zhenni17]

Hi @ubercool2 ,

First of all, thank you very much for spending your time looking into Apollo! We really appreciate your effort and your candid feedback. It really helps us to approach the autonomous driving challenge with different perspectives.

Just want to let you know that we hear you, and our engineers are working on a response for the issue you raised! Thank you for your patience with us.

Best, Zhenni Community Manager - Apollo Platform

[Zhenni17]

Hi @Ubercool2, thank you again for your interest in Apollo, we appreciate your patience with us.

To answer your question, Apollo is an evolving, open, platform that’s still in an early stage of development. We will continue to improve it by adding new functionality and features, and will address some of the issues that you mentioned here in future iterations. Apollo provides a solid autonomous driving foundation that our developers and partners leverage to create systems that can meet their specific needs. The Apollo Team is happy to provide engineering support for co-development efforts.

We value your feedback tremendously and I would like to invite you to connect with me directly if you would like to discuss these topics in greater detail. My email is wuzhenni01@baidu.com, and please feel free to send me an email anytime and I will be more than happy to schedule a follow up meeting with you.

Best, Zhenni Community Manager for Apollo Platform

[ubercool2]

@Zhenni17 I take your answer as confirmation of my hypotheses above since you provided no concrete technical answers. We are building production autonomous driving systems and we seriously considered Apollo but Apollo can not be just fixed by adding functionality and new features. Things like safety, security and real time are fundamental and need to be in from the day one.

However I agree with you that having a research system like this speeds up research and prototyping by a lot.