Kiwa21
commented
7 years ago Following. Very interesting
Open cglusky opened 7 years ago
Kiwa21
commented
7 years ago Following. Very interesting
cglusky
commented
7 years ago Will require a slight change in Maverick scope from "Unmanned Vehicle Companion" to something more general.
Two were suggested on gitter: Unmanned Systems Companion Computer Autonomus Systems Companion Computer
cglusky
commented
6 years ago No, not a new problem at all...
Dronecode discussion on RTPS http://discuss.dronecode.org/t/messaging-systems/87
In particular...
Instead of building a separate layer using RTPS, why not building on top of the existing ROS 2 client library directly and embrace all the fantastic work that is happening at the OSRF?.
And here http://discuss.dronecode.org/t/message-format-options/88
With this graphic...
fnoop
commented
6 years ago This issue is an excellent writeup of the various options that will be critical to the next stage of maverick - a multi-vehicle capable GCS/webui, and for multiple components within a vehicle that compete for resources, and for the widest possible interoperability.
Abstraction and Leverage are the two key words to consider, given that this is a community effort with very limited resources. The obvious starting point is ROS, and therefore MAVROS - this represents the the best current implementation of the above requirements. ROS2 looks very promising, can run alongside ROS(1), and is about to come out of beta in the upcoming week or two. From a system (companion computer) and flight controller point of view, this should be the first point of testing. In particular, simple tasks should be tested on very low power computers such as a Pi Zero, to determine if it can run on these types of resource-restricted embedded platforms.
cglusky
commented
6 years ago Thanks @fnoop. Been sketching out some block diagrams and I agree that ROS/2 is likely best place to start. If ROS2 works out I think it may solve many of these M2M issues for us. It appears we are haunting some of the same issues as I see your comment on the mavros ROS2 support thread. There are also some interesting issues regarding time sync with ROS/mavros and PX4.
Using MAVLink for autopilot to CC via serial and autopilot to GCS via wireless serial makes sense. Already done and well supported. However, it may make most sense to have Maverick instances talk via ROS2. There appears to be a spot for node.js in the GCS env since it would make integration easier with other projects such as Robot Web Tools, Cesium and even WebODM; assuming ROS2 node client is good to go. https://github.com/RobotWebTools/rclnodejs
A grossly oversimplify way to put it would be maximize ROS2 for M2M and Node.js for HMI.
Sorry, I don't grok Abstraction and Leverage in CS terms. I'll need to work up more anecdotes until I do.
cglusky
commented
6 years ago node.js API gateway. still young. https://github.com/Leanty/tree-gateway
so now we have to add the microservice buzzword to the mix.
cglusky
commented
6 years ago Another API gateway for node.js. http://www.express-gateway.io/getting-started/
At first glance these API gateways do allow for potential of data interop while addressing many of the core issues. But it also adds another layer that is perhaps best suited to run on a GCS env.
It's perhaps hard to ignore node.js just due to it's popularity, which is Leverage in a broader sense.
Cloud9 is built on node.js and is already in Maverick. It is also optimized as a node.js IDE. So part of the tooling is already there.
fnoop
commented
6 years ago 
This is a very good insight to read:
https://hackernoon.com/the-javascript-phenomenon-is-a-mass-psychosis-57adebb09359
This is humorous, but literally not an exaggeration:
https://hackernoon.com/how-it-feels-to-learn-javascript-in-2016-d3a717dd577f
Having used javascript a reasonable amount in the browser for a couple of years, I'm very much with the above. I can usually get things working, but only after a large amount of desk-head banging and wtf moment. I usually don't dare to touch things after they're working for fear of breaking everything. The thought of trying to implement an entire server-side framework using this mess of a language fills me with dread, particularly as it will have to be re-implemented at least once a year as whatever library/framework you choose is obsolete 6 months later.
Technology and frameworks should provide you with tools to help you get a job done and accelerate your work, not hinder you. Javascript isn't one of technologies, at least not yet. No doubt javascript and nodejs have all sorts of benefits, but unless you're an expert in them then they're a massive frustration to work with. At the moment, I'm the only contributor to Maverick so I have to use what I'm willing/capable of using, and that's not going to be nodejs. If anyone else here has expertise in javascript/nodejs and is willing to contribute time and code then I'm all ears.
cglusky
commented
6 years ago I think the fact that you are currently the main and only contributor is the best reason. I am willing and able to pick up enough JS and Node magic to be dangerous. Node is already there for Cloud9 so I can start playing around with node related services such as WebODM and Cesium. Gives me an excuse to get back into coding a bit. And will be a easy way to make sure you can artificially limit JS LOC in Maverick since it will take me many months to have anything meaningful.
Really enjoyed reading those articles. Just researching Node and all the frameworks and API's is enough to cause paradox of choice. So now we have the reactions such as Elm and WebAssembly which is to compile code into JS? The irony of someone doing that on a QWERTY keyboard does not escape me. It's like they analyzed what makes Node so popular and said "we must do the opposite to stop this craziness." I think Rust may be OK since you are not asking C and C++ devs to change as much. But trying to fix the issues created by Nodes explosive growth and wild west proliferation of libraries and mangled code by inserting a trans/compiler is probably not going to work IMO. There will be middle ground. I volunteer to see if I can find it for Maverick. On a not to interfere basis for what you are doing on server side of course.
fnoop
commented
6 years ago Whatever you choose now to base a javascript project on, particularly on the backend, is almost guaranteed to be obsolete within a year to 18 months, quite possibly sooner. As far as I can see, the popularity of Node (essentially a headless browser) is purely because it allowed front-end developers to work on the server which has always been a mysterious 'there be dragons' area for them. That's a bad thing on two fronts: 1) front-end developers with no concept of the back-end working in the back-end, and 2) javascript is inherently poor for back-end work - it's a very thin environment and lacks a standard library, and the less said about the object model the better. Of course it has advantages - it's event based and has an incredibly well optimised runtime engine and it has a huge developer community. Recent versions of Javascript are starting to provide a much richer, more mature environment to use. IMHO, javascript still belongs in the browser, and languages such as Python/Go/Java belong on the server. A good example of this is the ROSBridge suite - the server end is implemented in Python and the client side in Javascript, which is a fairly common workflow.
For UAV components, Python and C++ are the overwhelmingly popular languages - Python generally and C++ where speed is of the essence. There is an actively used and maintained Python binding for every component that we are looking to expose - ROS, OpenCV, Gstreamer, Tensorflow, Dronekit/Droncore, so that's the obvious choice for language in the backend, currently.
For UIs, there is only one option - Javascript. In order to minimise the implementation effort and time to market, basing the webui on one of the popular frontend frameworks - Angular, React or Vue - is a no-brainer. None of these fit in well with the traditional three-tier/MVC app architecture of server-side coding. React and Vue essentially provide the 'V' of MVC, and Angular essentially provides MV(V)C but all in the browser. Angular and React are massively popular, and have massive communities and support - Vue less so because it is not backed by a major corporate but becoming very popular. Angular is very complex and has a steep barrier to entry, React less so (because it tries to do less), and Vue is supposed to be the easiest of all. All are essentially component based and have optional immutable/state models. A brief discussion with @SamuelDudley came to a vague agreement that Vue currently looks like the easiest to learn and get going with, and is a nice clean software.
As I see it, this is less about what the utopian perfect design is, and more about what is practical now to implement, given very limited resources. A good architecture is more important than particular technology choices.
My back-of-a-paper-napkin-that-is-still-in-my-head design has a Python/ROS backend that fulfills the M-C of MVC model, that implements a pure, modular JSON API that is consumed by a Vue.js javascript frontend. No rendering is done in the server, it's the server's job to provide data to the frontend which does all of the rendering. The frontend should be detachable to as much of a degree as is possible given the current technology - that is, it should function offline using PWA, in a desktop using Electron or on mobile devices using PWA or automated native apps (eg. Weex). The GUI should be multi-tenant, multi-target (capable of managing swarms), multi-component (capable of managing Autopilots, multiple companion computers etc), and contain functionality to manage Maverick itself, manage connected autopilots, contain rich GCS features, live HUD/video, and rich reconstructed 3D viz in VR and AR. All of this is feasible given the current technologies, and with the incredible advance in web technologies represents possibly the best environment to achieve such lofty goals, even given the limited resources at hand.
That's my current thinking :)
cglusky
commented
6 years ago Fantastic stuff @fnoop! I got out of full time web dev before Node came along. Back then PHP/HTML/CSS and perhaps a sprinkle of JS would get it done. I even created my own CMS from scratch using ASP because the job at hand had some rather closed-minded rules. Dynamic in those days still allowed refreshing the entire page. Then came AJAX and jQuery and other JS black magic that most of us did not touch below whatever layer of abstraction was suitable. So I am a bit skeptical when people say Node became popular because front end devs were already using JS. Some were for sure. I suspect most others in the early Node days just picked it up to stay relevant. But I digress.
I would prefer Vue as well as it does not try recreate HTML/CSS with yet another language. The examples feel more natural as it jives with many of the template languages used in old school PHP frameworks I used to use. I'll leave the technical merits to those who can speak to them.
I think those of following Maverick are on board with your vision else we would not be here. I am just your pain in the ass that will keep asking questions until we form a bit of a plan. And I do mean we as I am willing to help with docs and such. The plan does not need to complex as we all know it will likely not survive first contact with reality. And we are getting off topic for gateway module.
Do you want me to start adding issues related to planning/governance? This is your baby so understand if you don't want help raising it right now. But I think if you want people to contribute they will need some guidance.
cglusky
commented
6 years ago Back to the gateway. This comment is mostly about RDF and Semantic Web. If you look at a cup of coffee and it says, "Made from 100% recycled material" you may safely (depending on how bad the coffee might be) infer that the statement pertains to the cup and not the coffee. A computer might have a harder time with that inference; thus, the semantic web. Or from someone a bunch smarter than me:
https://github.com/JoshData/rdfabout/blob/gh-pages/intro-to-rdf.md
Why do we care? Well, a Gateway Module would do everyone a favor if it was more than just a bunch of data transforms. Seems like a good idea. Why not? Well, it's a bunch of work. Assuming you can even get people to agree on structure and syntax of underlying protocols, APIs, etc etc, now you have to agree on a domain specific ontology for unmanned systems. Personally, I started my own business doing hardware so I could stop going to those kinds of meetings. Nevertheless, it is important.
There is an IEEE WG: https://standards.ieee.org/findstds/standard/1872-2015.html (I have to admit to almost wetting my pants in laughter thinking about being charged $90+ for a data standard that only comes in PDF or print)
Popular open source ontology tool (some people dig this stuff): https://protege.stanford.edu/products.php
FFRDC tech paper: https://www.mitre.org/publications/technical-papers/towards-the-use-of-an-upper-ontology-for-us-government-and-military-domains-an-evaluation
Just for fun.. Facebook Graph data to RDF: http://www.semantic-web-journal.net/content/facebook-linked-data-graph-api
NoSQL stuff: http://blog.datagraph.org/2010/04/rdf-nosql-diff
ROS has a similar concept with URDF but I am not sure the goal there was sem web, but I suspect it could be leveraged (the choice of name for URDF is unfortunate as I think it has/will cause confusion): http://wiki.ros.org/urdf
While the context here is a gateway for data interop it can be applied to machine reasoning: https://www.openrobots.org/wiki/oro-ontology
EDIT Forgot NIEM. It really has come a long way since I messed with version 1. https://github.com/NIEM Need to look back into this as there may be some goodness to leverage. https://www.niem.gov/techhub/json
OWL, SPARQL, RDFA
cglusky
commented
6 years ago JSON-LD and Hydra
Challenge with JSON-LD is comms assumption. What happens in an austere/ad-hoc network that does not connect to internet. Or does so sporadically? Does it matter assuming a Gateway Module is likely most useful when there is web scale? Caching?
Speaking of caching http://manu.sporny.org/2016/json-ld-context-caching/
Really good background post (context is important). Also addresses some of the reasons RDF has languished.
http://manu.sporny.org/2014/json-ld-origins-2/
Edit Interesting paper on Hydra http://www.markus-lanthaler.com/research/hydra-a-vocabulary-for-hypermedia-driven-web-apis.pdf
Not to be confused with Hydra package for Node https://github.com/flywheelsports/hydra
cglusky
commented
6 years ago UAVCAN via UDP
cglusky
commented
6 years ago A rather well organized and apparently well funded effort in Europe to address just about everything we have talked about so far...
BTW, I got there from a ref in FastRTPS as they have a module (GE?) in FIWARE.
I have only spent a few minutes on the FIWARE site. It appears to be focused on the concept of integrating verticals in Smart Cities. Obviously as soon as you scope an architecture to support a city it becomes large and complex by default. FIWARE appears to be rather ambitious (and big). My past experience indicates that data sharing is more of a social issue than technical so would need more time to understand their effort. Either way it is a current reference, so perhaps some tech and lesson can be taken from FIWARE to support a smaller use case that is easier to define/test/measure?
Not a new use case but germane...
http://www.bbc.com/news/technology-39004805 https://www.landrover.com/experiences/news/pothole-detection.html https://appsforcommunities.devpost.com/submissions/4556-pothole-alert-app-report-potholes-with-gps
WIP
Maverick support for broader array of existing hardware and software while minimizing integration effort.
tl;dr Most of the below is me trying to get back up to speed on latest web dev tech and data interop, after ignoring most but the biggest trends for about a decade.
Keeping in mind what Maverick is...
Means we do not want to create a new project/code base - although we could, it is less desirable here and should likely become it's own project such as fnoop's vision landing - but find current tech that can be included in and configured as part of Maverick.
A module for Maverick included in the Maverick GCS Environment that provides a gateway in order to support interoperability between various protocols/systems in a heterogeneous sensor network.
Goal (WIP): Create a Maverick Module for the GCS Environment that will provide data interoperability by providing bridging between MAVLink and ROS with ???
This is not a new challenge. Large sums of time and money are spent in gov, public, private, healthcare sectors to attempt to provide data interoperability. DHS has an entire department dedicated to it.
From the OCF on IoT and Interop... https://openconnectivity.org/blog/language-interoperable-internet-things
"Why ROS2?" explains many of the same issues for distinct robot systems http://design.ros2.org/articles/why_ros2.html
Example Use Cases
UAS AED Response: Public space likely monitored by Public Safety. Also likely to have CCTV network. Feeds into ops center/dispatch someplace. They should not need a new piece of GCS software (at least on every view) but will need data to display as part of larger common operating picture. First Responder could use PTT radio with GPS to provide terminal guidance just by keying mic or pressing big red button an a phone app or texting a shortcode. But that assumes we have a gateway from voice/http back to MAVLink on the robot.
Gitter Coffee Cart: This one is your free million dollar idea for the day. Your service robot/coffee cart has a gitter nick. Simply ask Jeeves to bring you coffee via gitter. Google probably already has one.
Connect robots to operational and maintenance systems
Connect robots to various wide area network sensors such as weather and ADS-B
DJI and other silo API's???
RTK server. SNIP exists but there is something here for a gateway, just not sure what it is yet.
Integration with Open Drone Map and WebODM
Integration with IoT Dashboards
Assumptions:
Considerations
Choices (mixing protocols, API's and frameworks here - perhaps better in a table)
https://www.slideshare.net/paolopat/mqtt-iot-protocols-comparison
More Assumptions...
It may be more appropriate to assume that Maverick Gateway Module would be a level up from Robot/GCS system and may have one or more MAVLink/ROS based robots on the network.
MAVLink to ROS/ROS2 (MAVROS) is likely best place to start at center/robot(s) and diverge from there. Where is best place to bridge again if needed. MAVLink/ROS2 <<>> ?? bridge << >> Other Systems. Need diagrams.
Use ROS to provide middleware at system level since that is it's function and Maverick is already supporting ROS. ROS2 is using DDS which is perhaps more appropriate for building dedicated, hard/firm real time systems.
MQTT. Requires bridging with a broker. Here's a good comparison of MQTT to DDS
Robot Web Tools good example of bridging ROS to web tech and wider applicability
Multi-protocol (HTTP/MQTT/CoAP) gateway/server on node.js https://github.com/eclipse/ponte
PR for direct MQTT support in ArduPilot: https://github.com/ArduPilot/ardupilot/pull/6325
Example Implementation of MQTT-MAVLink Bridge https://github.com/Octanis1/Octanis1-Field-Station
GSOC 2016 Soletta Framework https://summerofcode.withgoogle.com/archive/2016/organizations/6477188102619136/
ROS - MQTT Bridge https://github.com/groove-x/mqtt_bridge
MAVROS https://github.com/mavlink/mavros/tree/master/mavros
Robot Web Tools (Web based ROS) http://robotwebtools.org/tools.html#systems
Node MAVLink https://github.com/omcaree/node-mavlink
Yarp http://www.yarp.it/index.html
OCF Iotivity https://openconnectivity.org/developer/reference-implementation/iotivity