search

mavlink / qgroundcontrol

Cross-platform ground control station for drones (Android, iOS, Mac OS, Linux, Windows)
http://qgroundcontrol.io
3.21k stars 3.55k forks source link

Downloadable JSon file for firmware listings #2797

Open DonLakeFlyer opened 8 years ago

DonLakeFlyer commented 8 years ago

Currently the urls to the various firmware versions are hardcoded into QGC. Also QGC needs to jump through hoops to determine things like the versions of those firmware files. It would be much better if these were dynamically determined at runtime instead of hardcoded in. I'm proposing using a standard json file format to represent the available firmware versions. These json files could then be downloaded from multiple places such that QGC understands the full set available. The combination of all files would represent the full list of firmware available tto QGC. This gets QGC out of the business of understanding build system locations and custom parsing logic for different firmware providers.

Below is the proposed format. Keys which are shows as "foo|bar" represent the options which can be put into the key. I'm hoping the keys are self explanatory.

Each individual firmware entry is represented like this:

{
"version": 1,

"stable": [
{
"boardType": "PX4FMUV1|PX4FMUV2|PX4FMUV4|AeroCore",
"stack": "PX4|ArduPilot",
"vehicleType": "Default|Quad|X8|Hexa|Octo|Y|Y6|Heli|Plane|Rover",
"version": "#.#.##",
"url": "http://foo.px4|hex"
},
...
],

"olderVersions": [
{
"boardType": "PX4FMUV1|PX4FMUV2|PX4FMUV4|AeroCore"
"stack": "PX4|ArduPilot",
"releaseType": "Stable|Beta|Developer",
"vehicleType": "Default|Quad|X8|Hexa|Octo|Y|Y6|Heli|Plane|Rover",
"version": "#.#.##",
"url": "http://foo.px4|hex"
},
...
]
}

At this point the important thing is the set of information required to identify the firmware a versions. There are many ways this could be represented in a json hierarchy. I'm open to just about anything that has the needed set of information in it.

Note: There is nothing here that is QGC specific. This should work for any ground station.

DonLakeFlyer commented 8 years ago

Yeah, I agree. I think only going one version back is the way to go. Then if they want something older they can go digging themselves and install via Custom file. Working on a new format that simplifies that concept.

LorenzMeier commented 8 years ago

@DonLakeFlyer Ping me when you've converged format-wise.

LorenzMeier commented 8 years ago

@dagar Can you ping me in Gitter or Skype so we can discuss how to best generate the JSON file? I'd like to get this set up as soon as possible as part of our effort to raise awareness with our user base which release they're using.

dagar commented 8 years ago

@LorenzMeier If the current format is acceptable for PX4 (with only < 100 entries at the moment) I believe I have all the pieces to get this working on travis-ci today.

LorenzMeier commented 8 years ago

@dagar An initial cut would be fantastic. Even if we were to learn and adjust the format just having the plumbing in place would be extremely helpful and probably enable me or Don to do fine-tuning where required.

dagar commented 8 years ago

Alright I think I'm generating the manifest properly, I just need to get the details right. @DonLakeFlyer can you confirm a few things?

"mav-firmware-version-type" : "DEV" | "BETA" | "STABLE" "platform" : "PX4-v1" | "PX4-v2" | "PX4-v4" | - what about aerocore, mindpx? "latest" : - does it need to be set to 1 for latest dev/beta/stable?

Are 'mav-firmware-version' and major/minor/patch optional? Does it matter if I skip them for master, and sometimes fill in for beta if there's an appropriate tag?

How should I handle release candidates? Just 'mav-firmware-version': 'v1.0.0-rc4'.

I'll probably just do a one time fill of existing stable versioned releases, and then let travis-ci populate the rest.

DonLakeFlyer commented 8 years ago

"mav-firmware-version-type" Matches FIRMWARE_VERSION_TYPE from headers. So DEV, BETA, OFFICIAL

"platform" Need to make up naming for aerocore...

"latest" Correct, set to 1 for latest DEV, BETA, OFFICIAL

Are 'mav-firmware-version' and major/minor/patch optional? mav-firmware-version is the one that is used. The reset are not currently looked at. The value should be in the format #.#.#. No extra stuff around it.

DonLakeFlyer commented 8 years ago

Now that Flash shows PX4 Stable version number I'm moving this fancier version of support to 3.1

DonLakeFlyer commented 8 years ago

Ok, time to pick this up again.

@peterbarker I think the format a the file is good. But what I really need is a manifest file that only contains the versions you want to expose to the user to select from in a ground station as opposed to every verion ever built. Is that possible?

@dagar If you can get me a file I can download for PX4 I'll switch it to use that instead of the fixed list.

dagar commented 8 years ago

@DonLakeFlyer What about making multiple branches available as dev versions (#4024)? Where should I put the branch name?

Can you summarize what you see as the final format (at least so far)?

LorenzMeier commented 7 years ago

@dagar @DonLakeFlyer We really need this now from a user perspective. It would be great to have a stab at it.

dagar commented 7 years ago

I got a bit lost with the history here. @DonLakeFlyer can you verify the final format?

@LorenzMeier How much do we want to store + expose on the PX4 side? Everything tagged is kept on s3 and available indefinitely. All branches (master/beta/etc) only the latest version?

LorenzMeier commented 7 years ago

I think the last version will initially do. Eventually the last 20 or so (or all until it breaks).

dagar commented 7 years ago

Might as well aim for what you ultimately want.

@DonLakeFlyer single massive json for all px4 builds, or can we have multiple?

LorenzMeier commented 7 years ago

We just have 50 builds right now, very manageable. Let's shoot for the one massive and then trim things down.

mrpollo commented 5 years ago

An excellent conversation, I think we should revive this initiative based on the hot mess that is FirmwareUpgradeController

I had a brief conversation with @DonLakeFlyer, and he's interested in defining a standard schema that would allow QGC to keep their firmware upgrade logic under control.

I'll bring this up for discussion in the next MAVLink call to check for cross-community interest, and we can follow-up in this issue (or a new one)

peterbarker commented 5 years ago

On Tue, 20 Nov 2018, Ramon Roche wrote:

An excellent conversation, I think we should revive this initiative based on the hot mess that is FirmwareUpgradeController

I had a brief conversation with @DonLakeFlyer, and he's interested in defining a standard schema that would allow QGC to keep their firmware upgrade logic under control.

I'll bring this up for discussion in the next MAVLink call to check for cross-community interest, and we can follow-up in this issue (or a new one)

We've published http://firmware.ap.ardupilot.org/manifest.json.gz

for quite some time now.

Peter

dogmaphobic commented 5 years ago

We've published http://firmware.ap.ardupilot.org/manifest.json.gz for quite some time now.

Would this suffice? We would adopt the same schema across the board. It doesn't make sense to have different schemas per firmware. While at it, is this redundant or has it been replaced: https://github.com/ArduPilot/binary/tree/master/Firmware

peterbarker commented 5 years ago

On Tue, 20 Nov 2018, Gus Grubba wrote:

Would this suffice? We would adopt the same schema across the board. It doesn't make sense to have different schemas per firmware. While at it, is this redundant or has it been replaced: https://github.com/ArduPilot/binary/tree/master/Firmware

Kind of redundant. I believe Michael Oborne uses a semi-scripted approach to filling that repository in for Mission Planner to use.

DonLakeFlyer commented 5 years ago

https://github.com/ArduPilot/binary/tree/master/Firmware

Looks like it only has Stable builds in there. Also don't see Chibios?

DonLakeFlyer commented 4 years ago

I've got ArduPilot support for this with their firmware manifest file. But no PX4 support yet.