FLASH space analysis

[bkueng]

I analyzed the FLASH space usage on the v2 target, mostly with a focus on strings. These are the results:

Symbols & Source files

Output of bloaty, we've seen this before (via bloaty -s vm -d compileunits nuttx_px4fmu-v2_default.elf -n 25):

     VM SIZE                                                                        FILE SIZE
 --------------                                                                  --------------
  53.4%   443Ki [867 Others]                                                      8.75Mi  78.7%
   6.6%  55.0Ki ROMFS/nsh_romfsimg.c                                              55.4Ki   0.5%
   6.4%  52.8Ki [section .text]                                                   52.8Ki   0.5%
   2.7%  22.7Ki ../../src/lib/ecl/EKF/covariance.cpp                              61.5Ki   0.5%
   2.6%  21.2Ki ../../src/modules/mavlink/mavlink_messages.cpp                     285Ki   2.5%
   2.2%  18.6Ki ../../src/modules/mc_pos_control/mc_pos_control_main.cpp           144Ki   1.3%
   2.2%  17.9Ki ../../src/modules/ekf2/ekf2_main.cpp                               352Ki   3.1%
   2.1%  17.2Ki ../../src/modules/commander/commander.cpp                          116Ki   1.0%
   1.9%  16.0Ki [LOAD [RWX]]                                                      16.0Ki   0.1%
   1.9%  15.9Ki ../../src/modules/mavlink/mavlink_receiver.cpp                     135Ki   1.2%
   1.7%  14.5Ki ../../src/lib/ecl/EKF/mag_fusion.cpp                              58.6Ki   0.5%
   1.6%  13.2Ki ../../src/modules/fw_pos_control_l1/FixedwingPositionControl.cpp   138Ki   1.2%
   1.4%  11.5Ki src/modules/systemlib/param/px4_parameters.c                      25.3Ki   0.2%
   1.4%  11.3Ki ../../src/lib/ecl/EKF/ekf_helper.cpp                               101Ki   0.9%
   1.2%  10.1Ki ../../src/drivers/px4io/px4io.cpp                                 63.2Ki   0.6%
   1.2%  10.0Ki ../../src/lib/ecl/EKF/control.cpp                                 76.2Ki   0.7%
   1.2%  9.62Ki ../../src/modules/mavlink/mavlink_main.cpp                         113Ki   1.0%
   1.1%  9.16Ki ../../src/modules/mc_att_control/mc_att_control_main.cpp           164Ki   1.4%

Output of nm, sorted by size (nm -C --print-size --size-sort --radix=d nuttx_px4fmu-v2_default.elf):

134745305 00002700 T Ekf::fuseAirspeed()
134888617 00002744 T __ieee754_pow
134359929 00003336 T GPSDriverAshtech::handleMessage(int)
134693965 00003480 T MulticopterPositionControl::control_auto()
134298057 00003516 T __dtoa
134543813 00003664 T Mavlink::task_main(int, char**)
134653537 00003820 T Ekf2::Ekf2()
134653537 00003820 T Ekf2::Ekf2()
134827009 00005174 T Ekf::fuseDrag()
134847045 00005618 T Ekf::fuseSideslip()
134644889 00006252 T Ekf2::run()
134796869 00008318 T Ekf::fuseOptFlow()
134493369 00009104 T Commander::run()
134833521 00011356 T Ekf::fuseMag()
135033448 00011764 R px4_parameters
134750177 00021108 T Ekf::predictCovariance()
134977096 00056320 T romfs_img

Strings

The following shows the iterative reduction of the binary size when removing more and more strings:

                    file size,     size reduced by
--------------------------------------------------
- full:             1021.0kb
- airframes:         987.2kb       33.8kb
- mixers:            967.8kb       19.4kb
- ROMFS (rest: scripts):944.2kb    23.6kb
  -> total romfs strings: 76.8 (plus 57.3kb for the IO firmware binary)
     IO FW binary                  57.3kb
- PX4_WARN:          924.2kb       20.0kb
- PX4_INFO:          910.5kb       13.7kb
- PX4_ERR:           886.4kb       24.1kb
- PRINT_MODULE_*:    875.2kb       11.2kb
- printf:            853.8kb       21.4kb
  - most of these are module help & status outputs, some errors & warnings
- mavlink log:       833.7kb       20.1kb
- uorb message format: 815.2kb     18.5kb

Observations

• EKF2 is huge: 121KB
• mavlink messages are quite large too
• user-facing strings (mavlink log) make only a small portion (20.1KB)

Possible Actions

• add a MINIMAL_BUILD define, which removes INFO, PRINT_MODULE, printf (reduction: 46kb). It can also disable some mavlink message streams.
• modularize ekf? For example disable optical flow on some targets. Though we'll need to be more careful that all configurations still work.
• compress the ROMFS / reduce its size in another way. For example:
  • reduce multiple consecutive spaces into a single space for mixers: #9457
  • many mixers contain a line like: 'O: 10000 10000 0 -10000 10000'. It accounts for about 4kb. We can remove these and make these values the default if the parser does not find the 'O: ' line.
• my understanding is that most boards flashed with v2 could support 2MB of flash, but they still have an older bootloader. We could update bootloader and flash v3 on newer boards. We actually have most things already in place to make this easy for a user:
  • QGC can push the bootloader binary via FTP to the SD card.
  • then use the bl_update command to flash the BL. I already tested this on a pixhawk mini and was able to flash v3 afterwards.
  • the bl_update command can be directly executed from QGC via mavlink shell (automatically), or by adding logic to the ROMFS that does it on the next reboot when the BL binary is found.

[dagar]

• ROMFS compression was proposed a couple years ago, but never merged. https://github.com/PX4/Firmware/pull/656
• upstream NuttX added cromfs support a couple months ago. https://groups.google.com/forum/#!searchin/nuttx/cromfs%7Csort:date/nuttx/rF32tHdGzWY/542TkS_9CAAJ

[dagar]

There are some relatively easy (small) savings in the ROMFS. Nearly identical mixers and airframe configurations can be merged. The init scripts could be refactored so that there isn't anything board specific. For example each board could provide an rc.sensors. Here's the full listing from px4fmu-v2.

0    NSHInitVol           [0xffffffff, 0xffffffff] 37777777777, sz     0, at 0x0     
1    .                    [0x10301   , 0x17a58a  ] 0040755, sz     0, at 0x20    
1    ..                   [0x10301   , 0x17a589  ] 0040755, sz     0, at 0x40     [link to 0x20    ]
1    mixers               [0x10301   , 0x17c585  ] 0040755, sz     0, at 0x60    
2    FMU_pass.mix         [0x10301   , 0x169e87  ] 0100664, sz   356, at 0x80    
2    dodeca_bottom_cox.aux.mix [0x10301   , 0x169eee  ] 0100664, sz    25, at 0x210   
2    IO_pass.main.mix     [0x10301   , 0x169ef4  ] 0100664, sz   712, at 0x260   
2    firefly6.aux.mix     [0x10301   , 0x169ef6  ] 0100664, sz   440, at 0x560   
2    Viper.main.mix       [0x10301   , 0x169ef9  ] 0100664, sz   621, at 0x750   
2    AAERTWF.main.mix     [0x10301   , 0x169c0c  ] 0100664, sz   889, at 0x9e0   
2    hexa_cox.main.mix    [0x10301   , 0x169f05  ] 0100664, sz    26, at 0xd90   
2    tri_y_yaw-.main.mix  [0x10301   , 0x169e8b  ] 0100664, sz   115, at 0xde0   
2    quad_+_vtol.main.mix [0x10301   , 0x169e96  ] 0100664, sz   466, at 0xe90   
2    octo_cox_w.main.mix  [0x10301   , 0x169f06  ] 0100664, sz    27, at 0x10a0  
2    tri_y_yaw+.main.mix  [0x10301   , 0x169f07  ] 0100664, sz   115, at 0x10f0  
2    mount.aux.mix        [0x10301   , 0x169f09  ] 0100664, sz   356, at 0x11a0  
2    blade130.main.mix    [0x10301   , 0x169f0a  ] 0100664, sz   305, at 0x1330  
2    caipirinha_vtol.main.mix [0x10301   , 0x169f0c  ] 0100664, sz   294, at 0x14a0  
2    .                    [0x10301   , 0x17c585  ] 0040755, sz     0, at 0x1600   [link to 0x60    ]
2    octo_x.main.mix      [0x10301   , 0x169f0e  ] 0100664, sz    26, at 0x1620  
2    coax.main.mix        [0x10301   , 0x169f0f  ] 0100664, sz   444, at 0x1660  
2    hexa_x.main.mix      [0x10301   , 0x169f10  ] 0100664, sz    26, at 0x1840  
2    fw_generic_wing.main.mix [0x10301   , 0x169f11  ] 0100664, sz   354, at 0x1880  
2    AETRFG.main.mix      [0x10301   , 0x169f12  ] 0100664, sz   570, at 0x1a20  
2    delta.main.mix       [0x10301   , 0x169f13  ] 0100664, sz   354, at 0x1c80  
2    ocpoc_quad_x.main.mix [0x10301   , 0x169f14  ] 0100664, sz    29, at 0x1e10  
2    stampede.main.mix    [0x10301   , 0x169f16  ] 0100664, sz   188, at 0x1e60  
2    deltaquad.main.mix   [0x10301   , 0x169f17  ] 0100664, sz   465, at 0x1f50  
2    AERT.main.mix        [0x10301   , 0x169f18  ] 0100664, sz   496, at 0x2160  
2    hexa_+.main.mix      [0x10301   , 0x169f19  ] 0100664, sz    26, at 0x2370  
2    quad_+.main.mix      [0x10301   , 0x169f1b  ] 0100664, sz   204, at 0x23b0  
2    quad_x_vtol.main.mix [0x10301   , 0x169f1f  ] 0100664, sz   371, at 0x24a0  
2    octo_cox.main.mix    [0x10301   , 0x169f20  ] 0100664, sz    26, at 0x2650  
2    vtol_delta.aux.mix   [0x10301   , 0x169f21  ] 0100664, sz   529, at 0x26a0  
2    octo_+.main.mix      [0x10301   , 0x169f23  ] 0100664, sz    26, at 0x28f0  
2    claire.main.mix      [0x10301   , 0x162713  ] 0100664, sz    26, at 0x2930  
2    ..                   [0x10301   , 0x17a58a  ] 0040755, sz     0, at 0x2970   [link to 0x20    ]
2    claire.aux.mix       [0x10301   , 0x169f24  ] 0100664, sz   351, at 0x2990  
2    FX79.main.mix        [0x10301   , 0x169f25  ] 0100664, sz   354, at 0x2b10  
2    quad_w.main.mix      [0x10301   , 0x169f26  ] 0100664, sz   204, at 0x2ca0  
2    quad_s250aq.main.mix [0x10301   , 0x169f27  ] 0100664, sz   204, at 0x2d90  
2    vtol_AAERT.aux.mix   [0x10301   , 0x169f2a  ] 0100664, sz   445, at 0x2e90  
2    vtol_convergence.main.mix [0x10301   , 0x169f2b  ] 0100664, sz   553, at 0x3080  
2    mount_legs.aux.mix   [0x10301   , 0x169f2c  ] 0100664, sz   445, at 0x32e0  
2    pass.aux.mix         [0x10301   , 0x169f2e  ] 0100664, sz   356, at 0x34d0  
2    wingwing.main.mix    [0x10301   , 0x169f2f  ] 0100664, sz   354, at 0x3660  
2    dodeca_top_cox.main.mix [0x10301   , 0x169f30  ] 0100664, sz    25, at 0x3800  
2    quad_dc.main.mix     [0x10301   , 0x169efd  ] 0100664, sz    27, at 0x3850  
2    vtol_AAVVT.aux.mix   [0x10301   , 0x169f32  ] 0100664, sz   613, at 0x38a0  
2    CCPM.main.mix        [0x10301   , 0x169ed1  ] 0100664, sz   671, at 0x3b40  
2    phantom.main.mix     [0x10301   , 0x169efa  ] 0100664, sz   354, at 0x3e00  
2    firefly6.main.mix    [0x10301   , 0x169f34  ] 0100664, sz    26, at 0x3fa0  
2    quad_x_can.main.mix  [0x10301   , 0x169f37  ] 0100664, sz    26, at 0x3ff0  
2    caipi.main.mix       [0x10301   , 0x169f39  ] 0100664, sz   354, at 0x4040  
2    ugv_generic.main.mix [0x10301   , 0x169f3a  ] 0100664, sz   188, at 0x41d0  
2    quad_h.main.mix      [0x10301   , 0x169f3b  ] 0100664, sz   204, at 0x42c0  
2    quad_x.main.mix      [0x10301   , 0x169f3d  ] 0100664, sz   204, at 0x43b0  
2    quad_x_vtol_sim.main.mix [0x10301   , 0x169f3f  ] 0100664, sz   299, at 0x44a0  
2    AAVVTWFF.main.mix    [0x10301   , 0x169f40  ] 0100664, sz   880, at 0x4600  
1    init.d               [0x10301   , 0x17a58c  ] 0040755, sz     0, at 0x49a0  
2    13008_QuadRanger     [0x10301   , 0x169f41  ] 0100664, sz  1250, at 0x49c0  
2    rc.autostart         [0x10301   , 0x16237f  ] 0100644, sz  4123, at 0x4ee0  
2    rc.axialracing_ax10_apps [0x10301   , 0x169f42  ] 0100664, sz    11, at 0x5f20  
2    4040_reaper          [0x10301   , 0x169f43  ] 0100664, sz   583, at 0x5f60  
2    3034_fx79            [0x10301   , 0x169f44  ] 0100664, sz   187, at 0x61d0  
2    10015_tbs_discovery  [0x10301   , 0x169f45  ] 0100664, sz   428, at 0x62b0  
2    14001_tri_y_yaw+     [0x10301   , 0x169f46  ] 0100664, sz    68, at 0x6490  
2    rc.ugv_defaults      [0x10301   , 0x169f47  ] 0100664, sz   286, at 0x6510  
2    1000_rc_fw_easystar.hil [0x10301   , 0x169f49  ] 0100664, sz   664, at 0x6650  
2    4031_3dr_quad        [0x10301   , 0x169f4b  ] 0100664, sz   306, at 0x6920  
2    rc.mc_defaults       [0x10301   , 0x169f4c  ] 0100664, sz   294, at 0x6a80  
2    13000_generic_vtol_standard [0x10301   , 0x169f4d  ] 0100664, sz   242, at 0x6bd0  
2    4012_quad_x_can      [0x10301   , 0x169f4e  ] 0100664, sz   438, at 0x6d00  
2    3033_wingwing        [0x10301   , 0x169f4f  ] 0100664, sz   560, at 0x6ee0  
2    14002_tri_y_yaw-     [0x10301   , 0x169f52  ] 0100664, sz    68, at 0x7130  
2    4060_dji_matrice_100 [0x10301   , 0x169f53  ] 0100664, sz   414, at 0x71b0  
2    2106_albatross       [0x10301   , 0x169f54  ] 0100664, sz   390, at 0x7380  
2    4050_generic_250     [0x10301   , 0x169f55  ] 0100664, sz   765, at 0x7530  
2    13007_vtol_AAVVT_quad [0x10301   , 0x169f56  ] 0100664, sz   851, at 0x7860  
2    .                    [0x10301   , 0x17a58c  ] 0040755, sz     0, at 0x7bf0   [link to 0x49a0  ]
2    3037_parrot_disco_mod [0x10301   , 0x169f58  ] 0100664, sz   613, at 0x7c10  
2    rc.axialracing_ax10_defaults [0x10301   , 0x169f59  ] 0100664, sz   159, at 0x7eb0  
2    4009_qav250          [0x10301   , 0x169f5a  ] 0100664, sz   609, at 0x7f80  
2    rc.fw_defaults       [0x10301   , 0x169f5b  ] 0100664, sz   610, at 0x8210  
2    5001_quad_+          [0x10301   , 0x169f5c  ] 0100664, sz    64, at 0x84a0  
2    10017_steadidrone_qu4d [0x10301   , 0x169f5d  ] 0100664, sz   475, at 0x8500  
2    13013_deltaquad      [0x10301   , 0x169f5e  ] 0100664, sz  2425, at 0x8710  
2    3031_phantom         [0x10301   , 0x169f5f  ] 0100664, sz   591, at 0x90b0  
2    10016_3dr_iris       [0x10301   , 0x169f60  ] 0100664, sz   489, at 0x9320  
2    1001_rc_quad_x.hil   [0x10301   , 0x169f61  ] 0100664, sz    48, at 0x9530  
2    3036_pigeon          [0x10301   , 0x169f62  ] 0100664, sz   645, at 0x9590  
2    24001_dodeca_cox     [0x10301   , 0x169f63  ] 0100664, sz   435, at 0x9840  
2    rc.vtol_defaults     [0x10301   , 0x169f65  ] 0100664, sz   322, at 0x9a30  
2    3032_skywalker_x5    [0x10301   , 0x169f67  ] 0100664, sz   492, at 0x9bb0  
2    rc.interface         [0x10301   , 0x169f6a  ] 0100664, sz  4397, at 0x9dd0  
2    rc.vtol_apps         [0x10301   , 0x169f6b  ] 0100664, sz   550, at 0xaf20  
2    15001_coax_heli      [0x10301   , 0x169f6c  ] 0100664, sz   763, at 0xb170  
2    3100_tbs_caipirinha  [0x10301   , 0x169f6d  ] 0100664, sz  1084, at 0xb490  
2    13003_quad_tailsitter [0x10301   , 0x169f71  ] 0100664, sz   256, at 0xb900  
2    3000_generic_wing    [0x10301   , 0x169f73  ] 0100664, sz    87, at 0xba30  
2    13002_firefly6       [0x10301   , 0x169f74  ] 0100664, sz   789, at 0xbac0  
2    16001_helicopter     [0x10301   , 0x169f75  ] 0100664, sz   798, at 0xbe00  
2    4002_quad_x_mount    [0x10301   , 0x169f77  ] 0100664, sz   127, at 0xc150  
2    4011_dji_f450        [0x10301   , 0x169f79  ] 0100664, sz   413, at 0xc200  
2    13005_vtol_AAERT_quad [0x10301   , 0x169f7b  ] 0100664, sz  1121, at 0xc3c0  
2    13012_convergence    [0x10301   , 0x169f7c  ] 0100664, sz  1490, at 0xc860  
2    rcS                  [0x10301   , 0x169f7d  ] 0100664, sz 11988, at 0xce70  
2    ..                   [0x10301   , 0x17a58a  ] 0040755, sz     0, at 0xfd70   [link to 0x20    ]
2    6001_hexa_x          [0x10301   , 0x169f7f  ] 0100664, sz    68, at 0xfd90  
2    rc.fw_apps           [0x10301   , 0x169f83  ] 0100664, sz   244, at 0xfe00  
2    12002_steadidrone_mavrik [0x10301   , 0x169f84  ] 0100664, sz   637, at 0xff20  
2    3035_viper           [0x10301   , 0x169f86  ] 0100664, sz    46, at 0x101d0 
2    7001_hexa_+          [0x10301   , 0x169f87  ] 0100664, sz    68, at 0x10220 
2    13006_vtol_standard_delta [0x10301   , 0x169f88  ] 0100664, sz   927, at 0x10290 
2    rc.sensors           [0x10301   , 0x169f89  ] 0100664, sz  3760, at 0x10660 
2    rc.io                [0x10301   , 0x169f8b  ] 0100664, sz    73, at 0x11530 
2    12001_octo_cox       [0x10301   , 0x169f8c  ] 0100664, sz    70, at 0x115a0 
2    10018_tbs_endurance  [0x10301   , 0x169f8f  ] 0100664, sz   527, at 0x11610 
2    11001_hexa_cox       [0x10301   , 0x169f90  ] 0100664, sz    70, at 0x11850 
2    9001_octo_+          [0x10301   , 0x169f91  ] 0100664, sz    68, at 0x118c0 
2    2105_maja            [0x10301   , 0x169f94  ] 0100664, sz   389, at 0x11930 
2    13009_vtol_spt_ranger [0x10301   , 0x169f95  ] 0100664, sz  1834, at 0x11ae0 
2    4010_dji_f330        [0x10301   , 0x169f99  ] 0100664, sz   413, at 0x12240 
2    13001_caipirinha_vtol [0x10301   , 0x169f9b  ] 0100664, sz   678, at 0x12400 
2    4004_H4_680mm        [0x10301   , 0x169f9f  ] 0100664, sz   178, at 0x126e0 
2    rc.ugv_apps          [0x10301   , 0x169fa1  ] 0100664, sz    79, at 0x127c0 
2    2100_standard_plane  [0x10301   , 0x169fa4  ] 0100664, sz    83, at 0x12830 
2    8001_octo_x          [0x10301   , 0x169fa7  ] 0100664, sz    68, at 0x128c0 
2    rc.mc_apps           [0x10301   , 0x169fa8  ] 0100664, sz   489, at 0x12930 
2    3030_io_camflyer     [0x10301   , 0x169fa9  ] 0100664, sz   601, at 0x12b40 
2    4001_quad_x          [0x10301   , 0x169faa  ] 0100664, sz    64, at 0x12dd0 
2    rc.logging           [0x10301   , 0x169fac  ] 0100664, sz   354, at 0x12e30 
1    extras               [0x10301   , 0x17a58b  ] 0040755, sz     0, at 0x12fc0 
2    .                    [0x10301   , 0x17a58b  ] 0040755, sz     0, at 0x12fe0  [link to 0x12fc0 ]
2    px4io-v2.bin         [0x10301   , 0x162596  ] 0100755, sz 56356, at 0x13000 
2    ..                   [0x10301   , 0x17a58a  ] 0040755, sz     0, at 0x20c50  [link to 0x20    ]

[dagar]

• the generated rc.autostart is 4kB, I'm sure there's a better way to do this.

    if param compare SYS_AUTOSTART 15001
    then
    sh /etc/init.d/15001_coax_heli
    fi
    if param compare SYS_AUTOSTART 24001
    then
    sh /etc/init.d/24001_dodeca_cox
    fi

• parameters (11.5Ki px4_parameters) could be stored by data type. This would remove the param_type per parameter.
  • We'd then be able to have an actual boolean param type that only requires 1 bit instead of 4 bytes for the default value.
  • instead of setting volatile per parameter we could instead have a (very small) list of volatile parameters

[thomasgubler]

At the dev-call 2018-07-04 the flash space issue on fmuv2 was discussed. Another possibility for a short term solution to unblock development came up:

1) fix cross module params 2) change cmake for fmuv2 to do 3 separate builds for FW, Copter, VTOL and activate in CI system --> development is unblocked 3) without changes to qgc: master for 2 out of 3 vehicle types could not be flashed out of QGC GUI. The thinking was that this could be acceptable as a short term solution

But, overall we need to discuss if creating separating releases for vehicle types is something we want at all because it breaks fundamentally with the model we followed so far.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

Closing as stale.