search

nasa / fprime

F´ - A flight software and embedded systems framework
https://fprime.jpl.nasa.gov
Apache License 2.0
10.06k stars 1.31k forks source link

Porting to Platforms with Constrained Memory #81

Closed SterlingPeet closed 2 years ago

SterlingPeet commented 5 years ago

I am trying to port F Prime to one of the largest available parts in the ATmega family of embedded processors from Microchip (previously Atmel). With some modifications, @zeroping and I have managed to get it to compile all the expected frameworks parts, port objects, and components for the ArduinoBlink deployment in @LeStarch's fork.

The problem is that when we get to the linking step, it fails to link because it cannot find references to new and delete. These keywords are explicitly left out of the avr-g++ compiler support, because using them on many of the smaller supported processors is actually a mistake. e.g.: an ATtiny85 with 512 bytes of ram, or worse, an ATtiny10 with only 32 bytes of RAM! Not that we care about running F Prime on those targets, but the authors of the compiler do care, and made appropriate decisions about the built-in support.

Similarly, there is an avr-libc that is provided but STL, libc++, and libstdc++ are not. These issues are approachable by providing stubs or extern "C" statements for the Fw/Types/AVR directory. In fact, that is how I have gotten it to compile, needing cstring, cstdio, and cstdarg shims in the Types folder.

The following is a snippet of my [cmake build system] compilation output where the linker fails:

[100%] Linking CXX executable ../../../bin/avr-gcc/ArduinoBlink
cd /home/vagrant/src/ATmega-fprime/build-atmega/examples/ArduinoBlink/Top && /usr/bin/cmake -E cmake_link_script CMakeFiles/ArduinoBlink.dir/link.txt --verbose=1
/usr/bin/avr-g++  -g   CMakeFiles/ArduinoBlink.dir/Main.cpp.obj CMakeFiles/ArduinoBlink.dir/Topology.cpp.obj CMakeFiles/ArduinoBlink.dir/write.c.obj CMakeFiles/ArduinoBlink.dir/ArduinoTopologyAppAc.cpp.obj  -o ../../../bin/avr-gcc/ArduinoBlink ../../../lib/avr-gcc/libFw_Types.a ../../../lib/avr-gcc/libFw_Cfg.a ../../../lib/avr-gcc/libSvc_ActiveRateGroup.a ../../../lib/avr-gcc/libSvc_Health.a ../../../lib/avr-gcc/libSvc_Time.a ../../../lib/avr-gcc/libSvc_FatalHandler.a ../../../lib/avr-gcc/libSvc_GroundInterface.a ../../../lib/avr-gcc/libSvc_TlmChan.a ../../../lib/avr-gcc/libSvc_BufferManager.a ../../../lib/avr-gcc/libSvc_FileDownlink.a ../../../lib/avr-gcc/libSvc_ActiveLogger.a ../../../lib/avr-gcc/libSvc_AssertFatalAdapter.a ../../../lib/avr-gcc/libexamples_ArduinoGpsTracker_SerialDriver.a ../../../lib/avr-gcc/libSvc_CmdDispatcher.a ../../../lib/avr-gcc/libexamples_ArduinoGpsTracker_LedBlinker.a ../../../lib/avr-gcc/libSvc_FileUplink.a ../../../lib/avr-gcc/libexamples_ArduinoGpsTracker_HardwareRateDriver.a ../../../lib/avr-gcc/libSvc_PrmDb.a ../../../lib/avr-gcc/libSvc_RateGroupDriver.a ../../../lib/avr-gcc/libSvc_CmdSequencer.a ../../../lib/avr-gcc/libFw_Logger.a ../../../lib/avr-gcc/libSvc_GroundInterface.a ../../../lib/avr-gcc/libSvc_LinuxTime.a ../../../lib/avr-gcc/libOs.a ../../../lib/avr-gcc/libOs_Baremetal_TaskRunner.a ../../../lib/avr-gcc/libSvc_WatchDog.a ../../../lib/avr-gcc/libUtils_Types.a ../../../lib/avr-gcc/libSvc_Fatal.a ../../../lib/avr-gcc/libFw_FilePacket.a ../../../lib/avr-gcc/libFw_Buffer.a ../../../lib/avr-gcc/libCFDP_Checksum.a ../../../lib/avr-gcc/libFw_Prm.a ../../../lib/avr-gcc/libSvc_Cycle.a ../../../lib/avr-gcc/libFw_Tlm.a ../../../lib/avr-gcc/libFw_Log.a ../../../lib/avr-gcc/libSvc_Sched.a ../../../lib/avr-gcc/libSvc_Ping.a ../../../lib/avr-gcc/libFw_Cmd.a ../../../lib/avr-gcc/libFw_Com.a ../../../lib/avr-gcc/libSvc_Seq.a ../../../lib/avr-gcc/libSvc_Time.a ../../../lib/avr-gcc/libFw_Comp.a ../../../lib/avr-gcc/libOs.a ../../../lib/avr-gcc/libFw_Time.a ../../../lib/avr-gcc/libFw_Port.a ../../../lib/avr-gcc/libFw_Obj.a ../../../lib/avr-gcc/libFw_Logger.a ../../../lib/avr-gcc/libUtils_Hash.a ../../../lib/avr-gcc/libFw_Types.a ../../../lib/avr-gcc/libFw_Cfg.a 
/usr/lib/gcc/avr/5.4.0/../../../avr/bin/ld: ../../../bin/avr-gcc/ArduinoBlink section `.text' will not fit in region `text'
/usr/lib/gcc/avr/5.4.0/../../../avr/bin/ld: region `text' overflowed by 271484 bytes
../../../lib/avr-gcc/libFw_Types.a(Assert.cpp.obj): In function `Fw::defaultPrintAssert(signed char const*)':
/home/vagrant/src/ATmega-fprime/Fw/Types/Assert.cpp:23: undefined reference to `operator delete'
/home/vagrant/src/ATmega-fprime/Fw/Types/Assert.cpp:23: undefined reference to `operator delete'
../../../lib/avr-gcc/libSvc_ActiveLogger.a(ActiveLoggerImpl.cpp.obj): In function `Svc::ActiveLoggerImpl::~ActiveLoggerImpl()':
/home/vagrant/src/ATmega-fprime/Svc/ActiveLogger/ActiveLoggerImpl.cpp:59: undefined reference to `operator delete'
../../../lib/avr-gcc/libSvc_ActiveLogger.a(ActiveLoggerImpl.cpp.obj): In function `Svc::ActiveLoggerImpl::~ActiveLoggerImpl()':
/home/vagrant/src/ATmega-fprime/Svc/ActiveLogger/ActiveLoggerImpl.cpp:59: undefined reference to `operator delete'
../../../lib/avr-gcc/libSvc_CmdDispatcher.a(CommandDispatcherComponentAc.cpp.obj): In function `Svc::CommandDispatcherComponentBase::~CommandDispatcherComponentBase()':
/home/vagrant/src/ATmega-fprime/build-atmega/F-Prime/Svc/CmdDispatcher/CommandDispatcherComponentAc.cpp:730: undefined reference to `operator delete'

The Arduino solution for cross-platform code between AVR and ARM architectures is to implement new and delete in the Arduino library. It basically just thinly wraps the avr-libc implementations of malloc() and free():

void *operator new( size_t size ){
  return malloc( size );
}

void *operator new[]( size_t size ){
  return malloc( size );
}

void operator delete( void *ptr ){
  if( ptr )
    free( ptr );
}

void operator delete[]( void *ptr ){
  if( ptr )
    free( ptr );
}

So, clearly it is possible to find a place to plug this into the framework. However, @timcanham has also suggested that perhaps there are patterns we can leverage that avoid using malloc.

What is a good approach here? Hopefully we can use the results of this issue to enhance Issue #76.

timcanham commented 5 years ago

Huh - None of those files have explicit new and delete calls. Perhaps the C++ libraries carry an implicit dependency in the event they are created on the stack? Does the compiler possibly have arguments to trim them off?

timcanham commented 5 years ago

There are components (e.g. Svc::CmdSequencer) that want a buffer of memory to perform their function. The user can select a size based on their needs and resources.

We created a base class in Fw/Types/MemAllocator.hpp. The user writes a derived class that fetches memory from whatever pool they have using whatever mechanism manages it. F-prime provides a malloc-based derived class (Fw/Types/MallocAllocator.hpp) for use in systems that support malloc/free. It is not uncommon in embedded applications to have a region of user-managed memory. A derived class can manually allocate memory from this region. The identifier parameter has been used on previous JPL projects to look up an entry in a table of predefined memory blocks, otherwise it can be ignored.

LeStarch commented 5 years ago

I think the issue is the Pthreads queue implementation. It uses new internally, and back-up many of the Os-independent queue solutions we ship.

https://github.com/nasa/fprime/blob/0cab90e238cff1b50c20f1e148a44cf8827a5bf8/Os/Pthreads/FIFOBufferQueue.cpp#L36

LeStarch commented 5 years ago

Anywhere an active component is being used, a queue is allocating new memory. Thus, it requires calling into new.

timcanham commented 5 years ago

It's probably queued components as well since they create a queue too. If it's part of the porting layer, then the OS adaptation can remove the calls. That doesn't explain the Fw:Assert dependencies.

LeStarch commented 5 years ago

Yes, I agree. Any components using a queue will implicitly call new and delete in the creation of the queue.

LeStarch commented 5 years ago

We could build an Os-independent queue that is not dependent on dynamic memory, but it likely will waste memory.

LeStarch commented 5 years ago

This evening I will write-up my findings on memory constrained platforms.

timcanham commented 5 years ago

The other thing we could do is to add a parameter to the Queue::create call that takes a Fw::MemAllocator instance to allow the developer to provide for allocation.

SterlingPeet commented 5 years ago

OK, so I am guessing that I need to add some flags that look roughly like -Wall -g -Os -MMD -ffunction-sections -fdata-sections to the C++ flags field for the platform, to see if that helps with the new/delete issue

LeStarch commented 5 years ago

-Os reduces code size. That is important for the size-overflow you are getting.

-ffunction-sections
-fdata-sections

Add sections to allow for striping of unused functions and data. I don't see how this will help.

LeStarch commented 5 years ago 
-MMD

Creates dependency files.

SterlingPeet commented 5 years ago

@timcanham originally asked about trying to add an option to trim out unused new/delete, which was the purpose of trying out the -ffunction-sections option. Spoiler alert, didn't make the error go away.

LeStarch commented 5 years ago

I am not seeing that dependency in a Mac OSX build of the Fw::Assert object:

> objdump -t ./F-Prime/Fw/Types/CMakeFiles/Fw_Types.dir/Assert.o

0000000000000000         *UND*  __ZTVN10__cxxabiv117__class_type_infoE
0000000000000000         *UND*  __ZdlPv
0000000000000000         *UND*  ___assert_rtn
0000000000000000         *UND*  ___stack_chk_fail
0000000000000000         *UND*  ___stack_chk_guard
0000000000000000         *UND*  ___stderrp
0000000000000000         *UND*  _fprintf
0000000000000000         *UND*  _llvm_gcda_emit_arcs
0000000000000000         *UND*  _llvm_gcda_emit_function
0000000000000000         *UND*  _llvm_gcda_end_file
0000000000000000         *UND*  _llvm_gcda_start_file
0000000000000000         *UND*  _llvm_gcda_summary_info
0000000000000000         *UND*  _llvm_gcov_init
0000000000000000         *UND*  _snprintf
LeStarch commented 5 years ago

No trace of a delete symbol on ARM either.

SterlingPeet commented 5 years ago

I don't have a Assert.o file, but there is this:

objdump -t ./F-Prime/Fw/Types/CMakeFiles/Fw_Types.dir/Assert.cpp.obj 

./F-Prime/Fw/Types/CMakeFiles/Fw_Types.dir/Assert.cpp.obj:     file format elf32-little

SYMBOL TABLE:
00000000 l    df *ABS*  00000000 Assert.cpp
00000000 l    d  .text  00000000 .text
00000000 l    d  .data  00000000 .data
00000000 l    d  .bss   00000000 .bss
0000003e l       *ABS*  00000000 __SP_H__
0000003d l       *ABS*  00000000 __SP_L__
0000003f l       *ABS*  00000000 __SREG__
00000000 l       *ABS*  00000000 __tmp_reg__
00000001 l       *ABS*  00000000 __zero_reg__
00000000 l    d  .text._ZN2Fw10AssertHookD2Ev   00000000 .text._ZN2Fw10AssertHookD2Ev
00000000 l    d  .text._ZN2Fw10AssertHook8doAssertEv    00000000 .text._ZN2Fw10AssertHook8doAssertEv
00000000 l    d  .text._ZN2Fw10AssertHookD0Ev   00000000 .text._ZN2Fw10AssertHookD0Ev
00000000 l    d  .rodata.str1.1 00000000 .rodata.str1.1
00000000 l    d  .text._ZN2Fw18defaultPrintAssertEPKa   00000000 .text._ZN2Fw18defaultPrintAssertEPKa
00000000 l    d  .text._ZN2Fw10AssertHook11printAssertEPKa  00000000 .text._ZN2Fw10AssertHook11printAssertEPKa
00000000 l    d  .text._ZN2Fw19defaultReportAssertEPhjjmmmmmmPai    00000000 .text._ZN2Fw19defaultReportAssertEPhjjmmmmmmPai
00000000 l    d  .progmem.gcc_sw_table._ZN2Fw19defaultReportAssertEPhjjmmmmmmPai    00000000 .progmem.gcc_sw_table._ZN2Fw19defaultReportAssertEPhjjmmmmmmPai
00000000 l    d  .text._ZN2Fw10AssertHook12reportAssertEPhjjmmmmmm  00000000 .text._ZN2Fw10AssertHook12reportAssertEPhjjmmmmmm
00000000 l    d  .text._ZN2Fw10AssertHook12registerHookEv   00000000 .text._ZN2Fw10AssertHook12registerHookEv
00000000 l     O .bss._ZN2FwL12s_assertHookE    00000002 _ZN2FwL12s_assertHookE
00000000 l    d  .text._ZN2Fw8SwAssertEPhj  00000000 .text._ZN2Fw8SwAssertEPhj
00000000 l    d  .text._ZN2Fw8SwAssertEPhjm 00000000 .text._ZN2Fw8SwAssertEPhjm
00000000 l    d  .text._ZN2Fw8SwAssertEPhjmm    00000000 .text._ZN2Fw8SwAssertEPhjmm
00000000 l    d  .text._ZN2Fw8SwAssertEPhjmmm   00000000 .text._ZN2Fw8SwAssertEPhjmmm
00000000 l    d  .text._ZN2Fw8SwAssertEPhjmmmm  00000000 .text._ZN2Fw8SwAssertEPhjmmmm
00000000 l    d  .text._ZN2Fw8SwAssertEPhjmmmmm 00000000 .text._ZN2Fw8SwAssertEPhjmmmmm
00000000 l    d  .text._ZN2Fw8SwAssertEPhjmmmmmm    00000000 .text._ZN2Fw8SwAssertEPhjmmmmmm
00000000 l    d  .text.CAssert0 00000000 .text.CAssert0
00000000 l    d  .rodata._ZTVN2Fw10AssertHookE  00000000 .rodata._ZTVN2Fw10AssertHookE
00000000 l    d  .bss._ZN2FwL12s_assertHookE    00000000 .bss._ZN2FwL12s_assertHookE
00000000 l       .group 00000000 _ZN2Fw10AssertHookD5Ev
00000000 l    d  .stab  00000000 .stab
00000000 l    d  .stabstr   00000000 .stabstr
00000000 l    d  .comment   00000000 .comment
00000000 l    d  .avr.prop  00000000 .avr.prop
00000000 l    d  .group 00000000 .group
00000000 l    d  .group 00000000 .group
00000000  w    F .text._ZN2Fw10AssertHookD2Ev   00000002 _ZN2Fw10AssertHookD2Ev
00000000  w    F .text._ZN2Fw10AssertHookD2Ev   00000002 _ZN2Fw10AssertHookD1Ev
00000000 g     F .text._ZN2Fw10AssertHook8doAssertEv    00000002 _ZN2Fw10AssertHook8doAssertEv
00000000         *UND*  00000000 abort
00000000  w    F .text._ZN2Fw10AssertHookD0Ev   00000002 _ZN2Fw10AssertHookD0Ev
00000000         *UND*  00000000 _ZdlPv
00000000 g     F .text._ZN2Fw18defaultPrintAssertEPKa   00000028 _ZN2Fw18defaultPrintAssertEPKa
00000000         *UND*  00000000 __iob
00000000         *UND*  00000000 fprintf
00000000 g     F .text._ZN2Fw10AssertHook11printAssertEPKa  00000006 _ZN2Fw10AssertHook11printAssertEPKa
00000000 g     F .text._ZN2Fw19defaultReportAssertEPhjjmmmmmmPai    0000022e _ZN2Fw19defaultReportAssertEPhjjmmmmmmPai
00000000         *UND*  00000000 snprintf
00000000 g     F .text._ZN2Fw10AssertHook12reportAssertEPhjjmmmmmm  000001b4 _ZN2Fw10AssertHook12reportAssertEPhjjmmmmmm
00000000 g     F .text._ZN2Fw10AssertHook12registerHookEv   0000000a _ZN2Fw10AssertHook12registerHookEv
00000000 g     F .text._ZN2Fw8SwAssertEPhj  000000fc _ZN2Fw8SwAssertEPhj
00000000 g     F .text._ZN2Fw8SwAssertEPhjm 00000104 _ZN2Fw8SwAssertEPhjm
00000000 g     F .text._ZN2Fw8SwAssertEPhjmm    0000010c _ZN2Fw8SwAssertEPhjmm
00000000 g     F .text._ZN2Fw8SwAssertEPhjmmm   00000114 _ZN2Fw8SwAssertEPhjmmm
00000000 g     F .text._ZN2Fw8SwAssertEPhjmmmm  00000154 _ZN2Fw8SwAssertEPhjmmmm
00000000 g     F .text._ZN2Fw8SwAssertEPhjmmmmm 00000194 _ZN2Fw8SwAssertEPhjmmmmm
00000000 g     F .text._ZN2Fw8SwAssertEPhjmmmmmm    0000024c _ZN2Fw8SwAssertEPhjmmmmmm
00000000 g     F .text.CAssert0 000000ea CAssert0
00000000  w    O .rodata._ZTVN2Fw10AssertHookE  0000000e _ZTVN2Fw10AssertHookE
00000000         *UND*  00000000 __do_copy_data
00000000         *UND*  00000000 __do_clear_bss
timcanham commented 5 years ago

@SterlingPeet I don't see the new/delete undefined symbols.

SterlingPeet commented 5 years ago

I agree

SterlingPeet commented 5 years ago

I just realized that I was not targeting the correct AVR MCU instruction set, no idea what the default is. Here is the snippet:

[100%] Linking CXX executable ../../../bin/avr-gcc/ArduinoBlink
cd /home/vagrant/src/ATmega-fprime/build-atmega/examples/ArduinoBlink/Top && /usr/bin/cmake -E cmake_link_script CMakeFiles/ArduinoBlink.dir/link.txt --verbose=1
/usr/bin/avr-g++   -mmcu=atmega128a -Wall -Wextra -g -Os -MMD -ffunction-sections -fdata-sections -std=c++11 -g   CMakeFiles/ArduinoBlink.dir/Main.cpp.obj CMakeFiles/ArduinoBlink.dir/Topology.cpp.obj CMakeFiles/ArduinoBlink.dir/write.c.obj CMakeFiles/ArduinoBlink.dir/ArduinoTopologyAppAc.cpp.obj  -o ../../../bin/avr-gcc/ArduinoBlink ../../../lib/avr-gcc/libFw_Types.a ../../../lib/avr-gcc/libFw_Cfg.a ../../../lib/avr-gcc/libSvc_TlmChan.a ../../../lib/avr-gcc/libexamples_ArduinoGpsTracker_LedBlinker.a ../../../lib/avr-gcc/libSvc_Time.a ../../../lib/avr-gcc/libSvc_CmdDispatcher.a ../../../lib/avr-gcc/libSvc_GroundInterface.a ../../../lib/avr-gcc/libexamples_ArduinoGpsTracker_SerialDriver.a ../../../lib/avr-gcc/libSvc_FatalHandler.a ../../../lib/avr-gcc/libSvc_RateGroupDriver.a ../../../lib/avr-gcc/libSvc_ActiveRateGroup.a ../../../lib/avr-gcc/libSvc_FileDownlink.a ../../../lib/avr-gcc/libSvc_CmdSequencer.a ../../../lib/avr-gcc/libexamples_ArduinoGpsTracker_HardwareRateDriver.a ../../../lib/avr-gcc/libSvc_ActiveLogger.a ../../../lib/avr-gcc/libSvc_BufferManager.a ../../../lib/avr-gcc/libSvc_PrmDb.a ../../../lib/avr-gcc/libSvc_FileUplink.a ../../../lib/avr-gcc/libSvc_AssertFatalAdapter.a ../../../lib/avr-gcc/libSvc_Health.a ../../../lib/avr-gcc/libFw_Logger.a ../../../lib/avr-gcc/libSvc_GroundInterface.a ../../../lib/avr-gcc/libSvc_LinuxTime.a ../../../lib/avr-gcc/libOs.a ../../../lib/avr-gcc/libOs_Baremetal_TaskRunner.a ../../../lib/avr-gcc/libUtils_Types.a ../../../lib/avr-gcc/libSvc_Seq.a ../../../lib/avr-gcc/libSvc_Cycle.a ../../../lib/avr-gcc/libSvc_Fatal.a ../../../lib/avr-gcc/libFw_Prm.a ../../../lib/avr-gcc/libFw_FilePacket.a ../../../lib/avr-gcc/libFw_Buffer.a ../../../lib/avr-gcc/libCFDP_Checksum.a ../../../lib/avr-gcc/libFw_Tlm.a ../../../lib/avr-gcc/libSvc_Ping.a ../../../lib/avr-gcc/libSvc_Sched.a ../../../lib/avr-gcc/libFw_Log.a ../../../lib/avr-gcc/libFw_Cmd.a ../../../lib/avr-gcc/libFw_Com.a ../../../lib/avr-gcc/libSvc_WatchDog.a ../../../lib/avr-gcc/libSvc_Time.a ../../../lib/avr-gcc/libFw_Comp.a ../../../lib/avr-gcc/libOs.a ../../../lib/avr-gcc/libFw_Time.a ../../../lib/avr-gcc/libFw_Port.a ../../../lib/avr-gcc/libFw_Obj.a ../../../lib/avr-gcc/libFw_Logger.a ../../../lib/avr-gcc/libUtils_Hash.a ../../../lib/avr-gcc/libFw_Types.a ../../../lib/avr-gcc/libFw_Cfg.a 
/usr/lib/gcc/avr/5.4.0/../../../avr/bin/ld: ../../../bin/avr-gcc/ArduinoBlink section `.data' will not fit in region `text'
/usr/lib/gcc/avr/5.4.0/../../../avr/bin/ld: region `text' overflowed by 2942 bytes
CMakeFiles/ArduinoBlink.dir/Main.cpp.obj: In function `_GLOBAL__sub_D_assert':
/home/vagrant/src/ATmega-fprime/examples/ArduinoBlink/Top/Main.cpp:10: undefined reference to `Arduino::SERIAL_PORT'
/home/vagrant/src/ATmega-fprime/examples/ArduinoBlink/Top/Main.cpp:10: undefined reference to `Arduino::SERIAL_PORT'
CMakeFiles/ArduinoBlink.dir/Topology.cpp.obj: In function `_GLOBAL__sub_D_rateGroupDriverComp':
/home/vagrant/src/ATmega-fprime/examples/ArduinoBlink/Top/Topology.cpp:13: undefined reference to `Arduino::LedBlinkerComponentImpl::init(int)'
/home/vagrant/src/ATmega-fprime/examples/ArduinoBlink/Top/Topology.cpp:13: undefined reference to `Arduino::SerialDriverComponentImpl::init(int, unsigned int)'
/home/vagrant/src/ATmega-fprime/examples/ArduinoBlink/Top/Topology.cpp:13: undefined reference to `Arduino::HardwareRateDriver::start()'
../../../lib/avr-gcc/libFw_Types.a(Assert.cpp.obj): In function `CAssert0':
/home/vagrant/src/ATmega-fprime/Fw/Types/Assert.cpp:314: undefined reference to `operator delete(void*)'
../../../lib/avr-gcc/libFw_Types.a(EightyCharString.cpp.obj): In function `Fw::EightyCharString::terminate(unsigned int)':
/home/vagrant/src/ATmega-fprime/Fw/Types/EightyCharString.cpp:75: undefined reference to `operator delete(void*)'
../../../lib/avr-gcc/libFw_Types.a(Serializable.cpp.obj): In function `Fw::ExternalSerializeBuffer::clear()':
/home/vagrant/src/ATmega-fprime/Fw/Types/Serializable.cpp:740: undefined reference to `operator delete(void*)'
/home/vagrant/src/ATmega-fprime/Fw/Types/Serializable.cpp:740: undefined reference to `operator delete(void*)'
/home/vagrant/src/ATmega-fprime/Fw/Types/Serializable.cpp:740: undefined reference to `operator delete(void*)'
/home/vagrant/src/ATmega-fprime/Fw/Types/Serializable.cpp:740: undefined reference to `__cxa_pure_virtual'
/home/vagrant/src/ATmega-fprime/Fw/Types/Serializable.cpp:741: undefined reference to `__cxa_pure_virtual'
/home/vagrant/src/ATmega-fprime/Fw/Types/Serializable.cpp:741: undefined reference to `__cxa_pure_virtual'
/home/vagrant/src/ATmega-fprime/Fw/Types/Serializable.cpp:741: undefined reference to `__cxa_pure_virtual'
/home/vagrant/src/ATmega-fprime/Fw/Types/Serializable.cpp:741: undefined reference to `__cxa_pure_virtual'
../../../lib/avr-gcc/libFw_Types.a(StringType.cpp.obj): In function `Fw::StringBase::operator+=(Fw::StringBase const&)':
/home/vagrant/src/ATmega-fprime/Fw/Types/StringType.cpp:35: undefined reference to `operator delete(void*)'
/home/vagrant/src/ATmega-fprime/Fw/Types/StringType.cpp:34: undefined reference to `__cxa_pure_virtual'
/home/vagrant/src/ATmega-fprime/Fw/Types/StringType.cpp:34: undefined reference to `__cxa_pure_virtual'
/home/vagrant/src/ATmega-fprime/Fw/Types/StringType.cpp:34: undefined reference to `__cxa_pure_virtual'
/home/vagrant/src/ATmega-fprime/Fw/Types/StringType.cpp:35: undefined reference to `__cxa_pure_virtual'
/home/vagrant/src/ATmega-fprime/Fw/Types/StringType.cpp:35: undefined reference to `__cxa_pure_virtual'
../../../lib/avr-gcc/libFw_Types.a(StringType.cpp.obj):/home/vagrant/src/ATmega-fprime/Fw/Types/StringType.cpp:35: more undefined references to `__cxa_pure_virtual' follow

This pushed the missing Arduino-specific functionality to the top, but then still complains about the missing delete operators lower down. At this point, it probably makes sense to adjust the strategy from compiling for plain AVR ATmega to the Arduino Mega platform with all the Arduino libs linked in.

@LeStarch's ArduinoBlink deployment is the closest thing to a sane deployment for me, and it means pushing some of the platform lib development down the road in favor of proving it can or cannot work, soon.

LeStarch commented 5 years ago

@SterlingPeet here is my write-up with advice for memory constrain platforms. Any help you can give me in fleshing out this guide would be appreciated.

https://github.com/LeStarch/fprime/wiki/F%C2%B4-on-Constrained-Memory-Devices

SterlingPeet commented 5 years ago

@LeStarch: Excellent resource, and I certainly expect to have things to add. I'm currently targeting the Arduino Mega 2560. It has 256k of program memory, so that matches up with your Teensy, but 8kb of RAM. So it looks like RAM is going to be the exciting challenge.

From there I will try to move to a MCU that has half of both values. I am currently working on compiling the Arduino core as a library. My first whack at it is basically translating your Teensyduino toolchain file.

Speaking of Arduino toolchain files, are you familiar with the arduino-cmake project?

LeStarch commented 5 years ago

Glad I could help. I used that cmake file to build a few Arduino projects, but it fundamentally does not support ARM. Thus I wrote my own.

I used the same pattern I thought they were using, compile the core libs, then add it as a dependency to my project.

One note: they compile the Arduino 2nd Order libs, which are built on the framework but the same code is supported across all Arduino. I did not build those libs, as I just wanted to core framework.

SterlingPeet commented 5 years ago

OK, I got all the Arduino core stuff to compile and get added to the other targets. Its currently struggling with the Arduino serial ComLogger component.

SterlingPeet commented 5 years ago

@timcanham and @LeStarch: I got the the Arduino SDK based build all the way down to the end of the compiler step, and it still fails to link with the undefined reference to operator delete problem.

make VERBOSE=1

[ 99%] Linking CXX executable ../../../bin/ArduinoMega/ArduinoBlink
cd /home/vagrant/src/fprime/build-mega/examples/ArduinoBlink/Top && /usr/bin/cmake -E cmake_link_script CMakeFiles/ArduinoBlink.dir/link.txt --verbose=1
/opt/arduino-1.8.9/hardware/tools/avr/bin/avr-g++  -mmcu=atmega2560  -Wall -g -Os -MMD -ffunction-sections -fdata-sections -DF_CPU=16000000 -DARDUINO=10805 -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -felide-constructors -Wno-error=narrowing -g  -mmcu=atmega2560   CMakeFiles/ArduinoBlink.dir/Main.cpp.obj CMakeFiles/ArduinoBlink.dir/Topology.cpp.obj CMakeFiles/ArduinoBlink.dir/write.c.obj CMakeFiles/ArduinoBlink.dir/ArduinoTopologyAppAc.cpp.obj  -o ../../../bin/ArduinoMega/ArduinoBlink ../../../lib/ArduinoMega/libFw_Types.a ../../../lib/ArduinoMega/libFw_Cfg.a ../../../lib/ArduinoMega/libSvc_PrmDb.a ../../../lib/ArduinoMega/libSvc_ActiveLogger.a ../../../lib/ArduinoMega/libSvc_CmdDispatcher.a ../../../lib/ArduinoMega/libSvc_CmdSequencer.a ../../../lib/ArduinoMega/libSvc_FatalHandler.a ../../../lib/ArduinoMega/libSvc_ActiveRateGroup.a ../../../lib/ArduinoMega/libSvc_TlmChan.a ../../../lib/ArduinoMega/libSvc_Health.a ../../../lib/ArduinoMega/libSvc_Time.a ../../../lib/ArduinoMega/libSvc_AssertFatalAdapter.a ../../../lib/ArduinoMega/libSvc_FileUplink.a ../../../lib/ArduinoMega/libexamples_ArduinoGpsTracker_HardwareRateDriver.a ../../../lib/ArduinoMega/libSvc_GroundInterface.a ../../../lib/ArduinoMega/libSvc_BufferManager.a ../../../lib/ArduinoMega/libexamples_ArduinoGpsTracker_LedBlinker.a ../../../lib/ArduinoMega/libSvc_RateGroupDriver.a ../../../lib/ArduinoMega/libexamples_ArduinoGpsTracker_SerialDriver.a ../../../lib/ArduinoMega/libSvc_FileDownlink.a ../../../lib/ArduinoMega/libFw_Logger.a ../../../lib/ArduinoMega/libSvc_GroundInterface.a ../../../lib/ArduinoMega/libSvc_LinuxTime.a ../../../lib/ArduinoMega/libOs.a ../../../lib/ArduinoMega/libOs_Baremetal_TaskRunner.a ../../../lib/ArduinoMega/libarduinocore.a -lm ../../../lib/ArduinoMega/libFw_Prm.a ../../../lib/ArduinoMega/libSvc_Seq.a ../../../lib/ArduinoMega/libSvc_Fatal.a ../../../lib/ArduinoMega/libSvc_WatchDog.a ../../../lib/ArduinoMega/libUtils_Types.a ../../../lib/ArduinoMega/libSvc_Cycle.a ../../../lib/ArduinoMega/libSvc_Sched.a ../../../lib/ArduinoMega/libFw_Log.a ../../../lib/ArduinoMega/libFw_Cmd.a ../../../lib/ArduinoMega/libSvc_Ping.a ../../../lib/ArduinoMega/libFw_Tlm.a ../../../lib/ArduinoMega/libFw_Com.a ../../../lib/ArduinoMega/libFw_FilePacket.a ../../../lib/ArduinoMega/libFw_Buffer.a ../../../lib/ArduinoMega/libCFDP_Checksum.a ../../../lib/ArduinoMega/libSvc_Time.a ../../../lib/ArduinoMega/libFw_Comp.a ../../../lib/ArduinoMega/libOs.a ../../../lib/ArduinoMega/libFw_Time.a ../../../lib/ArduinoMega/libFw_Port.a ../../../lib/ArduinoMega/libFw_Obj.a ../../../lib/ArduinoMega/libFw_Logger.a ../../../lib/ArduinoMega/libUtils_Hash.a ../../../lib/ArduinoMega/libFw_Types.a ../../../lib/ArduinoMega/libFw_Cfg.a ../../../lib/ArduinoMega/libarduinocore.a -lm 
../../../lib/ArduinoMega/libFw_Types.a(Assert.cpp.obj): In function `Fw::AssertHook::~AssertHook()':
/home/vagrant/src/fprime/cmake/../Fw/Types/Assert.hpp:46: undefined reference to `operator delete(void*, unsigned int)'
../../../lib/ArduinoMega/libFw_Types.a(EightyCharString.cpp.obj): In function `Fw::EightyCharString::~EightyCharString()':
/home/vagrant/src/fprime/Fw/Types/EightyCharString.cpp:28: undefined reference to `operator delete(void*, unsigned int)'
../../../lib/ArduinoMega/libFw_Types.a(Serializable.cpp.obj): In function `Fw::ExternalSerializeBuffer::~ExternalSerializeBuffer()':
/home/vagrant/src/fprime/cmake/../Fw/Types/Serializable.hpp:153: undefined reference to `operator delete(void*, unsigned int)'
../../../lib/ArduinoMega/libFw_Types.a(Serializable.cpp.obj): In function `Fw::Serializable::~Serializable()':
/home/vagrant/src/fprime/Fw/Types/Serializable.cpp:20: undefined reference to `operator delete(void*, unsigned int)'
../../../lib/ArduinoMega/libFw_Types.a(Serializable.cpp.obj): In function `Fw::SerializeBufferBase::~SerializeBufferBase()':
/home/vagrant/src/fprime/Fw/Types/Serializable.cpp:46: undefined reference to `operator delete(void*, unsigned int)'
../../../lib/ArduinoMega/libFw_Types.a(StringType.cpp.obj):/home/vagrant/src/fprime/Fw/Types/StringType.cpp:27: more undefined references to `operator delete(void*, unsigned int)' follow
collect2: error: ld returned 1 exit status
examples/ArduinoBlink/Top/CMakeFiles/ArduinoBlink.dir/build.make:239: recipe for target 'bin/ArduinoMega/ArduinoBlink' failed
make[2]: *** [bin/ArduinoMega/ArduinoBlink] Error 1
make[2]: Leaving directory '/home/vagrant/src/fprime/build-mega'
CMakeFiles/Makefile2:4742: recipe for target 'examples/ArduinoBlink/Top/CMakeFiles/ArduinoBlink.dir/all' failed
make[1]: *** [examples/ArduinoBlink/Top/CMakeFiles/ArduinoBlink.dir/all] Error 2
make[1]: Leaving directory '/home/vagrant/src/fprime/build-mega'
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

I have made a branch with my edits to get here over on my fork.

SterlingPeet commented 5 years ago

I will say that the linker line [in cmake] in @LeStarch's branch also has a "fixme" note on it, I'm wondering if I need to look into that

SterlingPeet commented 5 years ago

The specific problem seems to be that operator delete(void*, unsigned int) is not provided for my platform, but operator delete(void*) is provided [via added source, not built-in to the compiler].

SterlingPeet commented 5 years ago

Oh guess what, it links with -std=gnu++11, but not with -std=gnu++14

LeStarch commented 5 years ago

Interesting. I just replicated the error, but it seems you can provide that implementation, or compile via gnu++11

SterlingPeet commented 5 years ago

What does the unsigned int do? I would need to know, in order to implement something sane. Otherwise I could just assume that calling these destructors means we are about to reboot, in which case they can basically be useless stubs.

SterlingPeet commented 5 years ago

@LeStarch: I went through and (somewhat blindly) adjusted the values in your linked wiki page. Here is the current status:

[ 99%] Linking CXX executable ../../../bin/ArduinoMega/ArduinoBlink
   text    data     bss     dec     hex filename
 156056    9638   23052  188746   2e14a /home/vagrant/src/fprime/build-mega/bin/ArduinoMega/ArduinoBlink

~32K is only barely going to fit in the memory, but its OK with the expansion up to 64k of SRAM. 156K of program memory is a little over the 128K allowable. Next thought is to start building my own slim topology with slim components.

SterlingPeet commented 4 years ago

Is there a convenient way to strip out the executable information for objects that aren't getting used on a small platform like this?

i.e.: There is only one "thread" on this bare metal platform, so the health pinger is not very useful. I have removed it from the topology, but it still shows up in the objdump as taking up program memory space.

LeStarch commented 4 years ago

Some questions:

  1. Have you purged your CMake build directory? It is possible something is caching poorly. If this helps, let me know and I will file a bug on the CMake setup.

  2. You might try the graphiviz cmake flags to see a dep graph: https://github.com/openenclave/openenclave/issues/1055

SterlingPeet commented 4 years ago

Purging seems to have no effect. I'll try the graphviz dep graph when I can get to it, but that seems to be taking longer than I hoped.

SterlingPeet commented 4 years ago

@LeStarch : I found the optimization flag that gets my ArduinoBlink deployment below 8k of static memory: Link Time Optimization -flto.

So, between this and getting the C++ standards sorted out, it seems that this platform is a viable option for small deployments. I have added 64k of SRAM using the XMEM interface on the ATmega2560. I can confirm that deployments requiring larger amounts RAM than the chip provides onboard do actually work.

My platform support still depends on the Arduino libraries and the Teensy timer, so I still need to excise those to feel like its ready for primetime. The current work will be pushed to my Github repo shortly.

The next concern is how to organize and contribute the platform support, as well as documentation strategies and requirements. This includes both documentation for adding a new platform, and documentation for this platform I have just ported. Is this still appropriate for this issue, or should I be opening a (or multiple) new ones?

LeStarch commented 4 years ago

Ok, first off: thanks for the link there. That is an awesome article. Also, it isn't just new C/C++ programmers. Experienced engineers can fall apart too, as it is hard to be vigilant all the time.

As for contributions, code etc should be contributed to fprime-community, as Arduino is outside the scope of core F´, however; the documentation should be contributed to core as it tells how to add new platforms.

Does this approach make sense?

I am happy to continue using this issue to continue discussing.

SterlingPeet commented 4 years ago

Ok cool! I am happy to contribute the platform port to wherever makes sense.

First question: are we able to use F´ as a library yet? I can see ways to set up a project that would make pulling in F´ and F´-community elements much easier to manage for 3rd party projects.

I do expect that some things will be easy to separate out, and other things might be more challenging. Setting the StandardTypes.hpp for a platform comes to mind quickly but what about porting a component to a platform that has different needs, like LinuxTime? Or, how about cmake considerations that come up like this line from LeStarch teensyduino:

if(NOT ${CMAKE_SYSTEM_NAME} STREQUAL "Teensyduino")
    add_fprime_subdirectory("${CMAKE_CURRENT_LIST_DIR}/SocketGndIf/")
ENDIF()

I expect it is going to be a process to get 3rd party stuff integrated in a clean way, so this is partly food for thought.

One more item! I have looked into how Arduino support works, and I don't think I want to try and port F´ into the Arduino environment in any official way. I would rather create a support package for deploying F´ onto any of the ATmega series chips that support the XMEM interface. This keeps me from needing to attempt to write drivers and libraries that support multiple architectures, which feels more appropriate for flight-ready code. From this direction, I could also provide reference hardware designs with the XMEM hardware baked in, which could totally come with Arduino board definitions, so people (especially students familiar with Arduino) can prototype in the Arduino IDE and port working code over to F´. Extra bonus, the F´ support package for ATmega parts can easily lose the Arduino dependency all-together (but it has been great for bootstrapping the support package).

LeStarch commented 4 years ago

As of now, F´ cannot operate as an external library. There are major changes that need to happen in the autocoder.

That is mostly how I structured my Teensy code, as I didn't fully implement all of Arduino, just the one chip family supporting F´ using the Arduino framework as a crutch.

LeStarch commented 4 years ago

See #76 as a super-issue

LeStarch commented 2 years ago

@SterlingPeet I am going to close this issue. We got the porting guide done, and worked through the initial implementation. Now we are off elsewhere. Reopen if there is more this issue is needed for.

SterlingPeet commented 2 years ago

I think the new arduino-cli based library is the thing that actually closes this issue. Its not quite done yet, correct?

SterlingPeet commented 2 years ago

In re-reading the ticket, I think we can track that development in other places and leave this issue closed

LeStarch commented 2 years ago

@SterlingPeet my thoughts exactly.

capsulecorplab commented 2 years ago

Is there documentation for cross compiling the system reference for an Arduino or Teensy board? If not, I don't understand why this issue is closed

SterlingPeet commented 2 years ago

@capsulecorplab There is not, although GT-1 has launched and de-orbited using F Prime 1.3 as the FSW baseline. So that proves it is/was possible. @LeStarch and I are working on a FPP/F Prime 3+ library that will get pushed to The F Prime Community Repository as soon as it works.

That should be Any Day Now:tm:.

So, since we are tracking that work elsewhere, and it is not happening within the core F Prime repository, it makes sense to close this issue. Which we are. We will announce the availability of the code and the instructions on the normal discussions location for this repository.

capsulecorplab commented 2 years ago

@SterlingPeet got it. Let me know if you need a guinea pig to help test and/or document that effort :slightly_smiling_face:

SterlingPeet commented 2 years ago

Excellent, and don't worry, we will want the help!