Investigate switch to DMBS

[abcminiuser]

Since I've forked out a modified version of the LUFA build system as DMBS, it would be better maintenance-wise if I can integrate DMBS with some extra modifications in LUFA rather than having duplicated code.

[abcminiuser]

Experimental branch https://github.com/abcminiuser/lufa/commit/231658362eca0f882f1fc28642ffe160926461f1 has an initial working redirection/subclassing of DMBS. Requires more testing and documentation updates.

[NicoHood]

I'd also like to see this change. DMBS is the best avr "makefile" solution ive seen and its easy as hell. If Lufa would built ontop of this it might be easier to integrate it into a normal dmbs project etc.

Is there anything that I can help to make this work?

[abcminiuser]

Right now the DMBS branch is working and usable -- I've redirected all the LUFA modules to it to retain backwards compatibility (and add some LUFA-specific shims and modules). The conundrum I haven't had time to solve yet is documentation.

Currently the LUFA build system documentation is the comment blocks at the top of each makefile module, plus the hand-written documentation in the LUFA manual. Ideally I'd like to move this out to the DMBS project, where the documentation and modules can be kept together. However, I still want to include the documentation (however it's sourced and stored) inside the LUFA docs, since it will need to be expanded upon to document the LUFA specific additions.

Having multiple redundant versions of the documentation is something I'd like to avoid, but I haven't thought of a nice way to write one set of documentation in DMBS, then include and extend it in LUFA. I started with some simple Markdown docs for DMBS (since it doesn't need Doxygen) but the include system in Doxygen was a bit lacking.

Any ideas on how I could solve this to reduce maintenance while keeping the LUFA documentation in-sync with the DMBS version I drop into each release?

[abcminiuser]

(Thinking some more) - if nothing else, I can merge the DMBS branch into master, and update the hand-written LUFA docs to mention that it's based on DMBS. That doesn't solve any of the documentation issues and would still require a separate set of docs in DMBS, but would at least save me from having to keep both the LUFA build system and DMBS makefile modules in sync all the time.

[NicoHood]

I'd also say that merging DMBS into master would make sense if it completely works. You might want to try if the submodule works correct and add a note to recursively clone (and update) DMBS.

About the docs: It would be really nice to have a proper DMBS documentation. As I wrote in the linked issue above it was somehow hard to include several stuff. With some simple examples of some features that could be solved quite easy.

Then I'd just document that LUFA uses DMBS to build. If you add special modules into LUFA, i'd document them there, but keep the current DMBS docs separate. Since you are using git submodules with a tagged version (or git commit), you can just refere to this documentation. It will be shipped just with the DMBS download and is always suiting the correct git revision.

[NicoHood]

Any thoughts about this?

[NicoHood]

I found some diffs beteen lufa and dmbs build system (only hihglighted the diffs):

atprogram.mk

Lufa:
ATPROGRAM_PROGRAMMER ?= atmelice
.PHONY: atprogram atprogram-ee

DMBS:
ATPROGRAM_PROGRAMMER ?= jtagice3
.PHONY: $(DMBS_BUILD_TARGETS)

avrdude.mk

Lufa:
.PHONY: avrdude avrdude-ee

DMBS:
.PHONY: $(DMBS_BUILD_TARGETS)

core.mk

Lufa:
.PHONY: help list_modules list_targets list_mandatory list_optional list_provided list_macros

DMBS:
# Debugging; "make print-VARNAME" will output the variable VARNAME's value
print-%:
    @printf "%s = %s" $(@:print-%=%) $($(@:print-%=%))
.PHONY: $(DMBS_BUILD_TARGETS)

cppcheck.mk

Lufa:
.PHONY: cppcheck-config cppcheck

DMBS:
.PHONY: $(DMBS_BUILD_TARGETS)

dfu.mk

Lufa:
.PHONY: flip flip-ee dfu dfu-ee

DMBS:
.PHONY: $(DMBS_BUILD_TARGETS)

Doxygen also differs. Havent looked into detail into that.

hid.mk

Lufa:
.PHONY: hid hid-ee teensy teensy-ee

DMBS:
.PHONY: $(DMBS_BUILD_TARGETS)

source.mk is not available (sure thing).

Beside the Phony of build/gcc.mk I just want to post a picture. I think the default compiler flags differ. (Thatswhy my code compiled with bigger size under DMBS. The reason was the missing -fno-jump-tables).

I do not know what the thing about the PHONY. But those other changes should be considered to be equal between those two build systems.

[abcminiuser]

Ah hell, time keeps getting away from me - I've still got all your issues to sit down and look at, plus a billion emails that are starting to fester.

I need to pull the trigger on this; I want DMBS in LUFA to ease the parallel development burden. I'm going a source-dump route rather than submodule route as people seem to have trouble with submodules -- if in the future GitHub adds an option to download archives with the submodules expanded I'll switch over, but for now to ease support I'll just manually upgrade it as-needed.

The documentation issue is still up in the air. Right now I'm leaning towards markdown docs in DMBS for all the modules, which I can then render as a PDF or HTML for inclusion in LUFA.

Re: the differences you spotted: 1) The ATPROGRAM_PROGRAMMER default should be carried over to DMBS. It's the latest programmer from Atmel, so I'd like it to be the default.

2) The DMBS debug output target should have been in the LUFA build system, but I forgot. It's literally just a debug target for my own use however, so its omission isn't a real issue.

3) The minor differences in the GCC build modules are mostly just me tidying up the logic in DMBS, but the missing jump table makes bootloaders unstable under certain versions of avr-gcc. I'll add that.

4) All the .PHONY differences are fine, the DMBS version is better. The .PHONY directive lists certain targets as "magic" to make, so that it doesn't ignore them if a file exists with the same name. For example, without a .PHONY gcc directive, make would get confused if you had a file in your build directory called gcc. The DMBS way of re-using the module targets variable makes this cleaner and more maintainable than the older LUFA way of repeating the targets.

[abcminiuser]

@NicoHood: I've made a PR with the proposed changes; when you have a moment I'd love another set of eyes over it before merge.