Python configuration script for Makefile & xboot.h

[GoogleCodeExporter]

I've started writing a quick little Python script that can be used to configure 
XBoot.  I'm going for a quick and dirty terminal version.  Later this may lead 
up to a GUI version.  Python is nice because it's very cross-platform 
compatible.

I've checked out the most recent version of the code and am working off the 
settings available in that version of xboot.h.  Will it be OK to commit my 
changes, including the new Python file, back up to the repository for review 
and inspection?

Original issue reported on code.google.com by b.jamin....@gmail.com on 8 Sep 2010 at 12:29

[GoogleCodeExporter]

Yep, that's fine.  You should create a development branch for working on the 
python script.  Just download the whole source tree, strip the metadata, rename 
it, and check it back in.  Then do your work in there.  When it's done, all the 
changes can be merged back into the main source tree.

Also, I think we need to keep a GUI-less version available.  It's very handy 
for those people out there who prefer doing things manually.  

I'm thinking the best method for dealing with settings is to create some sort 
of settings definition file, maybe in XML format.  It would define all the 
settings, a little description blurb, data type and range/enumeration of 
acceptable values, perhaps even chip-specific stuff like what UART pins are 
available on what devices.  Then all your script needs to do is read in the XML 
file, configure all the settings, and generate xboot.h on the fly.  The main 
point of that method is then your program doesn't have to deal with weird 
changes in xboot.h and it can adapt with little or no changes to 
added/removed/changed configuration values through different versions of xboot. 
 It can also be used to generate a pretty UI on the fly with tool tips/context 
sensitive help that stays in sync with xboot.  How does that sound to you?  I'm 
thinking it's very doable and would help minimize code redundancy 
significantly.  

The actual settings for xboot.h could come from a user-generated conf file.  
The script would read in the conf file and the xml file, then spit out xboot.h. 
 It could be called from the makefile.  And you could create a separate wizard 
style program to generate the conf file in the file in the first place.  
Another advantage to having a separate conf file is people don't have to redo 
xboot.h manually every time they grab a different version of xboot, all they 
have to do is rerun the program to generate it based on the new xml file and 
their custom conf file.  

If you want, I can try to figure out a good format for the xml file.  I'm 
thinking xml would be easiest since it's pretty easy to read in a regular text 
editor and there are lots of tools for parsing xml files.  I'm definitely open 
to suggestions, though.  

Original comment by alex.for...@gmail.com on 8 Sep 2010 at 8:25

[GoogleCodeExporter]

Yeah, that all sounds good.  I kind of brute forced it - It will output a valid 
xboot.h file and compile, but I haven't tested all setting entries or actually 
loading the compiled program yet.

Yeah, I put the GUI aside for now and went with a terminal program.  I 
committed it last night into the same folder as the rest, I will make a 
development branch before the next commit (unless you beat me to it).  Maybe if 
I get ambitious I might use the Python curses library also and make it a little 
fancier terminal program.

I think it would be good to restructure xboot.h into a little more logical 
flow.  It works fine and logically when editing by hand, but not so much when 
trying to output with a program.  The settings should all be next to each other 
in the middle, so the program can output a static header, dynamic settings 
content, and a static footer and be done with it instead of having static and 
dynamic sections interspersed throughout the file.

So doing this with XML would be great, and there are definitely a ton of Python 
XML parsers so I should figure it out instead of brute forcing things.  If it 
was set up so there was a header & footer section with large chunks of xboot.h, 
then all the settings along with how their configuration options and output 
format, that would work pretty well programmatically.

We'll also likely need to do the same for the Makefile, as there are a couple 
of things in it that we'll need to change to make this a complete configuration 
tool (F_CPU, target chip, and programmer name).

Feel free to try out the program I've committed, let me know what you think.  
I'm sure if we used XML, it would be a lot easier to make option lists for the 
user to select and improve the feel of the program.  But this at least lines 
things out and lists what settings need to be configured.

Original comment by b.jamin....@gmail.com on 8 Sep 2010 at 3:50

[GoogleCodeExporter]

I took a peek at it.  Definitely brute force, but that's usually the way I 
start these things too.  Do you want me to work on an XML format for the setup? 
 I think it would be possible to come up with a format flexible enough to 
generate any format of output needed.  Also, I think it's possible for a 
makefile to import other makefiles.  Perhaps all we need to do is spit out a 
makefile chunk that it can include to overwrite the default variables.  This 
might be a good solution because the makefile really needs more flexibility 
than the header file since it has to interface with the programmer as well.  
And the autoconfig program only needs to modify a handful of variables.  I 
suppose I will have to do some brushing up on my makefile skills.  I really 
haven't written vary many and I usually do it mostly through trial and error 
because I haven't yet been able to find some really good documentation that 
isn't impossible to follow.  

Original comment by alex.for...@gmail.com on 8 Sep 2010 at 3:57

[GoogleCodeExporter]

Heh heh ... yeah, brute forcing it makes you feel big for a little while and 
then stupid for a long while ;)

If you wanted to create the XML format, that would be awesome.  I'll rewrite 
the program to use it once you're ready.  Let me know.

I think the Makefile import would be perfect for all the reasons you mentioned. 
 I always borrow my makefiles and then fake that I know what I'm doing when I 
edit them :D

Original comment by b.jamin....@gmail.com on 8 Sep 2010 at 4:24

[GoogleCodeExporter]

Too true.  Actually, the main XBoot makefile was borrowed from a different 
source.  I just modified it so it would compile for XMegas.  I actually can't 
recall the original source at the moment.  It might have been some other random 
bootloader for regular atmega chips.  

I will work on the xml format then.  The file format has been stewing for a 
while already so it shouldn't take all that long to get a workable file format 
out of it.  One quick question though - would it be better if I used parameters 
or sub tags?  e.g. <tag param="value" /> vs. <tag><param>value</param></tag>.  
Which do you think would be easier to parse?  I'm probably just going to write 
the main xml file from scratch, so it would be good to know which would be 
easier so I don't have to change it later.  If it doesn't matter I will 
probably use a few parameters here and there.  

Original comment by alex.for...@gmail.com on 8 Sep 2010 at 5:19

[GoogleCodeExporter]

I've never really tried writing any XML, so I'm not sure.  It appears to me 
that the off-the-shelf XML parsers for Python can handle either method.  The 
following link shows some examples of what & how it can parse, maybe following 
similar structures would be useful.

http://oreilly.com/catalog/pythonxml/chapter/ch01.html

Original comment by b.jamin....@gmail.com on 8 Sep 2010 at 6:00

[GoogleCodeExporter]

I committed what I have so far for xboot.xml.  It's definitely not complete 
yet, but all of the configuration setting types are represented so far.  What 
needs to be done to store all the settings and relations needed isn't 
completely trivial so parsing will be slightly complicated.  I wrote it to be 
as extensible as possible and with a GUI style interface in mind.  It's 
definitely possible to make changes, though, if you have any ideas.  

Also, XML is really easy to understand.  If you've ever used HTML before, well, 
HTML is simply a specific variety of XML.  

Original comment by alex.for...@gmail.com on 9 Sep 2010 at 4:51

[GoogleCodeExporter]

I committed another version of xboot.xml.  It should be pretty much complete.  
I made a few changes to the structure from the last revision, be sure to check 
it out if you haven't already.  Also, I added the code necessary to deal with a 
makefile 'chunk' that could get spit out by a configuration script.  Look at 
the makefile and the bottom of the XML file to see how that would work.  It's 
quite straightforward.  

Original comment by alex.for...@gmail.com on 10 Sep 2010 at 8:29

[GoogleCodeExporter]

I just added xboot-sample-config.xml.  My idea is to have two programs: one 
that reads in xboot.xml and optionally a file like xboot-sample-config.xml, 
presents a GUI/wizard/what-have-you to the user to fill in all the appropriate 
settings, then outputs a file similar to xboot-sample-config.xml.  Then the 
other program reads in xboot.xml and the smaller file and generates the output 
files xboot.h and xboot-config.mk.  How does that sound to you?  It would be 
possible to combine the two in one program, but having the intermediate file 
adds more flexibility.  It separates the user-specific settings from the 
specific distribution of xboot so they can update xboot and recompile without 
having to enter all of their settings in again.  

Original comment by alex.for...@gmail.com on 10 Sep 2010 at 9:19

[GoogleCodeExporter]

Makefile updated to streamline configuration and decouple configuration 
information from the xboot sources while being very easy to maintain.  Separate 
python script no longer necessary.  

Original comment by alex.for...@gmail.com on 30 Jan 2012 at 1:16

• Changed state: WontFix