Traceability to source

[whaleygeek]

It would be really helpful to be able to trace an arbitrary running version of MicroPython on an arbitrary micro:bit, back to the github source that generated it.

This would be especially useful when diagnosing problems that users have, to say 'tell me the version string that you get (when you type this in) or (when you press the reset button), as anyone could then look through github for that version and see if a particular feature or bug-fix is, or is not, implemented.

Also, it's quite important to be able to setup an environment when reproducing issues, to be using the same version of micro:bit MicroPython as others are using. So, if there is an easy(ish) process by which that identifying information can be followed to a cached released .hex that could just be downloaded and used to reproduce the issue, that would additionally help.

The startup message that you get when you press the reset button seems to identify the root MicroPython version, but not the actual build of the micro:bit version that was based on that code.

[carlosperate]

Also, as I was mentioning in https://github.com/bbcmicrobit/micropython/issues/199#issuecomment-256902093 it would be ideal if we could identify the version from the hex file itself.

[whaleygeek]

Yes @carlosperate that would be a great facility too - someone could email a .hex file of an 'errant script' and a simple tool could be used to work out the version string and therefore the exact build of micropython for microbit that generated the interpreter inside it.

[whaleygeek]

It sounds like we need the magic of @ntoll to help us work up an initial spec for this, as that is what normally makes these things happen in the end!

[dpgeorge]

What we need is:

• proper tags in this repository for each version we release
• the tag exposed to the user, in the start-up banner and in os.uname()
• the git hash from this repo that was used to build the firmware exposed to the user
• the upstream micropython version and git hash exposed to the user

[markshannon]

IMO providing git hash, tags and other such artifacts can be fiddly and confusing. If we just updated the version number more often, it should solve @whaleygeek's problem. Ideally a new version of the flash would be uploaded to mu only when we add a new version. Version 1.1 now?

[carlosperate]

Not to sound like a broken record, but having the version in the hex file in a known and/or identifiable location would also help things like the online editor to chose the correct version of micropython when a user hex file is loaded. So, if future features reduce the memory available for user programs, old scripts that were on that limit could still be loaded, edited and run.

[dpgeorge]

IMO providing git hash, tags and other such artifacts can be fiddly and confusing.

But they are meaningful and useful. One should be able to identify the exact commit used to build their firmware, and we can't update the version each time we make a new commit. Upstream uPy uses version and git hash and it's proven to be extremely useful in helping users with bug reports.

Users are always going to be trying the latest and greatest version (not waiting for a tagged release) and we need a way for them to tell what version they used.

Not to sound like a broken record, but having the version in the hex file in a known and/or identifiable location would also help

It's a nice idea but it's really difficult to do. You'd need to modify the linker script to put a certain symbol at a fixed location, and with the current build system we can't do that.

[whaleygeek]

This work is already ongoing and progressing well in the upper editors https://github.com/mu-editor/mu/issues/185 and https://github.com/mu-editor/mu/pull/243

We need a way to trace a complete package (mu or PythonEditor) to the precise code that is inside it, in order to effectively support users. The biggest problem is being able to work out given what they have on their computer or what they used on the web, which 'known issues' are in a specific build, and which 'pending bug fixes' are not yet in it. We waste so much time working through issues that are 'already fixed' or 'in the pipeline to be fixed' and need a much more effective way to do this.

Can we re-think the link to the MicroPython.hex again and see if there is a way to do this? I think @ntoll had some ideas.

[dpgeorge]

See #420 for a way to have the version info available with direct traceability to source.

[jaustin]

@dpgeorge for ease of hex parsing and determinism of location, could we also please stick this information in CUSTOMER[16] (IE in the UICR)

The Makecode team are using that area for some structures around partial flashing. I think we should have the same location, and a new magic number to say "This is a MicroPython program" and then use the rest of the UICR as you like?

https://makecode.com/partial-flashing

[dpgeorge]

I think we should have the same location, and a new magic number to say "This is a MicroPython program" and then use the rest of the UICR as you like?

@jaustin OK we can do that. If I understand correctly, the UICR is a region of NVRAM that is separate to the flash (located at 0x10001000) and needs special treatment to place some data there. Probably we will need to modify the linker script to make a new UICR section so that C code can use __attribute__((section(".uicr"))) to place data in that section (and the generated hex file will have the data in it). Or is there a better way?