cefn commented 6 years ago

Wanting to create the board module with proper conventions matching the existing declarations. Prior examples can be inspected from within the board's REPL (here a Feather M0 express)...

>>> import board
>>> dir(board)
['A0', 'A1', 'A2', 'A3', 'A4', 'A5', 'SCK', 'MOSI', 'MISO', 'D0', 'RX', 'D1', 'TX', 'SDA', 'SCL', 'D5', 'D6', 'D9', 'D10', 'D11', 'D12', 'D13', 'NEOPIXEL']

but are implemented in C via includes conditionally on the board in use, such as this from circuitpython/ports/atmel-samd/boards/feature_m0_express/pins.c ...

#include "samd21_pins.h"

STATIC const mp_rom_map_elem_t board_global_dict_table[] = {
    { MP_ROM_QSTR(MP_QSTR_A0), MP_ROM_PTR(&pin_PA02) },
    { MP_ROM_QSTR(MP_QSTR_A1), MP_ROM_PTR(&pin_PB08) },
    { MP_ROM_QSTR(MP_QSTR_A2), MP_ROM_PTR(&pin_PB09) },
    { MP_ROM_QSTR(MP_QSTR_A3), MP_ROM_PTR(&pin_PA04) },
    { MP_ROM_QSTR(MP_QSTR_A4), MP_ROM_PTR(&pin_PA05) },
    { MP_ROM_QSTR(MP_QSTR_A5), MP_ROM_PTR(&pin_PB02) },
    { MP_ROM_QSTR(MP_QSTR_SCK), MP_ROM_PTR(&pin_PB11) },
    { MP_ROM_QSTR(MP_QSTR_MOSI), MP_ROM_PTR(&pin_PB10) },
    { MP_ROM_QSTR(MP_QSTR_MISO), MP_ROM_PTR(&pin_PA12) },
    { MP_ROM_QSTR(MP_QSTR_D0), MP_ROM_PTR(&pin_PA11) },
    { MP_ROM_QSTR(MP_QSTR_RX), MP_ROM_PTR(&pin_PA11) },
    { MP_ROM_QSTR(MP_QSTR_D1), MP_ROM_PTR(&pin_PA10) },
    { MP_ROM_QSTR(MP_QSTR_TX), MP_ROM_PTR(&pin_PA10) },
    { MP_ROM_QSTR(MP_QSTR_SDA), MP_ROM_PTR(&pin_PA22) },
    { MP_ROM_QSTR(MP_QSTR_SCL), MP_ROM_PTR(&pin_PA23) },
    { MP_ROM_QSTR(MP_QSTR_D5), MP_ROM_PTR(&pin_PA15) },
    { MP_ROM_QSTR(MP_QSTR_D6), MP_ROM_PTR(&pin_PA20) },
    { MP_ROM_QSTR(MP_QSTR_D9), MP_ROM_PTR(&pin_PA07) },
    { MP_ROM_QSTR(MP_QSTR_D10), MP_ROM_PTR(&pin_PA18) },
    { MP_ROM_QSTR(MP_QSTR_D11), MP_ROM_PTR(&pin_PA16) },
    { MP_ROM_QSTR(MP_QSTR_D12), MP_ROM_PTR(&pin_PA19) },
    { MP_ROM_QSTR(MP_QSTR_D13), MP_ROM_PTR(&pin_PA17) },
    { MP_ROM_QSTR(MP_QSTR_NEOPIXEL), MP_ROM_PTR(&pin_PA06) },
};
MP_DEFINE_CONST_DICT(board_module_globals, board_global_dict_table);

So there is a question how a cross-platform library should expose canonical pin names - should there be a different whole library per board which includes a different board/init.py or rather there should be a board/esp8266.py and a decision is made by the importer what board they are running on, or potentially conditionally included within init.py based on device introspection.

cefn commented 6 years ago

For now, will use device introspection and conditional includes.

ladyada commented 6 years ago

hi! ok i think to clarify - board definitely does not have to be cross-platform, in the sense that, you'll always need to tweak your board.pin names when running a python script on a new board. that said, the init.py should use try/except and uos.uname() or whatnot to figure out what its running on, and then populate board correctly. so, despite the pins being named differently, the import in the 'main script' is always the same i think thats the Q? :)

ladyada commented 6 years ago

ok actually spent time testing on a pyboard! using this test code as reference, its very close just needs a little rearranging: https://github.com/cefn/Adafruit_Micropython_Blinka/blob/master/python/testing/board/__init__.py

you can keep the agnostic helper for now - since it makes testing simpler while you bring things up. the actual test code will look more like:

import board
if agnostic.board == "pyboard":
   led_pin = board.LED_BLUE
elif agnostic.board == "feather_huzzah":
   led_pin = board.GPIO0
elif agnostic.board == "feather_m0_express":
   led_pin = board.D13
else:
   raise RuntimeError("Unknown board!")

(i removed the led_hardwired = True, led_inverted = False for legibility, you can re-add!) that is to say, within the import, it should figure out if its a pyboard or ESP8266.

whew - does that make sense?

cefn commented 6 years ago

Thanks for diving into testing already! Great.

I think we're on the same page, but want to make clear the stuff under testing.X is going to be 'thrown away' when the compatibility layer is distributed. The switches there are only helping to populate flags and values which the testing suite will draw upon. Emphasising the contrast between the testing module tree and the compatibility module tree...

Testing module tree

In respect of the code fragment picked out from testing.board makes sense to add an exception if the board cannot be identified at all. However also sense to deliberately allow test cases to throw errors for definitions which are not there. Thus if you actually DO have a led_pin for your configuration, but haven't got around to defining a switch_pin yet, the board config import goes fine and the led_pin test flies through, but any time you attempt to run the switch_pin test, it dies. This constrains the error, somewhat in an EAFP style, matching with progressively-increasing coverage driven by tests.

Compatibility module tree

Definitions of the board and microcontroller modules which are distributed with the compatibility layer also need to fail hard if they can't introspect the platform just as you say.

The source files for those are outside of the testing tree at https://github.com/cefn/Adafruit_Micropython_Blinka/blob/master/python/board/__init__.py and https://github.com/cefn/Adafruit_Micropython_Blinka/blob/master/python/microcontroller/__init__.py (this is probably why in the long run the testing and implementation mirror hierarchies shouldn't be sitting in the same place - very confusing).

Currently the compatibility board and microcontroller init.py just conditionally import from other microcontroller- and board- specific files. The basic idea is that a single unzipped version of the compatibility layer will work on all platforms (e.g. including all the definitions you don't need) but if you needed to save space or wanted to specialise, you could just drop e.g. all the esp8266 files you weren't using and it would make no difference. Still stricter, by overwriting board/init.py with board/esp8266.py for an ESP8266-target-specific bundle, you eliminate the conditional logic and the recursive import too. (Shockingly I managed to actually exceed the available stack with recursive imports on ESP8266 when the impact of lazy-loading modules got particularly bad - a workaround was to pre-load using import microcontroller.esp8266 and not just wait for a later access to trigger import of everything).

Sounds overall like the approach holds together and initial testing did what it was meant to.

The file agnostic.py is intended to have three roles

centralising introspection logic, (which would otherwise have to be replicated throughout)
providing a single import strategy for symbols which are different on different platforms (sometimes Micropython and CircuitPython and even CPython if you're hoping to benefit from e.g. using a PyCharm debugger against Mocks for testing.
platform-specific configuration steps (e.g. optimisation flags which differ on different platforms)

I have ended up needing this approach on another which needed to run on Micropython and CPython, and this example file illustrates the role it might have in the end, though the MCP one is fairly minimal now.

Perhaps we could shift all this into mcp.py meaning there's only one crufty extra namespace associated with the compatibility layer.

ladyada commented 6 years ago

hmm, i think im following you but i also just trust you're doing it in a good way :dagger: how about you keep going, we can always squash namespaces once the layer is working.

cefn commented 6 years ago

Fail-fast platform introspection in https://github.com/cefn/Adafruit_Micropython_Blinka/commit/c24f5e6fa8655fd86333243f5738ca4c0bf97fe7

cefn / Adafruit_Micropython_Blinka

Reference naming conventions for board #1

Testing module tree

Compatibility module tree