dir() calls properties when collecting their names

[caternuson]

Re this thread: https://forums.adafruit.com/viewtopic.php?f=58&t=175096

Some weirdness trying to use dir() with the CPX. Can recreate it:

Adafruit CircuitPython 6.1.0 on 2021-01-21; Adafruit CircuitPlayground Express with samd21g18
>>> from adafruit_circuitplayground import cp
>>> type(cp)
<class 'Express'>
>>> dir(cp)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "adafruit_circuitplayground/express.py", line 82, in _unsupported
NotImplementedError: This feature is not supported on Circuit Playground Express.
>>> 

It's not a simple issue with frozen modules, as trying with something else works fine:

Adafruit CircuitPython 6.1.0 on 2021-01-21; Adafruit CircuitPlayground Express with samd21g18
>>> import adafruit_lis3dh
>>> dir(adafruit_lis3dh)
['__class__', '__file__', '__name__', 'const', 'digitalio', 'math', 'namedtuple', 'struct', 'time', '__version__', '__repo__', 'LIS3DH_I2C', 'RANGE_8_G', 'RANGE_16_G', 'RANGE_4_G', 'RANGE_2_G', 'DATARATE_1344_HZ', 'DATARATE_400_HZ', 'DATARATE_200_HZ', 'DATARATE_100_HZ', 'DATARATE_50_HZ', 'DATARATE_25_HZ', 'DATARATE_10_HZ', 'DATARATE_1_HZ', 'DATARATE_POWERDOWN', 'DATARATE_LOWPOWER_1K6HZ', 'DATARATE_LOWPOWER_5KHZ', 'STANDARD_GRAVITY', 'AccelerationTuple', 'LIS3DH', 'LIS3DH_SPI']
>>> 

The imported cp otherwise works fine:

>>> cp.pixels.fill(0xADAF00)
>>> cp.play_tone(440, 1)
>>> 

got expected lit NeoPixels and a tone. Even tab completion is working.

[jfurcean]

I was able to reproduce this error. Commenting out the raise exception in _unsupported in adafruit_circuitplayground.express allowed for it to operate as expected.

    @property
    def _unsupported(self):
        """This feature is not supported on Circuit Playground Express."""
        raise NotImplementedError(
            "This feature is not supported on Circuit Playground Express."
        )

    @property
    def _unsupported(self):
        """This feature is not supported on Circuit Playground Express."""
        pass
        #raise NotImplementedError(
        #    "This feature is not supported on Circuit Playground Express."
        #)

[kattni]

Thank you for reproducing this. I assumed it was something connected to how we were handling unsupported features. I would like to figure out why it's happening, and sort out a way to still provide an error for the unsupported features.

[jfurcean]

I am not sure what causes it but you can fix this issue by adding a __dir__ function to this class and return the dir(super()). This is a little bit of a hack, but since this function doesn't have any other methods I think it should work.

def __dir__(self): 
        return dir(super())

[caternuson]

I'd be interested in knowing why dir() has this behavior also.

But, in terms of supporting this a different way, you could have stubbed versions of these functions added to circuit_playground_base that raise the exception. Boards that support the function would override and implement. Others would do nothing, in which case the base stub would get called and raise the exception.

[jfurcean]

I think it has something to do with it being a property, but I don't know why. I was able to reproduce this if I commented everything out related to the _unsupported property and created a new property that raised a different exception.

[caternuson]

Oh, good call. I've gotten so used to @property I don't even notice it much now. Hmmmm....why is it a property? For the items that get remapped:

    sound_level = _unsupported
    loud_sound = _unsupported
    play_mp3 = _unsupported

the last two are functions.

Simply commenting out the @property line gets around it also:

Adafruit CircuitPython 6.2.0-beta.1-194-gf6603aa56-dirty on 2021-02-10; Adafruit CircuitPlayground Express with samd21g18
>>> from adafruit_circuitplayground import cp
>>> dir(cp)
['__class__', '__dict__', '__init__', '__module__', '__qualname__', 'gamepad', 'temperature', 'acceleration', 'shake', 'tapped', '_int1', '_i2c', '_audio_out', 'sound_level', 'loud_sound', 'play_mp3', '_sample', 'stop_tone', '_speaker_enable', 'light', 'detect_taps', '_touch', 'touch_A1', 'touch_A2', 'touch_A3', 'touch_A4', 'touch_A5', 'touch_A6', 'touch_TX', 'adjust_touch_threshold', 'pixels', 'button_a', 'button_b', 'were_pressed', 'switch', 'red_led', '_sine_sample', '_generate_sample', 'play_tone', 'start_tone', 'play_file', '_a', '_b', '_switch', '_led', '_pixels', '_temp', '_light', '_touches', '_touch_threshold_adjustment', '_lis3dh', '_sine_wave', '_sine_wave_sample', '_detect_taps', 'touch_A7', '_unsupported']
>>> cp.sound_level
<bound_method>
>>> cp.sound_level()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "adafruit_circuitplayground/express.py", line 82, in _unsupported
NotImplementedError: This feature is not supported on Circuit Playground Express.
>>> 

[dhalbert]

It would appear that dir() is trying to call the unsupported properties. If you hit tab-complete, a property gets called. This may be a side-effect of that, but if so, bleh.

[jfurcean]

Interesting note in the python docs about dir().

Note: Because dir() is supplied primarily as a convenience for use at an interactive prompt, it tries to supply an interesting set of names more than it tries to supply a rigorously or consistently defined set of names, and its detailed behavior may change across releases. For example, metaclass attributes are not in the result list when the argument is a class.

Maybe __dir__ should be explicitly defined to avoid inconsistencies, especially for user libs?

[ajs256]

I also bumped into this in #3748. I agree that this is annoying and should be fixed.

[dhalbert]

The basic problem is that dir() is calling the properties:

test.py:

class Test:
    def func(self):
        pass

    @property
    def prt(self):
        print("prt was called")

    @property
    def prop(self):
        raise NotImplementedError("no prop")

Adafruit CircuitPython 6.2.0-beta.2-dirty on 2021-02-11; Adafruit PyPortal with samd51j20
>>> from test import Test
>>> t = Test()
>>> dir(t)
prt was called
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 11, in prop
NotImplementedError: no prop
>>> 

[caternuson]

Is this a duplicate of #3748 ?

[jfurcean]

It looks like a duplicate to me. Also, should this particular issue be raised in the library?

[dhalbert]

Yes, good catch, it's a dupe. I wouldn't say it's a library bug, because it's just the base bug being exercised. There may be other issues caused by calling dir(cp), such as instantiating all the TouchIn's, etc. I closed #3748 in favor of this more extensive discussion.

[dhalbert]

Same bug reported in MicroPython, not fixed there yet either: https://github.com/micropython/micropython/issues/4546.

[Neradoc]

Note that hasattr() and tab completion have the same effect (they use the same underlying method). Except tab catches the exception by passing true to mp_load_method_protected.

[dhalbert]

For interactive use, cp.<tab> is a workaround that can list the available functions and properties

[dhalbert]

Dupe of #2179