ValueError: invalid file open mode 'U' on Python 3.11

mock commented 1 year ago

Description

I'm logging this here for reference per the advice given by those in the IRC support channel.

After upgrading my OS, my default Python is now 3.11.1. When I tried to start up my bots, I received multiple errors for the my custom modules. The error manifested as:

[2023-01-18 14:24:24,238] sopel.bot            ERROR    - Error loading weather: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'

The full error file is included below for a complete trace.

It was suggested for the time being to pull a copy of the master branch and then make any adjustments to the code to avoid the 'U' mode. (The line causing the problems was located here: https://github.com/sopel-irc/sopel/blob/v7.1.9/sopel/plugins/handlers.py#L435.)

Reproduction steps

Upgrade/Install Fedora 37
pip install --upgrade sopel
sopel start -c bot.cfg -d

Expected behavior

Loading of my custom modules as with Python 3.10.x. Custom modules then will listen for triggers.

Relevant logs

[2023-01-18 09:16:02,039] sopel.irc            ERROR    - Fatal error in core, please review exceptions log.
[2023-01-18 09:16:01,492] sopel.irc.backends   ERROR    - Server timeout detected after 550.133971s; closing.
[2023-01-18 09:16:23,122] sopel.bot            ERROR    - Cannot register URL callback: Bugzilla URL callback requires "bugzilla.domains" to be configured; check your config file.
[2023-01-18 14:24:22,565] sopel.bot            ERROR    - Cannot register URL callback: Bugzilla URL callback requires "bugzilla.domains" to be configured; check your config file.
[2023-01-18 14:24:22,717] sopel.bot            ERROR    - Error in help setup: global flags not at the start of the expression at position 14
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 337, in setup_plugins
    plugin.register(self)
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 330, in register
    bot.add_plugin(self, *relevant_parts)
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 479, in add_plugin
    self.register_callables(callables)
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 620, in register_callables
    rule = plugin_rules.Rule.from_callable(settings, callbl)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/rules.py", line 815, in from_callable
    regexes = cls.regex_from_callable(settings, handler)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/rules.py", line 791, in regex_from_callable
    return tuple(_clean_rules(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/rules.py", line 87, in _clean_rules
    yield _compile_pattern(pattern, nick, aliases)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/rules.py", line 106, in _compile_pattern
    return re.compile(pattern, flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/__init__.py", line 227, in compile
    return _compile(pattern, flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/__init__.py", line 294, in _compile
    p = _compiler.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/_compiler.py", line 743, in compile
    p = _parser.parse(p, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/_parser.py", line 980, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/_parser.py", line 455, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/_parser.py", line 841, in _parse
    raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 14
[2023-01-18 14:24:24,221] sopel.bot            ERROR    - Error loading buzz_words: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,224] sopel.bot            ERROR    - Error loading camel: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,227] sopel.bot            ERROR    - Error loading hello_world: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,230] sopel.bot            ERROR    - Error loading test: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,233] sopel.bot            ERROR    - Error loading rick: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,235] sopel.bot            ERROR    - Error loading wthr: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,238] sopel.bot            ERROR    - Error loading weather: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'

Notes

No response

Sopel version

7.1.9

Installation method

pip install

Python version

3.11.1

Operating system

Fedora 37

IRCd

No response

Relevant plugins

all my custom plugins

SnoopJ commented 1 year ago

File mode 'U' was removed in 3.11, the code responsible for this error was dropped in 596adc4 but the change isn't yet in a release. Users who run into the problem should install Sopel from master for the time being, or maybe run their instance with 3.10 or lower until the next release.

Thanks for the report!

dgw commented 1 year ago

I've created and assigned a new milestone, with an arbitrarily assigned "pay attention to me!" date of 1 May. If we don't get 8.0 finalized by then, we can cut a hotfix release for the stable branch. (If the Travis CI release pipeline no longer works, I'll build and upload it myself!) Meanwhile, I will pin the issue.

Note: Hotfix urgency to be adjusted as needed if too many users start running into this, i.e. if Python 3.11 environments become too common in the wild to ignore.

mock commented 1 year ago

Just confirming that installing the master branch resolved the above issue without any other tweaking.

alethiophile commented 1 year ago

Just here to note that I ran into this error today on installing the latest release of sopel, which apparently is still not compatible with Python 3.11 despite this error being known for five months.

Just cut a release already, even if this is the only thing in it.

dgw commented 1 year ago

We can't run the maintenance branch's CI on GitHub Actions, and the old Travis-CI test + release pipeline is also broken due to changes from them (even though we registered as an OSS project 🙄)

Basically, creating a maintenance release that is adequately tested just for this bug (which has a workaround—installing from the master branch) would take too much of our limited developer time away from finishing Sopel 8.0, which is on track to be done next month. Seriously, we're very close! Prioritization is just very important for us, as a small project with few active contributors.

dgw commented 11 months ago

I'm writing the Sopel 8 upgrade guide, and @SnoopJ has put in a bunch of work on the changelog for 8.0.0. It's safe to say that this won't be fixed in a patch release for 7.1.x, so I'm re-tagging this for 8.0.0 and closing it (because it is, in fact, fixed).

sopel-irc / sopel