Error loading plugins in errbot version 6.1.5

alpinweis commented 4 years ago

In order to let us help you better, please fill out the following fields as best you can:

I am...

[x] Reporting a bug
[ ] Suggesting a new feature
[ ] Requesting help with running my bot
[ ] Requesting help writing plugins
[ ] Here about something else

I am running...

Errbot version: 6.1.5
OS version: ubuntu:18.04 in a docker image
Python version: 3.6.9
Using a virtual environment: no

Issue description

I pulled the latest version 6.1.5 and when starting the bot I get the following error for all plugins:

ERROR    errbot.plugin_manager     Something is wrong with the configuration of the plugin VersionChecker
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/errbot/plugin_manager.py", line 395, in _activate_plugin
    config = self.get_plugin_configuration(name)
  File "/usr/local/lib/python3.6/dist-packages/errbot/plugin_manager.py", line 344, in get_plugin_configuration
    if name not in configs:
TypeError: argument of type 'NoneType' is not iterable
ERROR    errbot.plugin_manager     Error loading VersionChecker.

Everything works fine with the previous version 6.1.4.

nzlosh commented 4 years ago

I just tested a fresh installation but wasn't able to replicate your issue.

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic
root@1d895822f349:/opt/errbot# python --version
Python 3.6.9
root@1d895822f349:/opt/errbot# errbot --version
Errbot version 6.1.5

When you say you get the error for all plugins, did you try removing the version checking plugin to see if errbot could start?

alpinweis commented 4 years ago

I mean all plugins report that error. When errbot starts it spits out something like this in the errbot slack DM:

Error: VersionChecker failed to activate: argument of type 'NoneType' is not iterable.
Error: Backup failed to activate: argument of type 'NoneType' is not iterable.
Error: Utils failed to activate: argument of type 'NoneType' is not iterable.
Error: TextCmds failed to activate: argument of type 'NoneType' is not iterable.
Error: Plugins failed to activate: argument of type 'NoneType' is not iterable.
Error: Help failed to activate: argument of type 'NoneType' is not iterable.
Error: ChatRoom failed to activate: argument of type 'NoneType' is not iterable.
Error: CommandNotFoundFilter failed to activate: argument of type 'NoneType' is not iterable.
Error: Health failed to activate: argument of type 'NoneType' is not iterable.
Error: Webserver failed to activate: argument of type 'NoneType' is not iterable.
Error: Flows failed to activate: argument of type 'NoneType' is not iterable.
Error: ACLs failed to activate: argument of type 'NoneType' is not iterable.
....my custom plugins go here...

sijis commented 4 years ago

@alpinweis would you be able to try with a minimal config.py? The simplest way to generate one is errbot --init.

alpinweis commented 4 years ago

in case it matters, here is my config.py

import logging
import os
BACKEND = "Slack"

ERRBOT_SLACK_TOKEN = os.getenv("ERRBOT_SLACK_TOKEN")

ERRBOT_DIR = os.getenv("ERRBOT_DIR", os.path.abspath(os.path.dirname(__file__)))

BOT_DATA_DIR = os.path.join(ERRBOT_DIR, "data")
BOT_EXTRA_PLUGIN_DIR = os.path.join(ERRBOT_DIR, "plugins")
BOT_LOG_FILE = os.path.join(ERRBOT_DIR, "errbot.log")

BOT_LOG_LEVEL = logging.DEBUG

BOT_ALT_PREFIXES = ("err", "sss")
BOT_ALT_PREFIX_SEPARATORS = (":", ",", ";")
BOT_ALT_PREFIX_CASEINSENSITIVE = True

BOT_IDENTITY = {"token": ERRBOT_SLACK_TOKEN}

BOT_ADMINS = ("@firstlast1", "@firstlast2", "@firstlast3")

ACCESS_CONTROLS = {
    "Backup:*": {"allowusers": BOT_ADMINS},
    "ChatRoom:*": {"allowusers": BOT_ADMINS},
    "Flows:*": {"allowusers": BOT_ADMINS},
    "Health:*": {"allowusers": BOT_ADMINS},
    "Plugins:*": {"allowusers": BOT_ADMINS},
    "Webserver:*": {"allowusers": BOT_ADMINS},
    "log_tail": {"allowusers": BOT_ADMINS},
}

HIDE_RESTRICTED_COMMANDS = True
HIDE_RESTRICTED_ACCESS = True

DIVERT_TO_PRIVATE = (
    "about",
    "apropos",
    "help",
    "history",
    "render_test",
    "status",
    "whoami",
)

nzlosh commented 4 years ago

I still can't replicate your errors even with the configuration you provided. Are you using the ubuntu docker image directly or building your own image?

alpinweis commented 4 years ago

here is my dockerfile for the errbot image

FROM ubuntu:18.04

ENV ERR_USER errbot
ENV ERRBOT_DIR /errbot

ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8

RUN useradd --no-create-home --no-user-group -g nogroup -s /bin/false $ERR_USER

RUN apt-get update && apt-get install -y --no-install-recommends \
    git \
    curl \
    sudo \
    build-essential \
    openssh-client \
    python3 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

WORKDIR $ERRBOT_DIR

COPY plugins plugins
COPY requirements.txt /tmp/requirements.txt
COPY config.py config.py

RUN mkdir $ERRBOT_DIR/data && chown -R $ERR_USER $ERRBOT_DIR

RUN python3 --version && pip3 --version \
    && pip3 install --no-cache-dir --upgrade setuptools wheel \
    && pip3 install --no-cache-dir -r /tmp/requirements.txt

RUN echo "{'configs': {'Webserver': {'HOST': '0.0.0.0', 'PORT': 3141, 'SSL': {'certificate': '', 'enabled': False, 'host': '0.0.0.0', 'key': '', 'port': 3142}}}}" | errbot --storage-merge core

EXPOSE 3141 3142

ENTRYPOINT ["errbot", "-c", "/errbot/config.py"]

The only thing configs-related is the RUN echo line to enable the web server. Could this be the problem? The original error message I get is related to configs

    if name not in configs:
TypeError: argument of type 'NoneType' is not iterable

^ I suspect this means configs is None ?

alpinweis commented 4 years ago

If I comment out this line in the dockerfile it fixes the issue and all plugins activate just fine

RUN echo "{'configs': {'Webserver': {'HOST': '0.0.0.0', 'PORT': 3141, 'SSL': {'certificate': '', 'enabled': False, 'host': '0.0.0.0', 'key': '', 'port': 3142}}}}" | errbot --storage-merge core

That said, what's the right way to enable webserver at image build time in v6.1.5?

sijis commented 4 years ago

It's the same process as in previous versions, that has not changed.

I'd have to double check but I believe the key is config not configs.

The quick way to validate is

errbot --init
errbot
!plugin config webseever

sijis commented 4 years ago

I'd have to double check but I believe the key is config not configs.

@alpinweis ok, so i was wrong. After a little Dockerfile trial and error, the thing that seemed to work was this

RUN echo "{'configs': {'Webserver': {'HOST': '0.0.0.0', 'PORT': 3141, 'SSL': {'certificate': '', 'enabled': False, 'host': '0.0.0.0', 'key': '', 'port': 3142}}}}" | errbot --storage-set core

Its basically changing --storage-merge with --storage-set. It makes sense in this case since you are setting the values of that plugin and its a new setup.

You can also add this line after the --storage-set to validate that there is data.

RUN errbot --storage-get core

Good output

STEP 12: RUN errbot --storage-get core
{'configs': {'Webserver': {'HOST': '0.0.0.0', 'PORT': 3141, 'SSL': {'certificate': '', 'enabled': False, 'host': '0.0.0.0', 'key': '', 'port': 3142}}}}

Bad output

STEP 12: RUN errbot --storage-get core
{'configs': None}

alpinweis commented 4 years ago

@sijis Thanks for figuring it out. The weird thing is that this exact dockerfile with --storage-merge core works fine with errbot v6.1.4, and the update around the configs merge in v6.1.5 looks like a breaking change

sijis commented 4 years ago

@alpinweis Yeah, definitely a regression issue. It should be addresed in https://github.com/errbotio/errbot/pull/1470. Would you be able to validate that it resolves the problem?

alpinweis commented 4 years ago

@sijis, I was able to test it with my existing dockefile by pulling errbot from your branch (in my requirements.txt): git+https://github.com/sijis/errbot@fix/cli_merge_unknown#egg=errbot

I can confirm the fix resolves the issue.

errbotio / errbot