from scrapy import signals, log as scrapy_log builtins.ImportError: cannot import name 'log'

[Hiroki111]

I installed scrapyrt today, but calling a GET request results in an error, which was reported at https://github.com/scrapy/scrapyd/issues/311

I followed the issue above, and downgraded Twisted with pip3 install Twisted==18.9.0.

After that, however, scrapyrt is producing a different error, saying...

2019-07-28 16:53:22+0200 [scrapyrt] Unhandled Error
    Traceback (most recent call last):

       ...

    File "/home/user/app/scrapy-test/my_venv/lib64/python3.6/site-packages/scrapyrt/core.py", line 9, in <module>
        from scrapy import signals, log as scrapy_log
    builtins.ImportError: cannot import name 'log'

Could I ask for help to fix it?

It's running on Python 3.6.3 Scrapy is 1.7.1

[Hiroki111]

According to the doc of scrapy, there is a module called logging from scrapy.

So, I assume that the line 9 in python3.6/site-packages/scrapyrt/core.py, which is currently

from scrapy import signals, log as scrapy_log

... was actually supposed to be

from scrapy import signals, logging as scrapy_log

Any ideas?

[korneel]

A workaround is to downgrade Scrapy to v1.6.0.

[myfoxsay]

scrapy.log module has been removed in 1.7 version, after a long deprecation period.

[llermaly]

As @korneel says downgrading scrapy fixes the issue

pip3 uninstall scrapy pip3 install scrapy==1.6

[pawelmhm]

thanks for pointing this out @Hiroki111 . I'm going to fix it. To fix it we need to refactor logging. At the moment we rely on scrapy.log when creating logging observer here https://github.com/scrapinghub/scrapyrt/blob/master/scrapyrt/core.py#L89 Need to refactor logging observer to use default Python logging system instead of Scrapy custom logging functions

[pawelmhm]

fixed in #90