ben-nathanson
commented
3 years ago When I run flake8 locally I get the following message:
(city-scrapers-pitt) $flake8
Traceback (most recent call last):
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/bin/flake8", line 5, in <module>
from flake8.main.cli import main
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/flake8/main/cli.py", line 5, in <module>
from flake8.main import application
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/flake8/main/application.py", line 11, in <module>
from flake8 import checker
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/flake8/checker.py", line 18, in <module>
from flake8 import processor
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/flake8/processor.py", line 13, in <module>
from flake8 import utils
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/flake8/utils.py", line 16, in <module>
from flake8._compat import lru_cache
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/flake8/_compat.py", line 12, in <module>
import importlib_metadata
ModuleNotFoundError: No module named 'importlib_metadata'When I run scrapy list I get a similar message:
(city-scrapers-pitt) $scrapy list
Traceback (most recent call last):
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/jsonschema/__init__.py", line 31, in <module>
from importlib import metadata
ImportError: cannot import name 'metadata' from 'importlib' (/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/importlib/__init__.py)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/bin/scrapy", line 8, in <module>
sys.exit(execute())
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/scrapy/cmdline.py", line 124, in execute
cmds = _get_commands_dict(settings, inproject)
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/scrapy/cmdline.py", line 56, in _get_commands_dict
cmds.update(_get_commands_from_module(cmds_module, inproject))
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/scrapy/cmdline.py", line 33, in _get_commands_from_module
for cmd in _iter_command_classes(module):
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/scrapy/cmdline.py", line 20, in _iter_command_classes
for module in walk_modules(module_name):
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/scrapy/utils/misc.py", line 78, in walk_modules
submod = import_module(fullpath)
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/city_scrapers_core/commands/validate.py", line 7, in <module>
from ..pipelines import ValidationPipeline
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/city_scrapers_core/pipelines/__init__.py", line 5, in <module>
from .validation import ValidationPipeline # noqa
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/city_scrapers_core/pipelines/validation.py", line 5, in <module>
from jsonschema.validators import Draft7Validator
File "/Users/ben/.local/share/virtualenvs/city-scrapers-pitt-qxX_3ASu/lib/python3.7/site-packages/jsonschema/__init__.py", line 33, in <module>
import importlib_metadata as metadata
ModuleNotFoundError: No module named 'importlib_metadata'
will-snavely
A spider for Bethel Park public meetings.
This spider retrieves meetings from the Bethel Park website at: http://bethelpark.net
The site allows one to download an event calendar in various formats. We choose to download the iCalendar (https://en.wikipedia.org/wiki/ICalendar) version of the calendar. To do this manually, you can navigate to: https://bethelpark.net/events/
Then, click on the "Subscribe to filtered calendar", and choose the "Add to other calendar" option. This will download a ".ics" file in the iCalendar format.
To parse this file, I would like to add a dependency on the Python package called "ics" (https://icspy.readthedocs.io/en/stable/). I think this is justified, because it's likely we will run into this format again in the future, and it will be useful to have a parser around for such cases. Am willing to hear other opinions, though.