wpull doesn't like some changes in SQLAlchemy 1.4:
$ wpull https://example.org/
psutil: No module named 'psutil'. Resource monitoring will be unavailable.
.../lib/python3.6/site-packages/sqlalchemy/sql/coercions.py:308: SAWarning: implicitly coercing SELECT object to scalar subquery; please use the .scalar_subquery() method to produce a scalar subquery.
"implicitly coercing SELECT object to scalar subquery; "
INFO Fetching ‘https://example.org/’.
100.0% [=========================] 1.2 KiB 0:00:00 3.2 KiB/s
INFO Fetched ‘https://example.org/’: 200 OK. Length: 1256 [text/html; charset=UTF-8].
.../lib/python3.6/site-packages/wpull/protocol/http/client.py:185: UserWarning: HTTP session did not complete.
warnings.warn(_('HTTP session did not complete.'))
ERROR Fatal exception.
Traceback (most recent call last):
File ".../lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 1936, in _do_pre_synchronize_evaluate
eval_condition = evaluator_compiler.process(*crit)
File ".../lib/python3.6/site-packages/sqlalchemy/orm/evaluator.py", line 85, in process
return meth(clause)
File ".../lib/python3.6/site-packages/sqlalchemy/orm/evaluator.py", line 181, in visit_binary
map(self.process, [clause.left, clause.right])
File ".../lib/python3.6/site-packages/sqlalchemy/orm/evaluator.py", line 85, in process
return meth(clause)
File ".../lib/python3.6/site-packages/sqlalchemy/orm/evaluator.py", line 88, in visit_grouping
return self.process(clause.element)
File ".../lib/python3.6/site-packages/sqlalchemy/orm/evaluator.py", line 83, in process
"Cannot evaluate %s" % type(clause).__name__
sqlalchemy.orm.evaluator.UnevaluatableError: Cannot evaluate Select
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File ".../lib/python3.6/site-packages/wpull/application/app.py", line 157, in run
yield from pipeline.process()
File ".../lib/python3.6/site-packages/wpull/pipeline/pipeline.py", line 194, in process
yield from self._process_one_worker()
File ".../lib/python3.6/site-packages/wpull/pipeline/pipeline.py", line 215, in _process_one_worker
task.result()
File ".../lib/python3.6/site-packages/wpull/pipeline/pipeline.py", line 119, in process
item = yield from self.process_one(_worker_id=worker_id)
File ".../lib/python3.6/site-packages/wpull/pipeline/pipeline.py", line 103, in process_one
yield from task.process(item)
File ".../lib/python3.6/site-packages/wpull/application/tasks/download.py", line 492, in process
yield from session.app_session.factory['Processor'].process(session)
File ".../lib/python3.6/site-packages/wpull/processor/delegate.py", line 29, in process
return (yield from processor.process(item_session))
File ".../lib/python3.6/site-packages/wpull/processor/web.py", line 92, in process
return (yield from session.process())
File ".../lib/python3.6/site-packages/wpull/processor/web.py", line 186, in process
yield from self._process_loop()
File ".../lib/python3.6/site-packages/wpull/processor/web.py", line 245, in _process_loop
exit_early, wait_time = yield from self._fetch_one(cast(Request, self._item_session.request))
File ".../lib/python3.6/site-packages/wpull/processor/web.py", line 309, in _fetch_one
action = self._handle_response(request, response)
File ".../lib/python3.6/site-packages/wpull/processor/web.py", line 411, in _handle_response
self._item_session.update_record_value(status_code=response.status_code)
File ".../lib/python3.6/site-packages/wpull/pipeline/session.py", line 176, in update_record_value
self.app_session.factory['URLTable'].update_one(self.url_record.url, **kwargs)
File ".../lib/python3.6/site-packages/wpull/database/wrap.py", line 72, in update_one
return self.url_table.update_one(*args, **kwargs)
File ".../lib/python3.6/site-packages/wpull/database/sqltable.py", line 196, in update_one
session.execute(query)
File ".../lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1616, in execute
_parent_execute_state is not None,
File ".../lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 1821, in orm_pre_session_exec
update_options,
File ".../lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 1949, in _do_pre_synchronize_evaluate
from_=err,
File ".../lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 198, in raise_
raise exception
sqlalchemy.exc.InvalidRequestError: Could not evaluate current criteria in Python: "Cannot evaluate Select". Specify 'fetch' or False for the synchronize_session execution option.
CRITICAL Sorry, Wpull unexpectedly crashed.
CRITICAL Please report this problem to the authors at Wpull's issue tracker so it may be fixed. If you know how to program, maybe help us fix it? Thank you for helping us help you help us all.
INFO Exiting with status 1.
wpull doesn't like some changes in SQLAlchemy 1.4:
See also: https://travis-ci.org/github/ArchiveTeam/ArchiveBot/jobs/764039716
Until fixed properly (or until SQLAlchemy is disposed of responsibly):
pip install 'SQLAlchemy<1.4'