Run timer tasks using the latest version

[ryanvin]

I have many spiders in one project, when I modify the common functions, I need to update timer task one by one to use the latest version. It will be very helpful if timer task can use the latest version of spiders every time. 😉

[my8100]

The demand has been implemented in the latest commit, see https://github.com/my8100/scrapydweb/issues/4#issuecomment-475145676 You can get it via git:

git clone https://github.com/my8100/scrapydweb.git
cd scrapydweb
python setup.py install

or:

pip install -U git+https://github.com/my8100/scrapydweb.git

[ryanvin]

Many thanks!!!!!

[nidexiaogege]

[my8100]

@nidexiaogege Could you tell me how to reproduce the problem? Please post some logs of ScrapydWeb, as well as contents from http://127.0.0.1:5000/tasks/history/

[nidexiaogege]

[2019-05-24 16:12:26,609] WARNING in apscheduler: Shutting down the scheduler for timer tasks gracefully, wait until all currently executing tasks are finished [2019-05-24 16:24:48,814] WARNING in apscheduler: Shutting down the scheduler for timer tasks gracefully, wait until all currently executing tasks are finished [2019-05-24 16:26:40,096] WARNING in apscheduler: Shutting down the scheduler for timer tasks gracefully, wait until all currently executing tasks are finished [2019-05-24 16:33:57,455] WARNING in apscheduler: Shutting down the scheduler for timer tasks gracefully, wait until all currently executing tasks are finished [2019-05-24 16:48:03,528] WARNING in apscheduler: Shutting down the scheduler for timer tasks gracefully, wait until all currently executing tasks are finished [2019-05-24 17:23:09,850] WARNING in apscheduler: Shutting down the scheduler for timer tasks gracefully, wait until all currently executing tasks are finished [2019-05-24 18:14:20,056] WARNING in apscheduler: Shutting down the scheduler for timer tasks gracefully, wait until all currently executing tasks are finished

[nidexiaogege]

What should I do

[my8100]

@nidexiaogege

1. Run scrapydweb with the argument --verbose
2. Try to reproduce the problem
3. Post related logs and the full traceback info in the fail page

[nidexiaogege]

[nidexiaogege]

Which argument is the problem

[my8100]

Run python -c "import apscheduler; print(apscheduler.__version__)", and post the version number. BTW, did you modify any source code of scrapydweb? How did you install scrapydweb, pip or git?

[nidexiaogege]

pip install

[my8100]

What is the version number of apscheduler? How to trigger it, by adding or editing a task?

[nidexiaogege]

the version number of apscheduler is 3.3.1

[my8100]

Run pip install -U APScheduler==3.5.3

[nidexiaogege]

The problem was solved，thanks!!!