search

nextcloud / news-updater

:newspaper: Fast, parallel feed updater for the News app; written in Python
GNU General Public License v3.0
107 stars 23 forks source link

Subprocess.CalledProcessError returned non-zero exit status 1 #9

Closed LexiconCode closed 7 years ago

LexiconCode commented 7 years ago

After installing via sudo pip3 install nextcloud_news_updater --install-option="--install-scripts=/usr/bin"

Verifying nextcloud news can update via Cron.

After ensuring that "useCronUpdates = true" in nextcloud/cron.php

When executing 'python3 -m nextcloud_news_updater /path/to/nextcloud' nextcloud-news-updater /var/www/nextcloud

Nextcloud News Updater - ERROR - Command '['php', '-f', '/var/www/nextcloud/occ', 'news:updater:before-update']' returned non-zero exit status 1: Trying again in 30 seconds
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/updater.py", line 69, in run
    self.before_update()
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 73, in before_update
    self.cli.run(self.api.before_cleanup_command)
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 12, in run
    return check_output(commands)
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['php', '-f', '/var/www/nextcloud/occ', 'news:updater:before-update']' returned non-zero exit status 1

I would like to follow the instructions below to further debug


    Check if the config.ini in nextcloud/data/news/config/config.ini contains useCronUpdates = false

    Start the updater in loglevel info mode and check if the feed update urls are polled, e.g.:

    nextcloud_news_updater --loglevel info -c /path/to/config.ini

    Check your data/nextcloud.log for errors

What does it mean by '/path/to/config.ini'? What's a default path generated during the nextcloud-news-updater install?

BernhardPosselt commented 7 years ago

Check your logs

LexiconCode commented 7 years ago

if you mean nextcloud.log, php5-fpm.log they do not show any related errors Edit I may have misunderstood the install instructions as I might have to create a config.ini

LexiconCode commented 7 years ago

@BernhardPosselt I have created news-updater.ini

[updater]
threads = 10
interval = 900
loglevel = info
# or https://domain.com/nextcloud when using the REST API
url = /var/www/nextcloud/
phpini = /etc/php5/cli/php.ini
# or v2 which is currently a draft
apilevel = v2
mode = singlerun

Executing 'nextcloud-news-updater -c /mnt/usb_1/nextcloud_data/news-updater.ini'

2017-02-27 18:57:10,490 - Nextcloud News Updater - ERROR - Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1: Trying again in 30 seconds
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/updater.py", line 69, in run
    self.before_update()
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 73, in before_update
    self.cli.run(self.api.before_cleanup_command)
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 12, in run
    return check_output(commands)
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1
^CTraceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/updater.py", line 69, in run
    self.before_update()
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 73, in before_update
    self.cli.run(self.api.before_cleanup_command)
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 12, in run
    return check_output(commands)
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1

Can you specify which and where the logs I should be checking for errors? As I've checked the ones mentioned in the previous post.

BernhardPosselt commented 7 years ago

Can you run it in loglevel info mode?

BernhardPosselt commented 7 years ago

and get the console output

BernhardPosselt commented 7 years ago

Also check out https://github.com/nextcloud/news-updater#using-the-cli-based-updater-fails

LexiconCode commented 7 years ago

@BernhardPosselt

Also check out https://github.com/nextcloud/news-updater#using-the-cli-based-updater-fails

sudo -u www-data php -f /var/www/nextcloud/occ news:updater:all-feeds {"feeds":[{"id":2,"userId":"User1"},{"id":4,"userId":"User1"},{"id":5,"userId":"User1"},{"id":6,"userId":"User1"},{"id":7,"userId":"User1"},{"id":8,"userId":"User2"},{"id":12,"userId":"User2"},{"id":13,"userId":"User3"},{"id":14,"userId":"User4"},{"id":15,"userId":"User4"},{"id":16,"userId":"User3"},{"id":17,"userId":"User3"},{"id":18,"userId":"User3"}]

sudo -u www-data php -f /var/www/nextcloud/occ news:updater:before-update Produces no output

sudo -u www-data php -f /var/www/nextcloud/occ news:updater:update-feed FEED_ID USER_ID Produces no output - I only tested with one set of feed ID and user ID

sudo -u www-data php -f /var/www/nextcloud/occ news:updater:after-update Produces no output

Can you run it in loglevel info mode? and get the console output

nextcloud-news-updater --loglevel info -c /mnt/usb_1/nextcloud_data/news-updater.ini

2017-02-28 16:37:26,967 - Nextcloud News Updater - INFO - Running update once with 10 threads
2017-02-28 16:37:26,968 - Nextcloud News Updater - INFO - Running before update command: php -f /var/www/nextcloud/occ -c /etc/php5/cli/php.ini news:updater:before-update
2017-02-28 16:37:27,415 - Nextcloud News Updater - ERROR - Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1: Trying again in 30 seconds
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/updater.py", line 69, in run
    self.before_update()
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 73, in before_update
    self.cli.run(self.api.before_cleanup_command)
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 12, in run
    return check_output(commands)
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1

During this experience with the commands above this made me think to add sudo -u www-data nextcloud-news-updater ... instead of nextcloud-news-updater...

sudo -u www-data nextcloud-news-updater --loglevel info -c /mnt/usb_1/nextcloud_data/news-updater.ini

2017-02-28 16:41:37,427 - Nextcloud News Updater - INFO - Running update once with 10 threads
2017-02-28 16:41:37,427 - Nextcloud News Updater - INFO - Running before update command: php -f /var/www/nextcloud/occ -c /etc/php5/cli/php.ini news:updater:before-update
2017-02-28 16:41:39,916 - Nextcloud News Updater - INFO - Running get all feeds command: php -f /var/www/nextcloud/occ -c /etc/php5/cli/php.ini news:updater:all-feeds
2017-02-28 16:41:39,917 - Nextcloud News Updater - INFO - Received these feeds to update: {"feeds":[{"id":2,"userId":"User1"},{"id":4,"userId":"User1"},{"id":5,"userId":"User1"},{"id":6,"userId":"User1"},{"id":7,"userId":"User1"},{"id":8,"userId":"User2"},{"id":12,"userId":"User2"},{"id":13,"userId":"User3"},{"id":14,"userId":"User4"},{"id":15,"userId":"User4"},{"id":16,"userId":"User3"},{"id":17,"userId":"User3"},{"id":18,"userId":"User3"}]
2017-02-28 16:41:39,918 - Nextcloud News Updater - ERROR - 'updater': Trying again in 30 seconds
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/updater.py", line 70, in run
    feeds = self.all_feeds()
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 84, in all_feeds
    return self.api.parse_feed(feeds_json)
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/api.py", line 22, in parse_feed
    return self._parse_json(feed_json)
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 38, in _parse_json
    feed_json = feed_json['updater']
KeyError: 'updater'

Thanks for your help and patience. I really appreciate news updater as I heavily rely on newsfeeds from nextcloud. Hence the issue of using Cron

BernhardPosselt commented 7 years ago

apilevel = v2

That's your issue ;D

v2 is not supported yet ;D

LexiconCode commented 7 years ago

@BernhardPosselt Might want to clarify that a little bit here. Although it now that I look at it it's in the documentation and the ini. :). I am blind as a bat!

nextcloud-news-updater --help
 --apilevel {v1-2,v2}, -a {v1-2,v2}
                        API level. Use v2 for News 9.0.0 or greater, v1-2 for
                        lower versions

My news-updater.ini is now configured as...

[updater]
threads = 10
interval = 900
loglevel = info
# or https://domain.com/nextcloud when using the REST API
url = /var/www/nextcloud/
phpini = /etc/php5/cli/php.ini
# or v2 which is currently a draft
apilevel = v1-2
mode = singlerun

Unfortunately sudo nextcloud-news-updater --loglevel info -c /mnt/usb_1/nextcloud_data/news-updater.ini

2017-02-28 16:58:26,481 - Nextcloud News Updater - INFO - Running update once with 10 threads
2017-02-28 16:58:26,482 - Nextcloud News Updater - INFO - Running before update command: php -f /var/www/nextcloud/occ -c /etc/php5/cli/php.ini news:updater:before-update
2017-02-28 16:58:26,932 - Nextcloud News Updater - ERROR - Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1: Trying again in 30 seconds
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/updater.py", line 69, in run
    self.before_update()
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 73, in before_update
    self.cli.run(self.api.before_cleanup_command)
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 12, in run
    return check_output(commands)
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1

Double check I'm using the right PHP ini php -i | grep php.ini

Configuration File (php.ini) Path => /etc/php5/cli
Loaded Configuration File => /etc/php5/cli/php.ini
LexiconCode commented 7 years ago

Nextcloud 11.02 Nextcloud News App: 10.1.0

If you would like to replicate my set up exactly it would be very easy to do so. The only differences I imported news nextcloud database from my previous install.

  1. Go to Dietpi.com
  2. Download the virtual box file and run it on virtual box
  3. It'll walk you through updating dietpi
  4. It'll reboot then execute dietpi-software
  5. Make sure Webserver Preference : Nginx and SSH Server : OpenSSH are selected
  6. Go to Software Optimized menu and select 114 NextCloud: your very own cloud (eg: dropbox)
  7. Navigate to and select Install Go >> Start installation for selected software
  8. It will install, reboot, and then login based on the local IP with user admin password dietpi
  9. Install the news app and make some subscriptions
  10. Test original Nextcloud News app update mechanism

Execute crontab -u www-data -l then add /15 * * * * php-cli -f /var/www/nextcloud/cron.php You should be able to update the News nextcloud app feeds. sudo -u www-data php -f /var/www/nextcloud/cron.php so long as "useCronUpdates = true" in nextcloud/cron.php

  1. Then follow your own instructions for news-updater

Useful file paths

/var/www/nextcloud/
/mnt/dietpi_userdata/nextcloud_data
/etc/php5/cli/php.ini

Edits have been made for clarity

BernhardPosselt commented 7 years ago

Can you check if you can call occ at all? My guess is that your php CLI ini does not have all required php extensions enabled

LexiconCode commented 7 years ago

root@DietPi:/var/www/nextcloud# sudo -u www-data php occ

Nextcloud version 11.0.2

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
      --no-warnings     Skip global warnings, show command output only
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
...

php -m

[PHP Modules]
apc
apcu
bcmath
bz2
calendar
Core
ctype
curl
date
dba
dom
ereg
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
json
libxml
mbstring
mcrypt
mhash
mysql
mysqli
openssl
pcntl
pcre
PDO
pdo_mysql
Phar
posix
redis
Reflection
session
shmop
SimpleXML
soap
sockets
SPL
standard
sysvmsg
sysvsem
sysvshm
tokenizer
wddx
xml
xmlreader
xmlwriter
xsl
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

I'm not sure if that's displaying available modules or modules enabled. Either case modules are necessary? I will pass this information on to Dietpi devs and try to make a pull request with the necessary modules.

LexiconCode commented 7 years ago

@BernhardPosselt Is there any other information I can provide you with?

BernhardPosselt commented 7 years ago

Just try to call the api manually, check the readme

LexiconCode commented 7 years ago

I've been taking incremental snapshots of my environment which I've been reverting to testing nextcloud-news-updater incrementally. I discovered re-signing through 'let's encrypt' fixed part of the issue.

The following command now works. The Council output is expected and everything updates and the news app. sudo -u www-data nextcloud-news-updater --loglevel info -c /mnt/usb_1/nextcloud_data/news-updater.ini

"nextcloud-news-updater --loglevel info -c /mnt/usb_1/nextcloud_data/news-updater.ini" outputs 2017-03-02 18:27:48,931 - Nextcloud News Updater - INFO - Running update once with 10 threads 2017-03-02 18:27:48,931 - Nextcloud News Updater - INFO - Running before update command: php -f /var/www/nextcloud/occ -c /etc/php5/cli/php.ini news:updater:before-update 2017-03-02 18:27:49,391 - Nextcloud News Updater - ERROR - Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1: Trying again in 30 seconds Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/updater.py", line 69, in run self.before_update() File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 73, in before_update self.cli.run(self.api.before_cleanup_command) File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 12, in run return check_output(commands) File "/usr/lib/python3.4/subprocess.py", line 620, in check_output raise CalledProcessError(retcode, process.args, output=output) subprocess.CalledProcessError: Command '['php', '-f', '/var/www/nextcloud/occ', '-c', '/etc/php5/cli/php.ini', 'news:updater:before-update']' returned non-zero exit status 1 python3 -m nextcloud_news_updater --loglevel info /var/www/nextcloud

2017-03-02 18:32:57,357 - Nextcloud News Updater - INFO - Running update in an interval of 900 seconds using 10 threads
2017-03-02 18:32:57,358 - Nextcloud News Updater - INFO - Running before update command: php -f /var/www/nextcloud/occ news:updater:before-update
2017-03-02 18:32:57,803 - Nextcloud News Updater - ERROR - Command '['php', '-f', '/var/www/nextcloud/occ', 'news:updater:before-update']' returned non-zero exit status 1: Trying again in 30 seconds
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/updater.py", line 69, in run
    self.before_update()
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 73, in before_update
    self.cli.run(self.api.before_cleanup_command)
  File "/usr/local/lib/python3.4/dist-packages/nextcloud_news_updater/api/cli.py", line 12, in run
    return check_output(commands)
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['php', '-f', '/var/www/nextcloud/occ', 'news:updater:before-update']' returned non-zero exit status 1
BernhardPosselt commented 7 years ago

So have you tried the tipps in the readme yet? Letsencrypt should have nothing to do with that

LexiconCode commented 7 years ago

@BernhardPosselt I have not tried in the tips yet as there was no point as nothing was working up until my last post. I'm not sure what changed my system configuration (The only part I configured in that snapshot was SSL) but I'm thankful it works. Thank you for your help. I'll set it up as a service.

Any conjecture on why I can't seem to use 'nextcloud-news-updater /var/www/nextcloud'?

BernhardPosselt commented 7 years ago

If you read the README you might know ;)

tanghus commented 4 years ago

This did the trick:

# or v2 which is currently a draft
apilevel = v1-2

Thx, @LexiconCode :+1: