20000 articles failed to insert! [Blacklight\Binaries::scan]

[Sweepr]

Describe the bug

Compression: On/Off makes no difference. M.G.R: On/Off makes no difference.

multigroup_missed_parts: being filled missed_parts: being filled

binaries/parts/collections stay empty.

Getting 20,000 articles (6,505,165,282 to 6,505,185,281) from alt.binaries.movies - (80,000 articles in queue). Received 20000 articles of 20,000 requested, 0 blacklisted, 0 not yEnc. 20000 articles failed to insert! [Blacklight\Binaries::scan] 1s to download articles, 77s to process collections, 0s to insert binaries/parts, 0s for part repair, 79s total.

Updating: 1 group(s) - Using compression? No [Blacklight\Binaries::updateAllGroups] Starting group 1 of 1 [Blacklight\Binaries::updateAllGroups]

To Reproduce

Don't know

Expected behavior

Successfull inserts

[Sweepr]

https://www.reddit.com/r/NNTmux/comments/a8cyo2/help_with_new_install/

Looks like a similar issue.

[DariusIII]

This is not always a bug. Can be, but it also can be a problem on your server.

On a side note, please use the issue template and state all of the required information.

[Sweepr]

Describe the bug

"Blacklight/Binaries::scan is reporting '20000 articles failed to insert!"

20000 articles failed to insert! [Blacklight\Binaries::scan]

To Reproduce

I tried to deleted the entire DB 3x, re-installed using php artisan nntmux:install 3x, went tru fine.

I can login, change edit/settings, activate groups, set up Tmux, browse the site but as soon as i want to run update_binaries or run Tmux i get

I'm also running sort a-like indexer using the same Installation , and that one is running and inserting fine.

Expected behavior

Inserting parts into proper table without warning message.

Screenshots Preparing group:

Starting update_binaries.php:

After 5-10 min's articles fail to insert:

First/last record has changed:

Everything end's up in multigroup_missed_parts table:

Installation (please complete the following information):

php -v ea-php-cli Copyright 2017 cPanel, Inc. PHP 7.2.13 (cli) (built: Dec 10 2018 18:08:17) ( NTS ) Copyright (c) 1997-2018 The PHP Group Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies with Xdebug v2.6.1, Copyright (c) 2002-2018, by Derick Rethans

cat /etc/centos-release CentOS release 6.10 (Final)

mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 262903 Server version: 10.2.20-MariaDB MariaDB Server

my.cnf file:

Additional context

Database server

Server: Localhost via UNIX socket
Server type: MariaDB
Server connection: SSL is not being used Documentation
Server version: 10.2.20-MariaDB - MariaDB Server
Protocol version: 10
User: ********@localhost
Server charset: UTF-8 Unicode (utf8)

Web server

cpsrvd 11.76.0.14
Database client version: libmysql - 5.1.73
PHP extension: mysqliDocumentation curlDocumentation mbstringDocumentation
PHP version: 7.2.7

phpMyAdmin

Version information: 4.8.3

composer show --all

installed: aharen/omdbapi v2.0.8 PHP package for OMDbAPI.com by Brian Fritz alchemy/binary-driver v2.0.0 A set of tools to build binary drivers anhskohbo/no-captcha 3.0.3 No CAPTCHA reCAPTCHA For Laravel. b3rs3rk/steamfront dev-master 286962e PHP Wrapper for Accessing the Steam Storefront API barracudanetworks/forkdaemon-php v1.1.1 A library to make setup and management of forking daemons in PHP easy. barryvdh/laravel-debugbar v3.2.1 PHP Debugbar integration for Laravel barryvdh/laravel-ide-helper v2.5.3 Laravel IDE Helper, generates correct PHPDocs for all Facade classes, to improve auto-completion. barryvdh/reflection-docblock v2.0.6 beyondcode/laravel-dump-server 1.2.2 Symfony Var-Dump Server for Laravel bhuvidya/laravel-countries v1.0.4 Laravel Countries is a bundle for Laravel, providing Almost ISO 3166_2, 3166_3, currency, Capital an... canihavesomecoffee/thetvdbapi 1.0.9 PHP implementation for the 2nd version of the TheTVDb API. chumper/zipper v1.0.2 This is a little neat helper for the ZipArchive methods with handy functions clue/stream-filter v1.4.0 A simple and modern approach to stream filtering in PHP composer/ca-bundle 1.1.3 Lets you find a path to the system CA bundle, and includes a fallback to the Mozilla CA bundle. composer/composer 1.8.0 Composer helps you declare, manage and install dependencies of PHP projects, ensuring you have the r... composer/semver 1.4.2 Semver library that offers utilities, version constraint parsing and validation. composer/spdx-licenses 1.5.0 SPDX licenses list and validation library. composer/xdebug-handler 1.3.1 Restarts a process without xdebug. czproject/git-php v3.14.0 Library for work with Git repository in PHP. dariusiii/php-itunes-api v1.0.6 Query iTunes API System dariusiii/rarinfo v2.6.1 PHP archive reader helper class dborsatto/php-giantbomb v1.0.2 A PHP library that acts as a wrapper for the GiantBomb API. dirape/token v2.1 Unique Token Generator For Laravel dnoegel/php-xdg-base-dir 0.1 implementation of xdg base directory specification for php doctrine/annotations v1.6.0 Docblock Annotations Parser doctrine/cache v1.8.0 Caching library offering an object-oriented API for many cache backends doctrine/dbal v2.9.1 Powerful PHP database abstraction layer (DBAL) with many features for database schema introspection ... doctrine/event-manager v1.0.0 Doctrine Event Manager component doctrine/inflector v1.3.0 Common String Manipulations with regard to casing and singular/plural rules. doctrine/instantiator 1.1.0 A small, lightweight utility to instantiate objects in PHP without invoking their constructors doctrine/lexer v1.0.1 Base library for a lexer that can be used in Top-Down, Recursive Descent Parsers. dragonmantank/cron-expression v2.2.0 CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due egulias/email-validator 2.1.7 A library for validating emails against several RFCs erusev/parsedown 1.7.1 Parser for Markdown. evenement/evenement v2.1.0 Événement is a very simple event dispatching library for PHP exeu/apai-io 2.2.0 Amazon Product Advertising PHP Library fideloper/proxy 4.0.0 Set trusted proxies for Laravel filp/whoops 2.3.1 php error handling for cool kids foolz/sphinxql-query-builder 2.0.0 A PHP query builder for SphinxQL. Uses MySQLi to connect to the Sphinx server. friendsofphp/php-cs-fixer v2.13.1 A tool to automatically fix PHP code style fzaninotto/faker v1.8.0 Faker is a PHP library that generates fake data for you. genealabs/laravel-caffeine 0.7.1 Keeping Your Laravel Forms Awake geoip2/geoip2 v2.9.0 MaxMind GeoIP2 PHP API google/recaptcha 1.2.1 Client library for reCAPTCHA, a free service that protects websites from spam and abuse. guzzlehttp/guzzle 6.3.3 Guzzle is a PHP HTTP client library guzzlehttp/promises v1.3.1 Guzzle promises library guzzlehttp/psr7 1.5.2 PSR-7 message implementation that also provides common utility methods hamcrest/hamcrest-php v2.0.0 This is the PHP port of Hamcrest Matchers imdbphp/imdbphp v6.1.3 Library for retrieving film and tv information from IMDb intervention/image 2.4.2 Image handling and manipulation library with support for Laravel integration intervention/imagecache 2.3.3 Caching extension for the Intervention Image Class jakub-onderka/php-console-color v0.2 jakub-onderka/php-console-highlighter v0.4 Highlight PHP code in terminal james-heinrich/getid3 v1.9.16 PHP script that extracts useful information from popular multimedia file formats jean85/pretty-package-versions 1.2 A wrapper for ocramius/package-versions to get pretty versions strings jenssegers/model v1.2.0 An Laravel eloquent-like model class, for Laravel and other frameworks jeremeamia/SuperClosure 2.4.0 Serialize Closure objects, including their context and binding joshpinkney/tv-maze-php-api dev-master 1744293 TVMaze-API-Wrapper jrean/laravel-user-verification v7.0.1 User Email Verification For Laravel 5.* justinrainbow/json-schema 5.2.7 A library to validate a json schema. kevinlebrun/colors.php 1.0.3 Colors for PHP CLI scripts kevinrob/guzzle-cache-middleware v2.1.1 A HTTP/1.1 Cache for Guzzle 6. It's a simple Middleware to be added in the HandlerStack. (RFC 7234) kylekatarnls/laravel-carbon-2 1.0.0 Carbon 2 adapter for Laravel laravel/framework v5.7.19 The Laravel Framework. laravel/nexmo-notification-channel v1.0.1 Nexmo Notification Channel for laravel. laravel/scout v6.1.3 Laravel Scout provides a driver based solution to searching your Eloquent models. laravel/slack-notification-channel v1.0.3 Slack Notification Channel for laravel. laravel/tinker v1.0.8 Powerful REPL for the Laravel framework. laravelcollective/html v5.7.1 HTML and Form Builders for the Laravel Framework lcobucci/jwt 3.2.5 A simple library to work with JSON Web Token and JSON Web Signature league/climate 3.4.1 PHP's best friend for the terminal. CLImate allows you to easily output colored text, special format... league/csv 8.0.0 Csv data manipulation made easy in PHP league/flysystem 1.0.49 Filesystem abstraction: Many filesystems, one API. league/fractal 0.17.0 Handle the output of complex data structures ready for API output. maximebf/debugbar v1.15.0 Debug bar in the browser for php application maxmind-db/reader v1.4.0 MaxMind DB Reader API maxmind/web-service-common v0.5.0 Internal MaxMind Web Service API mayconbordin/l5-fixtures dev-master 767b37f A fixtures package for Laravel 5 messerli90/igdb 1.0.0 Laravel PHP Facade/Wrapper for the IGDB API mhor/php-mediainfo 4.1.2 PHP wrapper around the mediainfo command mockery/mockery 1.2.0 Mockery is a simple yet flexible PHP mock object framework monolog/monolog 1.24.0 Sends your logs to files, sockets, inboxes, databases and various web services moontoast/math 1.1.2 A mathematics library, providing functionality for large numbers myclabs/deep-copy 1.8.1 Create deep copies (clones) of your objects nesbot/carbon 2.6.1 A simple API extension for DateTime. nette/bootstrap v2.4.6 🅱 Nette Bootstrap: the simple way to configure and bootstrap your Nette application. nette/di v2.4.14 💎 Nette Dependency Injection Container: Flexible, compiled and full-featured DIC with perfectly u... nette/finder v2.4.2 🔍 Nette Finder: find files and directories with an intuitive API. nette/neon v2.4.3 🍸 Nette NEON: encodes and decodes NEON file format. nette/php-generator v3.0.5 🐘 Nette PHP Generator: generates neat PHP code for you. Supports new PHP 7.2 features. nette/robot-loader v3.1.0 🍀 Nette RobotLoader: high performance and comfortable autoloader that will search and autoload cl... nette/utils v2.5.3 🛠 Nette Utils: lightweight utilities for string & array manipulation, image handling, safe JSON e... neutron/temporary-filesystem 2.3.0 Symfony filesystem extension to handle temporary files nexmo/client 1.6.0 PHP Client for using Nexmo's API. nikic/php-parser v4.1.1 A PHP parser written in PHP ntimyeboah/laravel-database-trigger v1.0.0 A package to add database trigger to laravel migrations nunomaduro/collision v2.1.1 Cli error handling for console/command-line PHP applications. nunomaduro/larastan v0.3.13 Larastan - Discover bugs in your code without running it. A phpstan/phpstan wrapper for Laravel ocramius/package-versions 1.3.0 Composer plugin that provides efficient querying for installed package versions (no runtime IO) opis/closure 3.1.2 A library that can be used to serialize closures (anonymous functions) and arbitrary objects. orangehill/iseed v2.6.1 Generate a new Laravel database seed file based on data from the existing database table. orchestra/testbench v3.7.6 Laravel Testing Helper for Packages Development orchestra/testbench-core v3.7.7 Testing Helper for Laravel Development paragonie/random_compat v9.99.99 PHP 5.x polyfill for random_bytes() and random_int() from PHP 7 phar-io/manifest 1.0.3 Component for reading phar.io manifest information from a PHP Archive (PHAR) phar-io/version 2.0.1 Library for handling version information and constraints php-coveralls/php-coveralls v2.1.0 PHP client library for Coveralls API php-cs-fixer/diff v1.3.0 sebastian/diff v2 backport support for PHP5.6 php-ffmpeg/php-ffmpeg v0.13 FFMpeg PHP, an Object Oriented library to communicate with AVconv / ffmpeg php-http/guzzle6-adapter v1.1.1 Guzzle 6 HTTP Adapter php-http/httplug v1.1.0 HTTPlug, the HTTP client abstraction for PHP php-http/message 1.7.2 HTTP Message related tools php-http/message-factory v1.0.2 Factory interfaces for PSR-7 HTTP Message php-http/promise v1.0.0 Promise used for asynchronous HTTP requests php-tmdb/api 2.1.16 PHP wrapper for TMDB (TheMovieDatabase) API v3. Supports two types of approaches, one modelled with ... php-tmdb/laravel v1.1.2 Laravel Package for TMDB ( The Movie Database ) API. Provides easy access to the wtfzdotnet/php-tmdb... phpdocumentor/reflection-common 1.0.1 Common reflection classes used by phpdocumentor to reflect the code structure phpdocumentor/reflection-docblock 4.3.0 With this component, a library can provide support for annotations via DocBlocks or otherwise retrie... phpdocumentor/type-resolver 0.4.0 phpspec/prophecy 1.8.0 Highly opinionated mocking framework for PHP 5.3+ phpstan/phpdoc-parser 0.3 PHPDoc parser with support for nullable, intersection and generic types phpstan/phpstan 0.10.7 PHPStan - PHP Static Analysis Tool phpunit/php-code-coverage 6.1.4 Library that provides collection, processing, and rendering functionality for PHP code coverage info... phpunit/php-file-iterator 2.0.2 FilterIterator implementation that filters files based on a list of suffixes. phpunit/php-text-template 1.2.1 Simple template engine. phpunit/php-timer 2.0.0 Utility class for timing phpunit/php-token-stream 3.0.1 Wrapper around PHP's tokenizer extension. phpunit/phpunit 7.5.1 The PHP Unit Testing framework. pragmarx/firewall v2.2.1 A Laravel IP whitelisting and blacklisting pragmarx/support v0.8.8 PragmaRX components support package predis/predis v1.1.1 Flexible and feature-complete Redis client for PHP and HHVM propaganistas/laravel-disposable-email 2.0.3 Disposable email validator psr/container 1.0.0 Common Container Interface (PHP FIG PSR-11) psr/http-message 1.0.1 Common interface for HTTP messages psr/log 1.1.0 Common interface for logging libraries psr/simple-cache 1.0.1 Common interfaces for simple caching psy/psysh v0.9.9 An interactive shell for modern PHP. ralouphie/getallheaders 2.0.5 A polyfill for getallheaders. ramsey/uuid 3.8.0 Formerly rhumsaa/uuid. A PHP 5.4+ library for generating RFC 4122 version 1, 3, 4, and 5 universally... rtconner/laravel-tagging 3.0.0 Use PHP traits to extend Laravel Eloquent Models to allow Tags. Models can be marked as Taggable. rtheunissen/guzzle-log-middleware v0.4.1 Guzzle middleware to log requests and responses sebastian/code-unit-reverse-lookup 1.0.1 Looks up which function or method a line of code belongs to sebastian/comparator 3.0.2 Provides the functionality to compare PHP values for equality sebastian/diff 3.0.1 Diff implementation sebastian/environment 4.0.1 Provides functionality to handle HHVM/PHP environments sebastian/exporter 3.1.0 Provides the functionality to export PHP variables for visualization sebastian/global-state 2.0.0 Snapshotting of global state sebastian/object-enumerator 3.0.3 Traverses array structures and object graphs to enumerate all referenced objects sebastian/object-reflector 1.1.1 Allows reflection of object attributes, including inherited and non-public ones sebastian/recursion-context 3.0.0 Provides functionality to recursively process PHP variables sebastian/resource-operations 2.0.1 Provides a list of PHP built-in functions that operate on resources sebastian/version 2.0.1 Library that helps with managing the version number of Git-hosted PHP projects seld/cli-prompt 1.0.3 Allows you to prompt for user input on the command line, and optionally hide the characters they type seld/jsonlint 1.7.1 JSON Linter seld/phar-utils 1.0.1 PHAR file format utilities, for when PHP phars you up smarty/smarty v3.1.33 Smarty - the compiling PHP template engine spatie/fractalistic 2.7.2 A developer friendly wrapper around Fractal spatie/laravel-directory-cleanup 1.2.2 This package will remove the expired files from the given directories. spatie/laravel-fractal 5.4.2 An easy to use Fractal integration for Laravel applications spatie/laravel-permission 2.29.0 Permission handling for Laravel 5.4 and up swiftmailer/swiftmailer v6.1.3 Swiftmailer, free feature-rich PHP mailer symfony/config v4.2.1 Symfony Config Component symfony/console v4.2.1 Symfony Console Component symfony/contracts v1.0.2 A set of abstractions extracted out of the Symfony components symfony/css-selector v4.2.1 Symfony CssSelector Component symfony/debug v4.2.1 Symfony Debug Component symfony/event-dispatcher v4.2.1 Symfony EventDispatcher Component symfony/filesystem v4.2.1 Symfony Filesystem Component symfony/finder v4.2.1 Symfony Finder Component symfony/http-foundation v4.2.1 Symfony HttpFoundation Component symfony/http-kernel v4.2.1 Symfony HttpKernel Component symfony/inflector v4.2.1 Symfony Inflector Component symfony/options-resolver v4.2.1 Symfony OptionsResolver Component symfony/polyfill-ctype v1.10.0 Symfony polyfill for ctype functions symfony/polyfill-mbstring v1.10.0 Symfony polyfill for the Mbstring extension symfony/polyfill-php56 v1.10.0 Symfony polyfill backporting some PHP 5.6+ features to lower PHP versions symfony/polyfill-php70 v1.10.0 Symfony polyfill backporting some PHP 7.0+ features to lower PHP versions symfony/polyfill-php72 v1.10.0 Symfony polyfill backporting some PHP 7.2+ features to lower PHP versions symfony/polyfill-util v1.10.0 Symfony utilities for portability of PHP codes symfony/process v4.2.1 Symfony Process Component symfony/property-access v3.4.20 Symfony PropertyAccess Component symfony/property-info v3.4.20 Symfony Property Info Component symfony/routing v4.2.1 Symfony Routing Component symfony/serializer v3.4.20 Symfony Serializer Component symfony/stopwatch v4.2.1 Symfony Stopwatch Component symfony/translation v4.2.1 Symfony Translation Component symfony/var-dumper v4.2.1 Symfony mechanism for exploring and dumping PHP variables symfony/yaml v4.2.1 Symfony Yaml Component theseer/tokenizer 1.1.0 A small library for converting tokenized PHP source code into XML and potentially other formats tijsverkoyen/css-to-inline-styles 2.2.1 CssToInlineStyles is a class that enables you to convert HTML-pages/files into HTML-pages/files with... vlucas/phpdotenv v2.5.1 Loads environment variables from .env to getenv(), $_ENV and $_SERVER automagically. voku/simple_html_dom 4.1.10 Simple HTML DOM package. wandersonwhcr/illuminate-romans v1.0.2 Laravel Illuminate Romans Integration wandersonwhcr/romans v1.1.5 A Simple PHP Roman Numerals Library watson/rememberable 2.0.4 Query caching for Laravel 5 webmozart/assert 1.4.0 Assertions to validate method input/output with nice error messages. xethron/laravel-4-generators 3.1.1 Rapidly generate resources, migrations, models, and much more. xethron/migrations-generator v2.0.2 Generates Laravel Migrations from an existing database yab/laravel-scout-mysql-driver v2.1.0 MySQL driver for Laravel Scout. yadakhov/insert-on-duplicate-key v1.2.0 A trait for MySQL INSERT ON DUPLICATE KEY UPDATE. ytake/laravel-smarty 2.4.0 Smarty template engine for Laravel and Lumen zendframework/zend-diactoros 1.8.6 PSR HTTP Message implementations

If you require anything else please let me know.

[DariusIII]

First, disable backfill for now. Second, like i said, most of the times, failing to insert is an database problem. Btw, 4 GB of RAM is not enough, so issues can come from that.

NNTmux should be run on dedicated server, not on VPS.

[Sweepr]

Disabled backfill for now, just to show that part is working OK.

I understand, but i'm not that rich ;-) it's just for fun and i'm only indexing 1 or 2 groups at a time for private use so shouldn't be an issue to run on a VPS.

Anyway, Running the scraper on Tmux show me this:

Altough they get inserted:

Tmux-UI:

[DariusIII]

Have you read the install instruction on nntmux github wiki page? That predb_rt error means that you did not tun sphinxsearch required scripts.

[Sweepr]

Right, i overlooked the word [Mandatory] , since i thought it was only usefull for speeding up website based searches.

[Sweepr]

Ok, the Sphinx part is working fine now, scraping the channels without error's

[Sweepr]

php artisan nntmux:resetdb

This script removes all releases, nzb files, samples, previews , nfos, truncates all article tables and resets all groups. Are you sure you want reset the DB? (yes/no) [no]:

yes

Reseting all groups completed. Truncating videos completed. Truncating tv_episodes completed. Truncating tv_info completed. Truncating release_nfos completed. Truncating release_comments completed. Truncating sharing completed. Truncating sharing_sites completed. Truncating users_releases completed. Truncating user_movies completed. Truncating user_series completed. Truncating movieinfo completed. Truncating musicinfo completed. Truncating release_files completed. Truncating audio_data completed. Truncating release_subtitles completed. Truncating video_data completed. Truncating releaseextrafull completed. Truncating releases completed. Truncating anidb_titles completed. Truncating anidb_info completed. Truncating anidb_episodes completed. Truncating releases_groups completed. Truncating binaries, collections, missed_parts and parts tables... Truncating completed. Deleting nzbfiles subfolders. Deleting all images, previews and samples that still remain. Deleted all releases, images, previews and samples. This script finished 2 seconds after start

php artisan nntmux:db Updating database Nothing to migrate.

php artisan migrate:status +------+--------------------------------------------------------+-------+ | Ran? | Migration | Batch | +------+--------------------------------------------------------+-------+ | Yes | 2014_01_07_073615_create_tagged_table | 1 | | Yes | 2014_01_07_073615_create_tags_table | 1 | | Yes | 2014_02_01_311070_create_firewall_table | 1 | | Yes | 2016_06_29_073615_create_tag_groups_table | 1 | | Yes | 2016_06_29_073615_update_tags_table | 1 | | Yes | 2017_11_29_223842_create_countries_table | 1 | | Yes | 2018_01_16_195548_create_users_table | 1 | | Yes | 2018_01_17_150719_create_permission_tables | 1 | | Yes | 2018_01_17_154034_create_categories_table | 1 | | Yes | 2018_01_18_101314_create_category_regexes_table | 1 | | Yes | 2018_01_18_102213_create_collection_regexes_table | 1 | | Yes | 2018_01_18_102716_create_binaryblacklist_table | 1 | | Yes | 2018_01_18_103104_create_content_table | 1 | | Yes | 2018_01_18_103520_create_forumpost_table | 1 | | Yes | 2018_01_18_103816_create_genres_table | 1 | | Yes | 2018_01_18_104345_create_groups_table | 1 | | Yes | 2018_01_18_104710_create_menu_table | 1 | | Yes | 2018_01_18_105455_create_release_naming_regexes_table | 1 | | Yes | 2018_01_18_105834_create_settings_table | 1 | | Yes | 2018_01_20_195500_create_collections_table | 1 | | Yes | 2018_01_20_195528_create_releases_table | 1 | | Yes | 2018_01_20_195604_create_anidb_episodes_table | 1 | | Yes | 2018_01_20_195615_create_anidb_info_table | 1 | | Yes | 2018_01_20_195624_create_anidb_titles_table | 1 | | Yes | 2018_01_20_195636_create_audio_data_table | 1 | | Yes | 2018_01_20_195648_create_binaries_table | 1 | | Yes | 2018_01_20_195703_create_bookinfo_table | 1 | | Yes | 2018_01_20_195716_create_consoleinfo_table | 1 | | Yes | 2018_01_20_195728_create_dnzb_failures_table | 1 | | Yes | 2018_01_20_195739_create_gamesinfo_table | 1 | | Yes | 2018_01_20_195752_create_invitations_table | 1 | | Yes | 2018_01_20_195801_create_logging_table | 1 | | Yes | 2018_01_20_195812_create_missed_parts_table | 1 | | Yes | 2018_01_20_195822_create_movieinfo_table | 1 | | Yes | 2018_01_20_195832_create_musicinfo_table | 1 | | Yes | 2018_01_20_195846_create_multigroup_collections_table | 1 | | Yes | 2018_01_20_195858_create_multigroup_posters_table | 1 | | Yes | 2018_01_20_195915_create_par_hashes_table | 1 | | Yes | 2018_01_20_195925_create_parts_table | 1 | | Yes | 2018_01_20_195934_create_predb_table | 1 | | Yes | 2018_01_20_195946_create_predb_hashes_table | 1 | | Yes | 2018_01_20_195954_create_predb_imports_table | 1 | | Yes | 2018_01_20_200005_create_release_comments_table | 1 | | Yes | 2018_01_20_200018_create_releases_groups_table | 1 | | Yes | 2018_01_20_200030_create_release_regexes_table | 1 | | Yes | 2018_01_20_200038_create_release_unique_table | 1 | | Yes | 2018_01_20_200046_create_releaseextrafull_table | 1 | | Yes | 2018_01_20_200056_create_release_files_table | 1 | | Yes | 2018_01_20_200104_create_release_nfos_table | 1 | | Yes | 2018_01_20_200124_create_release_subtitles_table | 1 | | Yes | 2018_01_20_200135_create_sharing_table | 1 | | Yes | 2018_01_20_200142_create_sharing_sites_table | 1 | | Yes | 2018_01_20_200151_create_short_groups_table | 1 | | Yes | 2018_01_20_200200_create_steam_apps_table | 1 | | Yes | 2018_01_20_200211_create_tv_episodes_table | 1 | | Yes | 2018_01_20_200218_create_tv_info_table | 1 | | Yes | 2018_01_20_200237_create_users_releases_table | 1 | | Yes | 2018_01_20_200248_create_user_downloads_table | 1 | | Yes | 2018_01_20_200318_create_user_movies_table | 1 | | Yes | 2018_01_20_200328_create_user_requests_table | 1 | | Yes | 2018_01_20_200336_create_user_series_table | 1 | | Yes | 2018_01_20_200346_create_video_data_table | 1 | | Yes | 2018_01_20_200353_create_videos_table | 1 | | Yes | 2018_01_20_200403_create_videos_aliases_table | 1 | | Yes | 2018_01_20_200417_create_xxxinfo_table | 1 | | Yes | 2018_01_20_200431_create_multigroup_binaries_table | 1 | | Yes | 2018_01_20_200442_create_multigroup_parts_table | 1 | | Yes | 2018_01_20_200451_create_multigroup_missed_parts_table | 1 | | Yes | 2018_01_22_220858_add_stored_procedures | 1 | | Yes | 2018_04_24_132758_create_cache_table | 1 | | Yes | 2018_09_13_070520_add_verification_to_user_table | 1 | | Yes | 2018_12_20_221744_create_check_insert_trigger | 1 | | Yes | 2018_12_20_222607_create_check_update_trigger | 1 | | Yes | 2018_12_20_222736_create_check_rfinsert_trigger | 1 | | Yes | 2018_12_20_222836_create_check_rfupdate_trigger | 1 | | Yes | 2018_12_20_222936_create_insert_hashes_trigger | 1 | | Yes | 2018_12_20_223037_create_update_hashes_trigger | 1 | | Yes | 2018_12_20_223131_create_delete_hashes_trigger | 1 | | Yes | 2018_12_29_160700_create_sessions_table | 1 | +------+--------------------------------------------------------+-------+

So that's seems fine, tried a different U.S.P. provider, tried SSL enabled, no avail.

I have an nZEDb Release v0.8.13.1 instance running on the same system just fine crawling/scraping doing what a spider-pig does using the same PHP 7.2.13 PHP-FPM and CLI settings, using MariaDB 10.2.20 .. so something fishy is going on.

Also have a few website's running using MyIsam/InnoDB mixed DB, MyIsam only DB, InnoDB only DB, they all work fine.

What else can i try to debunk this, any idea's/suggestions?

[ghost]

think its a centos thing getting near enough the same. binaries not being inserted,

blacklight::scan

only error i can see.

also on dev

no mgr option to test or its missing

[ghost]

It almost seems like it.

What are you server specs? OS, PHP version, SQL

MGR option should be under "edit site - release settings"

[Sweepr]

Been trying alot of things, different USP, ssl on/off , lowered the part amount to 5000, reinstallin 3 times from scratch, also the predb import script is bugging on me, is it working at your side?

[DariusIII]

Are you guys on master? MGR will be removed in next release ,which i will push tomorrow, most likely.

[ghost]

nope on dev:

no mgr, did try adding still no joy same error. with dev/master

centos 7 php 7.2 tmux 2.8

pre half working..

[DariusIII]

There is no mgr anymore. Is debugging enabled? Issue here is that headers are not inserted into collections table or binaries table is not getting populated. Can you check if parts are getting inserted?

[ghost]

no parts being inserted but missed parts there being inserted.

[DariusIII]

Can you enable debugging? Set APP_ENV = local and APP_DEBUG = true and see if any error pops out and check logs in storage/logs/laravel*.log

[ghost]

no, all seems strange as only error any where even in logs is the one in binaries panel in tmux.

[DariusIII]

Can you paste me the error from logs? Complete error.

[ghost]

there are no errors in log ? just panel in tmux, also just noticed the irc scraper panel in tmux saying, mysql 2002 :connection refused if any use. strange as was working.

[Sweepr]

Same here. no errors created by laravel when set env. app local debug true..

Is your sql service running? you could try to set the sql servername to localhost, or the IP address, double check your sql login settings.

[DariusIII]

Can any of you two give me access to your server? If yes, send me details to my email listed on github.

[DariusIII]

I have pushed a change to Binaries, so errors should be logged now for easier debugging.

[Sweepr]

Thanks, i'll check that out as soon as possible, currently afk @ work.. :unamused:

[Sweepr]

I just tried that change, https://github.com/NNTmux/newznab-tmux/commit/4ec6d8ea8290812f7e6ae9fc39f8f1d9eb0a9ba7?diff=split but i'm still not seeing anything special being logged other that 20000 articles failed to insert! [Blacklight\Binaries::scan].

Where and what should it log ?

[DariusIII]

It should be in storage/logs/laravel.log (or laravel_current_date.log)

[DariusIII]

If you are backfilling, please don't. Omicron had issues and headers did not have Bytes property set, Look only at first binaries pane.

[Sweepr]

Ah. i tried to run the update.binaries.php from cli.. i'll try Tmux now.

[ghost]

2019-01-10 10:54:25] NNTmux.ERROR: Call to undefined function Blacklight\sodium_bin2hex()

taken from log

think its a problem with not having libsodium installed ?

[DariusIII]

Yup. Although, PHP 7.2 should be compiled with it. Try instaling php-sodium.

[ghost]

installing libsodium has solved problem for me, thanks 👍

[Sweepr]

Alright, then i know what i should do, i'll report back when i have things running.

[Sweepr]

Problem solved here to, seems like Cpanel forgot to add the libsodium to php7.2. sighhh.. adding it solved the issue..

Thanks Darius!!