Migrate chandumps (from files to database).

bytestostring commented 5 years ago

Server Problem

Hello!

I tried:

[+] Run npm run build-server to regenerate lib/ from src/
[+] Run rm -rf node_modules && npm install to get a fresh install of dependencies
[-] Restarted the server

Description of the Problem

What triggers the problem? I want to migrate my chandump files to the database. I has tried use command: "node lib/channel-storage/migrator.js".
What happens? I got an error (check bellow).
What do you expect to happen instead? Successfully migration :)

System Information

Operating System: CentOS Linux release 7.7.1908 (3.10)
Node Version: (run node -v) v10.16.3
CyTube Version: (displayed at startup) CyTube v3.66.1

Error Messages Displayed:

Migration failed: TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type undefined
at validateString (internal/validators.js:125:11)
at Object.join (path.js:1147:7)
at FileStore.filenameForChannel (/home/www/user/data/sync/lib/channel-storage/filestore.js:38:26)
at FileStore.load (/home/www/user/data/sync/lib/channel-storage/filestore.js:42:27)
at _bluebird.default.reduce (/home/www/user/data/sync/lib/channel-storage/migrator.js:116:18)
at tryCatcher (/home/www/user/data/sync/node_modules/bluebird/js/release/util.js:16:23)
at Object.gotValue (/home/www/user/data/sync/node_modules/bluebird/js/release/reduce.js:168:18)
at Object.gotAccum (/home/www/user/data/sync/node_modules/bluebird/js/release/reduce.js:155:25)
at Object.tryCatcher (/home/www/user/data/sync/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/home/www/user/data/sync/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/home/www/user/data/sync/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromiseCtx (/home/www/user/data/sync/node_modules/bluebird/js/release/promise.js:641:10)
at _drainQueueStep (/home/www/user/data/sync/node_modules/bluebird/js/release/async.js:97:12)
at _drainQueue (/home/www/user/data/sync/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/home/www/user/data/sync/node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (/home/www/user/data/sync/node_modules/bluebird/js/release/async.js:15:14)
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)

Does migrator.js still work?

calzoneman commented 5 years ago

That script is probably broken since it hasn't been used in some years.

It shouldn't be too hard to fix, I've been meaning to do that for a while and deprecate the old flatfile-based channels.

calzoneman commented 5 years ago

Edited OP slightly to fix the formatting of the stack trace.

calzoneman commented 5 years ago

Hi,

I fixed a minor issue that I think was causing it not to work. Please try again now, and reopen if it still doesn't work.

bytestostring commented 5 years ago

Hi, calzoneman!

The script output is normal, but a table "channel_data" is still empty after start the script. I have tried 3-4 times. Before start script I have disabled the CyTube.

node lib/channel-storage/migrator.js
No log backend bound for jsli.  Logging to stdout.
[WARN] config: Missing config key config.listen.0.http; using default: true
[WARN] config: Missing config key config.listen.1; using default: {"ip":"0.0.0.0","port":1337,"io":true}
[WARN] config: No Twitch Client ID set.  Twitch VOD links will not work.  See twitch-client-id in config.template.yaml and https://github.com/justintv/Twitch-API/blob/master/authentication.md#developer-setupfor more information on registering a client ID
[WARN] config: No Mixer Client ID set.  Mixer.com links will not work.  See mixer-client-id in config.template.yaml for more information on registering a client ID
[INFO] config: Loaded configuration from config.yaml
[INFO] config: No camo configuration found, chat images will not be proxied.
[INFO] config: No prometheus configuration found, defaulting to disabled
[INFO] config: No email configuration found, defaulting to disabled
[WARN] metrics: No metrics reporter configured.  Metrics will not be recorded.
Migrated /r/---
Migrated /r/00000
Migrated /r/0012
Migrated /r/1
Migrated /r/animeserge
Migrated /r/animu
Migrated /r/animumu
Migrated /r/ararararagi
Migrated /r/arrdipuppy
Migrated /r/art
Migrated /r/artspace
Migrated /r/as_above_so_below
.....%and_over_100_channels_migrated_without_errors%.....
Migrated /r/zp
Migration complete

bytestostring commented 5 years ago

I can't re-open the issue because closed not by me. Could you re-open it or I need create new issue?

Thank you.

calzoneman commented 5 years ago

Please try the latest commit.

bytestostring commented 5 years ago

Please try the latest commit.

It works! Successfully migration. Thank you very much!

calzoneman / sync