search

DialmasterOrg / Youtarr

An application to automate downloading of Youtube videos for Plex
37 stars 1 forks source link

Errors starting and trying to download new videos #84

Closed DeanAtEC closed 1 month ago

DeanAtEC commented 1 month ago

Just tried getting Youtarr running today. Cloned the repo, ran setup.sh without any issues, set the video output directory as instructed. Launched it via start.sh and got it configured and running with a few channels. I went to the Manage Downloads page and clicked Download New From All Channels, and nothing happened. I checked the logs and saw this at the top:

Waiting for DB... Waiting for DB... .240930 18:05:34 mysqld_safe Logging to syslog. 240930 18:05:34 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql Connection has been established successfully. Scheduling task to run at: 0 /4 Auto-downloads enabled, task scheduled! Youtube downloads directory from config: /usr/src/app/data Server listening on port 3011 Error loading jobs from DB: Error at Query.run (/app/node_modules/sequelize/lib/dialects/mysql/query.js:52:25) at /app/node_modules/sequelize/lib/sequelize.js:315:28 at async MySQLQueryInterface.select (/app/node_modules/sequelize/lib/dialects/abstract/query-interface.js:407:12) at async Function.findAll (/app/node_modules/sequelize/lib/model.js:1140:21) at async JobModule.loadJobsFromDB (/app/server/modules/jobModule.js:62:20) { name: 'SequelizeDatabaseError', parent: Error: Table 'youtarr.jobs' doesn't exist at Packet.asError (/app/node_modules/mysql2/lib/packets/packet.js:728:17) at Query.execute (/app/node_modules/mysql2/lib/commands/command.js:29:26) at Connection.handlePacket (/app/node_modules/mysql2/lib/connection.js:481:34) at PacketParser.onPacket (/app/node_modules/mysql2/lib/connection.js:97:12) at PacketParser.executeStart (/app/node_modules/mysql2/lib/packet_parser.js:75:16) at Socket. (/app/node_modules/mysql2/lib/connection.js:104:25) at Socket.emit (events.js:400:28) at addChunk (internal/streams/readable.js:293:12) at readableAddChunk (internal/streams/readable.js:267:9) at Socket.Readable.push (internal/streams/readable.js:206:10) at TCP.onStreamRead (internal/stream_base_commons.js:188:23) { code: 'ER_NO_SUCH_TABLE', errno: 1146, sqlState: '42S02', sqlMessage: "Table 'youtarr.jobs' doesn't exist", sql: 'SELECT id, status, output, timeInitiated, timeCreated, jobType FROM jobs AS Job;', parameters: undefined }, original: Error: Table 'youtarr.jobs' doesn't exist at Packet.asError (/app/node_modules/mysql2/lib/packets/packet.js:728:17) at Query.execute (/app/node_modules/mysql2/lib/commands/command.js:29:26) at Connection.handlePacket (/app/node_modules/mysql2/lib/connection.js:481:34) at PacketParser.onPacket (/app/node_modules/mysql2/lib/connection.js:97:12) at PacketParser.executeStart (/app/node_modules/mysql2/lib/packet_parser.js:75:16) at Socket. (/app/node_modules/mysql2/lib/connection.js:104:25) at Socket.emit (events.js:400:28) at addChunk (internal/streams/readable.js:293:12) at readableAddChunk (internal/streams/readable.js:267:9) Looking for next job to start at Socket.Readable.push (internal/streams/readable.js:206:10) at TCP.onStreamRead (internal/stream_base_commons.js:188:23) { code: 'ER_NO_SUCH_TABLE', errno: 1146, sqlState: '42S02', sqlMessage: "Table 'youtarr.jobs' doesn't exist", sql: 'SELECT id, status, output, timeInitiated, timeCreated, jobType FROM jobs AS Job;', parameters: undefined }, sql: 'SELECT id, status, output, timeInitiated, timeCreated, jobType FROM jobs AS Job;', parameters: {} } New client connected Client disconnected New client connected Client disconnected

I'm not sure if that is expected and is handled by startup database migrations or something? After clicking the download button, I get the following errors in the logs:

New client connected Client disconnected TypeError: Cannot convert undefined or null to object at Function.entries () at JobModule.getRunningJobs (/app/server/modules/jobModule.js:253:28) at /app/server/server.js:172:37 at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at next (/app/node_modules/express/lib/router/route.js:144:13) at verifyToken (/app/server/server.js:43:7) at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at next (/app/node_modules/express/lib/router/route.js:144:13) at Route.dispatch (/app/node_modules/express/lib/router/route.js:114:3) at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at /app/node_modules/express/lib/router/index.js:284:15 at Function.process_params (/app/node_modules/express/lib/router/index.js:346:12) at next (/app/node_modules/express/lib/router/index.js:280:10) at SendStream.error (/app/node_modules/serve-static/index.js:121:7) at SendStream.emit (events.js:400:28) at SendStream.error (/app/node_modules/send/index.js:270:17) New client connected TypeError: Cannot convert undefined or null to object at Function.entries () at JobModule.getRunningJobs (/app/server/modules/jobModule.js:253:28) at /app/server/server.js:201:37 at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at next (/app/node_modules/express/lib/router/route.js:144:13) at verifyToken (/app/server/server.js:43:7) at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at next (/app/node_modules/express/lib/router/route.js:144:13) at Route.dispatch (/app/node_modules/express/lib/router/route.js:114:3) at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at /app/node_modules/express/lib/router/index.js:284:15 at Function.process_params (/app/node_modules/express/lib/router/index.js:346:12) at next (/app/node_modules/express/lib/router/index.js:280:10) at serveStatic (/app/node_modules/serve-static/index.js:75:16) at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at trim_prefix (/app/node_modules/express/lib/router/index.js:328:13) TypeError: Cannot convert undefined or null to object at Function.entries () at JobModule.getRunningJobs (/app/server/modules/jobModule.js:253:28) at /app/server/server.js:172:37 at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at next (/app/node_modules/express/lib/router/route.js:144:13) at verifyToken (/app/server/server.js:43:7) at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at next (/app/node_modules/express/lib/router/route.js:144:13) at Route.dispatch (/app/node_modules/express/lib/router/route.js:114:3) at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5) at /app/node_modules/express/lib/router/index.js:284:15 at Function.process_params (/app/node_modules/express/lib/router/index.js:346:12) at next (/app/node_modules/express/lib/router/index.js:280:10) at SendStream.error (/app/node_modules/serve-static/index.js:121:7) at SendStream.emit (events.js:400:28) at SendStream.error (/app/node_modules/send/index.js:270:17)

Anything I can do on this end to help track this down? Perhaps I've set something up incorrectly or something. Looking forward to getting this going and trying it out!

dialmaster commented 1 month ago

That's interesting. The DB migrations should run on app startup, but based on those errors it appears that they did not run. The very first migration in the migrations directory is the one that creates the jobs table. Can you give me a little more information about your overall system configuration? Are you running on Windows or Linux? If on Windows are you running from WSL2, git bash, powershell, etc? It sounds like the app itself is starting, which is good. Basically just your DB migrations didn't run for some reason on app startup.

You could try running the DB migrations manually by running ./scripts/db-migrate.sh (while the application is already running). I will need to try running a fresh install to see if I can replicate your issue locally, which is why it would be helpful to know what your overall system config is. I probably can't dig in deeper until the weekend though since this was just a side project outside my day job.

DeanAtEC commented 1 month ago

I'm running on Ubuntu Server 22.04.1. I'll give the migration script a shot and see what happens.

One other small thing, in the UI it says there is a new version (v1.17.14) available and to restart the app to update. I restarted the container but it didn't update. I tried removing the container and re-running start.sh again, I can see it grabs the latest image so I'm guessing it hasn't been updated with the latest version yet.

I get it's a side project, I have a bunch of those myself :) I promise not to hound you like I paid for this or anything, and honestly if I knew JS/TypeScript better I'd offer my help!

I'll post back if I make any headway with anything, and thanks for the reply!

DeanAtEC commented 1 month ago

Looks like the db-migrate.sh script is only looking for a container named youtarr-dev and not youtarr.

dialmaster commented 1 month ago

Aha. OK, a few things: The migrate script is designed more for development than for running against the built container from docker hub, but it should work.

It runs a command within the running container that should manually run all the migrations.

I think you can just run this command (from the directory where youtarr is installed, outside the container): docker exec youtarr npx sequelize-cli db:migrate --config config/dbconfig.json ^^ Note 'youtarr' instead of 'yourarr-dev'. You can ignore the "update available bit", but I will fix that this weekend as well just to bring everything up to date :) And... you are right, this is indeed a side project and one that I really just made for me to be able to manage auto-downloading youtube stuff for myself/my kids on Plex, but I shared it so others could also benefit from it, so, I will look into your issue this weekend and see if I can make a fix :)

Hopefully, in the meantime, you can just manually run the migrations and get it up and running :)

DeanAtEC commented 1 month ago

I was hoping so too :( Apparently it's already up to date:

Sequelize CLI [Node: 14.21.3, CLI: 6.6.0, ORM: 6.31.1]

Loaded configuration file "config/dbconfig.json". Using environment "development". No migrations were executed, database schema was already up to date.

I logged into MySql and there are a number of tables there:

MariaDB [youtarr]> show tables; +-------------------+ | Tables_in_youtarr | +-------------------+ | JobVideos | | Jobs | | SequelizeMeta | | Videos | | channels | | channelvideos | +-------------------+ 6 rows in set (0.000 sec)

The Jobs table is empty, unsure whether that is expected or not. I'll wait and see if you get the time to look into anything. Let me know if you need me to try anything else or need more info, happy to help any way I can!

dialmaster commented 1 month ago

Nope, this is plenty of information for now. I think my best bet is to fire up a Ubuntu 22 VM and try a fresh install in there and see what happens (and hopefully I can reproduce your issue).

It's odd because apparently the database and table exist as expected, given what you reported, but the error you reported initially indicates the table does not exist (which clearly it does). It's almost as if the running process is not connecting to the same DB that the migrations ran against (?)

I'll try to take a look in the next couple of days.

dialmaster commented 1 month ago

I was able to reproduce your issue and just pushed up a fix that works for me locally. I pushed it up as a new release to docker hub. Stop your running docker instance and then re-run ./start.sh and let me know if it works for you now.

DeanAtEC commented 1 month ago

That seems like it did the trick! Logs look like there are videos being downloaded now, although none have appeared yet in the output folder, it looks like your fix worked! Thanks so much!

dialmaster commented 1 month ago

OK, let me know if you run into any other issues. The only one I have personally seen is related to youtube -- it seems to block downloading videos over some VPNs, but as long as I run Youtarr somewhere that is not behind a VPN I don't have any issues. This is an evolving thing though -- I've had to update Youtarr multiple times as Youtube has changed things on their end. Please update me if all is well and I will close this issue

DeanAtEC commented 1 month ago

Looks like it's working now! I did noticed the output path was not working, but after so much churn here, I'm going to wipe everything out and start again and see if it happens again or is just me fat-fingering something during testing. Thanks again for the fast turnaround on this one!