HermanFassett / youtube-scrape

Scrape YouTube searches (API)
MIT License
192 stars 96 forks source link

Crash when executing search #24

Closed vicwomg closed 4 years ago

vicwomg commented 4 years ago

Youtube might have changed something yet again. Looks like the public heroku site is down, and likely because of the same crash I'm seeing locally. When executing a search, this exception brings down the server

Listening on port 8080
/Users/vw/coding/youtube-scrape/scraper.js:34
                        sectionList.itemSectionRenderer.contents.forEach(content => {
                                                        ^

TypeError: Cannot read property 'contents' of undefined
    at /Users/vw/coding/youtube-scrape/scraper.js:34:57
    at Array.forEach (<anonymous>)
    at Request._callback (/Users/vw/coding/youtube-scrape/scraper.js:33:34)
    at Request.self.callback (/Users/vw/coding/youtube-scrape/node_modules/request/request.js:185:22)
    at Request.emit (events.js:316:20)
    at Request.<anonymous> (/Users/vw/coding/youtube-scrape/node_modules/request/request.js:1154:10)
    at Request.emit (events.js:316:20)
    at IncomingMessage.<anonymous> (/Users/vw/coding/youtube-scrape/node_modules/request/request.js:1076:12)
    at Object.onceWrapper (events.js:422:28)
    at IncomingMessage.emit (events.js:328:22)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! youtube-scrape@0.1.0 start: `node server.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the youtube-scrape@0.1.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/vw/.npm/_logs/2020-07-25T08_20_18_028Z-debug.log
HermanFassett commented 4 years ago

Thanks for the issue! Looks like the same issue I found for #21 where some values aren't defined.

I previously made an update on the dev branch that at least catches the error so that app won't crash and I pushed that to the heroku site. It hasn't been merged to master yet though so that'll be why it's crashing your local. The heroku site is actually down because I ran out of free hours :(

vicwomg commented 4 years ago

Thank you! I checked out the the dev branch and that does indeed fix it. I deployed my own heroku for now. I'll probably run into the same hourly cap at some point :/

Not directly related to this ticket but I found that I had to add NPM_CONFIG_PRODUCTION=false to the config otherwise the default heroku configuration pruned away the node_modules. Wondering if you saw that and if there's a better way to fix it.

HermanFassett commented 4 years ago

I have not seen that issue personally, everything worked when deploying on default heroku setup. However, someone else has issue in #15 where it looked like their modules didn't get installed, is this the same issue?

vicwomg commented 4 years ago

Yes, that's exactly it. I will post my suggested fix there, and close this ticket as a dup of #21. Strange through.