[Bug]: Heroku request timeout after 30 secs

[ashutoshkrris]

Is there an existing issue for this?

• [X] I have searched the existing issues.

Current Behavior

Whenever you try downloading a large video from YouTube, or user stories with many items, Heroku request timeout occurs.

According to Heroku:

The countdown for this 30-second timeout begins after the entire request (all request headers and, if applicable, the request body) has been sent from the router to the dyno. The request must then be processed in the dyno by your application, and response delivered back to the router, within 30 seconds to avoid the timeout.

Expected Behavior

There should be no request timeout.

To Reproduce

• Go to YouTube Video Downloader
• Enter any lengthy video URL
• Select the 1080p resolution
• Click on Download
• Wait for 30 seconds and see error

Anything else?

This is a backend related task

Hacktoberfest

• [X] Yeah

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[BidyutBikashBharali]

Hey, I've almost solved this...there will not be any timeout issue...But the problem is heroku free dyno contains limited storage . Since the "download_single_video()" function first download the video and then send the file, so if video size is larger than free dyno storage, new error occurs. @ashutoshkrris

[ashutoshkrris]

Could you share some code?

[BidyutBikashBharali]

Okay....please tell on file yt_downloader.py , line 20 , what does this code do : # If 1080p vid = video.download() os.rename(vid, "video.mp4") audio = url.streams.get_audio_only() aud = audio.download() os.rename(aud, "audio.mp4") os.system("ffmpeg -y -i video.mp4 -i audio.mp4 -c:v copy -c:a aac out.mp4 -loglevel quiet -stats") os.remove("audio.mp4") os.remove("video.mp4") @ashutoshkrris

[ashutoshkrris]

See, sometimes, if we download high quality videos, the audio won't be there. So, we are downloading the video in 1080p and the audio separately. Afterward, we are merging them using ffmpeg.

[BidyutBikashBharali]

Oh...so you have uploaded "ffmpeg" file with codes on heroku?

[ashutoshkrris]

Yeah, that's working.

[BidyutBikashBharali]

Okay, then i can fix #35....but there is high chances of getting error if video size becomes larger than heroku free plan storage......So i think there is no way to get rid of this on heroku except increasing your budget for hosting!

[ashutoshkrris]

Haha You can start working on it. For the time being, we can limit the video size to 150-200MBs. If any day we get enough support, we'll be hosting the application on AWS.

[BidyutBikashBharali]

Sure, please share any idea, resources if you have for " how to continuously check if data is available to load and display on web page". I think this can be done using javascript or Ajax script on frontend part but don't know how to do!

This function will be used to show alert on user interface when file will be available to download

[ashutoshkrris]

That's where I was stuck too. The video was getting downloaded in the background, but unable to send it to the user.

[BidyutBikashBharali]

I've able to send the file to client after downloading process get completed in background but there involves continuous checking process whether data is available or not since I don't know javascript. So, to do this one should have good experience of javascript , Ajax etc. But i would find a way later to achieve this.

[BidyutBikashBharali]

Hey @ashutoshkrris , you can assign me this issue.....I've solved it......everything working working file .........but one minor problem I'm facing >> Thread is not running in heroku due to not knowing how to write Procfile for that.....could u help me in this?

[ashutoshkrris]

Go ahead! Assigning it to you.

[BidyutBikashBharali]

i tried both in Procfile : `web: gunicorn --worker-class eventlet -w 1 main:app

web: gunicorn -w 1 --threads 100 main:app`

but still not running the thread in heroku......could u help me in this case? In local environment everything working perfectly.

[ashutoshkrris]

I'm looking for some references too.

For now, can you try these:

• https://www.py4u.net/discuss/180920
• https://stackoverflow.com/questions/63749430/how-to-implement-threading-with-flask-on-heroku/63749492#63749492

[BidyutBikashBharali]

In my case , thread should be executed only, when user click on download button .....not in before first request

[ashutoshkrris]

I've created a development branch, whenever you push your code, push it to the development branch I'll deploy the development branch, and we'll be able to make changes and test without hampering the production server

[BidyutBikashBharali]

I've fixed this bug today completely .....now how do i have to push it in the new branch and if i send pull request and if get merged then will it be counted today by hacktoberfest community? @ashutoshkrris

on production Video finding is not working , showing this "Unable to fetch the video from YouTube " error

[ashutoshkrris]

When you are making the Pull request, you will see an option to select the branch. Choose the development branch

And yes, it will be counted towards Hacktoberfest You make the PR, let's debug it down completely

[BidyutBikashBharali]

Okay, Please keep checking github frequently tonight up to 11.59 pm. I will try to send PR as soon as possible......now assembling all parts.

[ashutoshkrris]

Yep sure. Go ahead!

[BidyutBikashBharali]

Hey @ashutoshkrris , please check its working perfectly https://bbb-task.herokuapp.com ....now no issue of request time out....but if video size is larger than heroku free storage , only then it will show error.........otherwise it works well

Your current production fails to load video from youtube....after merging it might show error so i think you need to fix it first.

[ashutoshkrris]

Can you find out the bug in the code? If you find it, create an issue and work on it.

[ashutoshkrris]

Yeah, great Looks like the error has gone away

[BidyutBikashBharali]

Did you check it? https://bbb-task.herokuapp.com

[ashutoshkrris]

Yeahhh!! Works perfectly!!

[BidyutBikashBharali]

See, you are trying to download large size video than the existing storage size ....that's why it's not working!....

[ashutoshkrris]

Yeah I tried it.. So, we can set a limit of around 200 MBs

[BidyutBikashBharali]

Yeah, Sure

[BidyutBikashBharali]

And when you are testing in on localhost .....don't forget to set 'https' to 'http' and vice-versa for prod.....in download.html file of youtube

[ashutoshkrris]

Alright

[BidyutBikashBharali]

Hey , where should do PR : on main or development?

[ashutoshkrris]

Main branch

[BidyutBikashBharali]

Now showing this....."This branch has conflicts that must be resolved"

[ashutoshkrris]

Did you make the PR? I suppose you did it in your own repo

[BidyutBikashBharali]

Omg.....how to do this correctly?

[ashutoshkrris]

Go to your repo. You will find an option to create pull request. In the next page, make sure you are making the Pull request to my repo

[BidyutBikashBharali]

Are u taking about to select like this?

[ashutoshkrris]

Yes

[BidyutBikashBharali]

Please resolve the conflict

[ashutoshkrris]

Yes, I'll do it and merge the PR

[BidyutBikashBharali]

Thank you so much brother 👍

[BidyutBikashBharali]

In my Hactoberfest profile , it is showing "project not participating". Why?

[ashutoshkrris]

Can you show a screenshot?

[BidyutBikashBharali]

[ashutoshkrris]

Haha That's not my Repository That's yours.. it's showing there bcz you made a PR on your repo too

[BidyutBikashBharali]

Ohh..i didn't notice the username...how silly i am!

[BidyutBikashBharali]

Will u merge it by today?

[ashutoshkrris]

I'll merge it today That won't affect your Hacktoberfest Contribution anyway.. It will still be valid

[ashutoshkrris]

@ThisIsBBBGithubAc there's a bug The bug for which I had to use ffmpeg

If we download a 1080p video, there's no audio. That's why we need to download audio and video separately and merge them using ffmpeg