Add support for BBB with authentification

[dmrbatuhan]

Hello, I could download the videos with bbb-dl a week ago. But today, it gives a error like this: bbb-dl https://scalelite.sdu2bbb.online/playback/presentation/2.3/ f7932ffc29b496194e314fab60b388ca8978a87e-1678428387252
Output directory for the final video is: /root
Directory for the temporary files is: /root/.local/share/bbb-dl/383011d326cfa600 3148de9fd405d1c4
Downloading meta information
Downloading metadata.xml...
Downloading shapes.svg...
shapes.svg could not be downloaded: 404 Not Found
metadata.xml could not be downloaded: 404 Not Found
Error: metadata.xml is essential. Abort! Please try again later! Can you help me with this situation?

[C0D3D3V]

Your playback url seems to be broken

[C0D3D3V]

At least for me the URL returns 404

[C0D3D3V]

Please open a new issue if this error persists with a correct URL

[dmrbatuhan]

Hello, as I mentioned before, the link is not broken. It is giving a metadata error. However, today I discovered something interesting. When I try to download an old recorded lesson using the link, it works without any problems, but it doesn't work with the new links. You will find the links below. These links are obtained from the live lesson recordings section in my university's Learning Management System. Working link (March 6th, 2023) https://scalelite.sdu2bbb.online/playback/presentation/2.3/7cb787fa0c008c41753dbab180f89208cf2eac80-1678081914350 Link that gives a metadata error (April 3rd, 2023) https://scalelite.sdu2bbb.online/playback/presentation/2.3/7cb787fa0c008c41753dbab180f89208cf2eac80-1680501512019 Thanks in advance.

[C0D3D3V]

Are you kidding me? Make a screenshot of the second link to show me it does not give an error for you.

[C0D3D3V]

My guess is, that second session was not recorded, thats why the link is broken.

[dmrbatuhan]

If I couldn't watch the link that gives a metadata error on the bbb-dl through the university website, I would have thought the same as you. I am trying to take a screenshot, but as I am visually impaired, I may not be able to provide a very clear screenshot.

[C0D3D3V]

Do you have to login on that website? You can also send the screenahot via mail

[dmrbatuhan]

Yes, I log in with my university email address and password, and then I click on this link. I hope I got it correctly. I'm blind.

[C0D3D3V]

I guess you can only view the URL when you have loged in. Can you send me the website where you log in? Is it moodle or directly on Big Blue Button? If it is moodle maybe it is some sort of LTI module for authentification. In any case... I can add an option that allows you to load cookies. So you only have to save your browser cookies to file and bbb-dl will load them.

[dmrbatuhan]

Please accept my apologies for the delayed response. As you may recall, I had previously sent a screenshot of the relevant window. To access the university's learning management system, users can use the login link provided at https://oys.sdu.edu.tr, where they will be prompted to enter their university email address and password. Upon successful login, the user will be directed to a control panel screen displaying a list of courses. By selecting any of the listed courses, users can access a link that redirects them to the Big Blue Button platform located under the "Live Classes and Recordings" section. Clicking on the link will launch a window that is similar to the one shown in the screenshot I provided earlier. The class can then be initiated by pressing the "Play Video" button.

[C0D3D3V]

The link you provided is a moodle instance. As I guessed correctly ^^ Very interesting, maybe I could even integrate a mechanic to send the cookies from moodle-dl to bbb-dl, or since moodle-dl will use yt-dlp for LTI Modules I would need to send the cookies from yt-dlp to bbb-dl... But that is something I will maybe do someday in future. For now I will keep it very simple and just allow you to login via a simple cookie file or a simple browser window... I do not think I will have time over eastern for this, but next week probably...

[C0D3D3V]

Please update and read the instructions: https://github.com/C0D3D3V/bbb-dl/releases/tag/v1.0.3

Please report if it works!

[C0D3D3V]

Have you tried it?

[dmrbatuhan]

I had a busy week, so I could only try now. Unfortunately, the result is still the same. First, I logged in to the browser window that appeared using the bbb-dl-browser command on Windows, but the result was negative. Then, I used a Google Chrome extension to save the cookies to a file named cookies.txt and copied it to the specified default folder. I even tried the cookies.txt file on Linux, but I still got the same error: "Downloading meta information Downloading metadata.xml... metadata.xml could not be downloaded: 404 Not Found Error: metadata.xml is essential. Abort! Please try again later!"

[C0D3D3V]

Have you opened the BBB session in the bbb-dl-browser? So that you could watch the recording in the browser? And after that closed the browser and tried to download the URL that was shown in bbb-dl-browser?

[C0D3D3V]

Mh I can add a button to the browser to directly start the download of a detected session. If that does not help, we can have a discord call and maybe then I see what is going wrong...

[Patrick92bs]

I have the same problem. How does it work exectly with the download of the BBB lesson is behind login? I need to download some lessons from my university 😅

[C0D3D3V]

I have currently no way to test it. Since I have no such login page.

But the idea is as follow: run bbb-dl-bowser. Login and open your BBB-Session for that you need login. Copy the URL that is displayed inside bbb-dl-browser. Close bbb-dl-browser. Run bbb-dl with the copied URL.

[yilmazali32]

bbb-dl --skip-cursor https://scalelite.sdu2bbb.online/playback/presentation/2.3/7cb787fa0c008c41753dbab180f89208cf2eac80-1680501512019

write this command and give me output

[C0D3D3V]

If one of you would give me your credentials I could test the download using login and fix the bug. You can contact me via email (on my profile) or discord (c0d3d3v)

[varuneranki]

I would like to add some more details as i face similar issue. it might not be authentication issue. It connects to the server and then does nothing. tired changing the working and output directories. that's not the issue. navigated directly to work dir and used wget to download like a normal file. it didn't download after getting connected to server.

I have stumbled upon this repo just because soon the server will be made offline and we are asked to take our data with us. I was able to download manually certain files like videos and presentation slide images using wget. what i wanted was the additional data and automated tool to combine like yours. Something is wrong with the chromium instance in venv. without venv i couldn't even get to this stage. it gives traceback errors saying syntax is wrong.

[C0D3D3V]

@varuneranki this is a diffrent issue. I can test it tomorrow. But please open a new issue. If it would be a problem with authentification you could use a cookies.txt as described above.

[varuneranki]

@varuneranki this is a diffrent issue. I can test it tomorrow. But please open a new issue. If it would be a problem with authentification you could use a cookies.txt as described above.

I have solved the problem. It is related to the above issue. I'm not opening a new issue. All it needed was a VPN connected to our Uni network and was able to download easily. I remember wget worked with RDP uni server but not with my local machine. Turns out it requires NAT hole punching support for application or a VPN tunnel for accessing certain servers hosed on secure networks. All across Europe universities use Eduroam network and it has additional layers of security.

[C0D3D3V]

@varuneranki It is not related to this issue. But its nice to hear that you got it working. The problem is that the server is configured wrongly, and does not allow connections with a TLSv1.0 Client Hello message. But thats also a problem of aiohttp, because you should be able to connect to such misconfigured servers. With the python requests library it works without problems. See https://github.com/aio-libs/aiohttp/issues/8478

[C0D3D3V]

@varuneranki If you want to download a lot of sessions, have a look at "Batch processing" and "How can I speed up the rendering process?" in the Readme

[varuneranki]

@varuneranki If you want to download a lot of sessions, have a look at "Batch processing" and "How can I speed up the rendering process?" in the Readme

I only have 2 sessions to download so it wouldn't bother me. It was good to know as networking is quite a complex topic to understand. I wouldn't have even used this github repo if our admin has configured the server properly and if BBB download option worked. Since server should have been shutdown yesterday, it was quite a task to figure out in a complex way. Thank you once again. :+1:

[C0D3D3V]

@varuneranki I fixed your problem in the new version. Now it also works without VPN.