Q: Will destreamer work on the new Microsoft SharePoint Stream?

[WG-]

So Microsoft is phasing out Microsoft Stream and is moving it towards SharePoint.

Will destreamer be modified such that it supports SharePoint Stream?

[cgonIT]

+1

[hakt0-r]

The short answer is yes! I've just managed to get this working, and that too on my M1 mac running macOS Ventura 13.4.1. I'll try to lay out what I've done to fix and hopefully can create a pull request and fix this too for all.

These are macOS specific instructions, though you can ignore macOS bits and make it work for other OS's.

1. brew install chromium

2. To avoid running into "chromium.app is damaged error" (see https://github.com/Homebrew/homebrew-cask/issues/112986#issue-1031594353) Run these two commands. (it'll set the right flags for macOS SIP/watchdog/whatever)

   xattr -cr /opt/homebrew/bin/chromium xattr -cr /Applications/Chromium.app

3. git clone --branch sharepoint https://github.com/snobu/destreamer.git

4. cd into that directory

5. edit the package.json file to change change "@types/puppeteer" : "5.5.0", to "puppeteer" : "^19.11.1",

change "puppeteer" : "5.5.0", to "puppeteer" : "^19.11.1",

6. edit the tsconfig.json file to add a line after "sourceMap"... "noImplicitAny": false,

7. now run npm install

8. (going for gold!) We'll also update all the node packages and their versions. Run npm update --save

9. at this stage you can now try and run npm run build You'll get a few errors! First kind... few of these... src/LoginModules.ts:14:20 - error TS2503: Cannot find namespace 'puppeteer'. and few of these... src/TokenCache.ts:60:20 - error TS2503: Cannot find namespace 'puppeteer'. to Fix this, just edit these two files and change this line => import puppeteer from 'puppeteer'; to this => import * as puppeteer from 'puppeteer';

10. running npm run build again, gives you one more error... src/TokenCache.ts:99:24 - error TS2339: Property 'waitFor' does not exist on type 'Page'. edit src/TokenCache.ts file line 99 and change waitFor to waitForTimeout (see: https://github.com/puppeteer/puppeteer/issues/9762)

11. Finally you are ready to build. npm run build will work.

I've successfully tested today downloading two recordings. 😄

[nfontan]

The short answer is yes! I've just managed to get this working, and that too on my M1 mac running macOS Ventura 13.4.1. I'll try to lay out what I've done to fix and hopefully can create a pull request and fix this too for all.

These are macOS specific instructions, though you can ignore macOS bits and make it work for other OS's.

1. brew install chromium
2. To avoid running into "chromium.app is damaged error" (see Can't install Chromium without --no-quarantine Homebrew/homebrew-cask#112986 (comment)) Run these two commands. (it'll set the right flags for macOS SIP/watchdog/whatever)

xattr -cr /opt/homebrew/bin/chromium xattr -cr /Applications/Chromium.app

3. git clone --branch sharepoint https://github.com/snobu/destreamer.git
4. cd into that directory
5. edit the package.json file to change change "@types/puppeteer" : "5.5.0", to "puppeteer" : "^19.11.1",

change "puppeteer" : "5.5.0", to "puppeteer" : "^19.11.1",

6. edit the tsconfig.json file to add a line after "sourceMap"... "noImplicitAny": false,
7. now run npm install
8. (going for gold!) We'll also update all the node packages and their versions. Run npm update --save
9. at this stage you can now try and run npm run build You'll get a few errors! First kind... few of these... src/LoginModules.ts:14:20 - error TS2503: Cannot find namespace 'puppeteer'. and few of these... src/TokenCache.ts:60:20 - error TS2503: Cannot find namespace 'puppeteer'. to Fix this, just edit these two files and change this line => import puppeteer from 'puppeteer'; to this => import * as puppeteer from 'puppeteer';
10. running npm run build again, gives you one more error... src/TokenCache.ts:99:24 - error TS2339: Property 'waitFor' does not exist on type 'Page'. edit src/TokenCache.ts file line 99 and change waitFor to waitForTimeout (see: [Bug]: TypeError: page.waitFor is not a function when using pupeteer puppeteer/puppeteer#9762)
11. Finally you are ready to build. npm run build will work.

I've successfully tested today downloading two recordings. 😄

Here not work!

Here the results: destreamer.cmd -i "https://xxxxxoutlook-my.sharepoint.com/:v:/g/personal/xxxx_xxxxx_com/EU8PrthK4gdGuXXxi1CY8AEBKhUcSjnfHmD-SayKpvBv2w?nav=eyJyZWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" Parsing video/group urls Downloading SharePoint videos...

Launching headless Chrome to perform the OpenID Connect dance... Navigating to login page... Waiting for target! We are logged in.

[FATAL ERROR] Could not evaluate sessionInfo on the page

[cr2007]

Hey @hakt0-r

I tried these steps you had mentioned on Windows (running via WSL) Even fixed all the lines of code before running npm run build

But I get this new error when I try to install from MS Stream:

$ ./destreamer.sh -i <Video-URL> -o <preferred-directory>
/bin/sh: 1: aria2c: not found

[FATAL ERROR] Aria2 is missing!
Destreamer requires a fairly recent release of Aria2 to download videos

If you encountered this, could you share the steps on how to fix this Or refer me to a similar issue where it has been resolved?

[nfontan]

Hey @hakt0-r

I tried these steps you had mentioned on Windows (running via WSL) Even fixed all the lines of code before running npm run build

But I get this new error when I try to install from MS Stream:

$ ./destreamer.sh -i <Video-URL> -o <preferred-directory>
/bin/sh: 1: aria2c: not found

[FATAL ERROR] Aria2 is missing!
Destreamer requires a fairly recent release of Aria2 to download videos

If you encountered this, could you share the steps on how to fix this Or refer me to a similar issue where it has been resolved?

You need to install aria2 apt install -y aria2

[cr2007]

I fixed that, and now a fatal error is appearing

$ ./destreamer.sh -i <VIDEO-URL> -o <DEST-DIRECTORY>
Parsing video/group urls
Downloading Microsoft Stream videos...

[WARNING] .token_cache not found.

Launching headless Chrome to perform the OpenID Connect dance...

[ERROR] Error: Failed to launch the browser process!
/home/<user>/.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome: error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory

TROUBLESHOOTING: https://pptr.dev/troubleshooting

    at Interface.onClose (/home/<user>/destreamer/node_modules/@puppeteer/browsers/lib/cjs/launch.js:262:24)
    at Interface.emit (node:events:525:35)
    at Interface.close (node:internal/readline/interface:536:10)
    at Socket.onend (node:internal/readline/interface:262:10)
    at Socket.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

[FATAL ERROR] Unhandled error!
Timeout or fatal error, please check your downloads directory and try again

[hakt0-r]

The key is…error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory Since you are running it via WSL, you’ll have to debug a bit more but here is what I found…Run this…. sudo apt install ca-certificates fonts-liberation libappindicator3-1 libasound2 libatk-bridge2.0-0 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 libglib2.0-0 libgtk-3-0 libnspr4 libnss3 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 lsb-release wget xdg-utilsHth.On 24 Sep 2023, at 01:07, CSK @.***> wrote: I fixed that, and now a fatal error is appearing $ ./destreamer.sh -i -o Parsing video/group urls Downloading Microsoft Stream videos...

[WARNING] .token_cache not found.

Launching headless Chrome to perform the OpenID Connect dance...

[ERROR] Error: Failed to launch the browser process! /home//.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome: error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory

TROUBLESHOOTING: https://pptr.dev/troubleshooting

at Interface.onClose ***@***.***/browsers/lib/cjs/launch.js:262:24)
at Interface.emit (node:events:525:35)
at Interface.close (node:internal/readline/interface:536:10)
at Socket.onend (node:internal/readline/interface:262:10)
at Socket.emit (node:events:525:35)
at endReadableNT (node:internal/streams/readable:1359:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

[FATAL ERROR] Unhandled error! Timeout or fatal error, please check your downloads directory and try again

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[nfontan]

I fixed that, and now a fatal error is appearing

$ ./destreamer.sh -i <VIDEO-URL> -o <DEST-DIRECTORY>
Parsing video/group urls
Downloading Microsoft Stream videos...

[WARNING] .token_cache not found.

Launching headless Chrome to perform the OpenID Connect dance...

[ERROR] Error: Failed to launch the browser process!
/home/<user>/.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome: error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory

TROUBLESHOOTING: https://pptr.dev/troubleshooting

    at Interface.onClose (/home/<user>/destreamer/node_modules/@puppeteer/browsers/lib/cjs/launch.js:262:24)
    at Interface.emit (node:events:525:35)
    at Interface.close (node:internal/readline/interface:536:10)
    at Socket.onend (node:internal/readline/interface:262:10)
    at Socket.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

[FATAL ERROR] Unhandled error!
Timeout or fatal error, please check your downloads directory and try again

In my case I tried many fixes with wsl, installing other dependencies to make it work, anyway I couldn't get it to work with wsl. Try again directly on windows with node for windows. but it didn't work either (error that I left above)

Finish downloading with ffmpeg -i "videomanifest" -codec copy downloadedVideo.mp4

[hakt0-r]

Here not work!

Here the results: destreamer.cmd -i "https://xxxxxoutlook-my.sharepoint.com/✌️/g/personal/xxxx_xxxxx_com/EU8PrthK4gdGuXXxi1CY8AEBKhUcSjnfHmD-SayKpvBv2w?nav=eyJyZWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" Parsing video/group urls Downloading SharePoint videos...

Launching headless Chrome to perform the OpenID Connect dance... Navigating to login page... Waiting for target! We are logged in.

[FATAL ERROR] Could not evaluate sessionInfo on the page

First of all, which branch did you clone? SharePoint URLs work best with this branch https://github.com/hakt0-r/destreamer/tree/sharepoint

Clone that branch and follow all the instructions again please.

If you still encounter problems please paste the verbose output so that we can debug.

[nfontan]

Here not work! Here the results: destreamer.cmd -i "https://xxxxxoutlook-my.sharepoint.com/✌️/g/personal/xxxx_xxxxx_com/EU8PrthK4gdGuXXxi1CY8AEBKhUcSjnfHmD-SayKpvBv2w?nav=eyJyZWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" Parsing video/group urls Downloading SharePoint videos... Launching headless Chrome to perform the OpenID Connect dance... Navigating to login page... Waiting for target! We are logged in. [FATAL ERROR] Could not evaluate sessionInfo on the page

First of all, which branch did you clone? SharePoint URLs work best with this branch https://github.com/hakt0-r/destreamer/tree/sharepoint

Clone that branch and follow all the instructions again please.

If you still encounter problems please paste the verbose output so that we can debug.

Yes, I'm testing with that branch. I have also tried with the 3.0 branch but the error in that case is invalid url, but the login behavior seems more accurate

[hakt0-r]

Yes, I'm testing with that branch. I have also tried with the 3.0 branch but the error in that case is invalid url, but the login behavior seems more accurate !

🤨 all I can say is that

1. try back-to-back attempts.
2. See if it’s an issue with .cache or .token_cache
3. Look at #380 please.

[recolic]

The short answer is yes! I've just managed to get this working, and that too on my M1 mac running macOS Ventura 13.4.1. I'll try to lay out what I've done to fix and hopefully can create a pull request and fix this too for all.

dp: It worked for me on Arch Linux. I forked this repo and applied these fixes, so you don't need to do everything again. Just update executablePath and enjoy.
https://github.com/recolic/destreamer-sharepoint

[cvee112]

The short answer is yes! I've just managed to get this working, and that too on my M1 mac running macOS Ventura 13.4.1. I'll try to lay out what I've done to fix and hopefully can create a pull request and fix this too for all.

Thanks a lot! This worked for me and I was even able to download one 18-min video because of it! But then when I tried running it again, Chromium now crashes a few seconds after launch. It has been like that ever since (even after restarting), and I don't know why. I'm on an M2 Mac with macOS Sonoma and I also have Arc, but when my first run succeeded, I had Arc open so I'm not sure what else could be causing the issue. Any guess on why it's happening and what I can do about it?

It shows either of the following errors:

Launching headless Chrome to perform the OpenID Connect dance...
Navigating to login page...

[ERROR] Error: Navigation failed because browser has disconnected!
    at new LifecycleWatcher (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/LifecycleWatcher.js:83:223)
    at Frame.goto (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Frame.js:183:25)
    at CDPPage.goto (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:438:91)
    at doShareLogin (/Users/cvee/Stream/destreamer/build/src/LoginModules.js:121:20)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async downloadShareVideo (/Users/cvee/Stream/destreamer/build/src/Downloaders.js:146:21)
    at async main (/Users/cvee/Stream/destreamer/build/src/destreamer.js:48:9)

[FATAL ERROR] Unhandled error!
Timeout or fatal error, please check your downloads directory and try again

or

Launching headless Chrome to perform the OpenID Connect dance...
Navigating to login page...

[ERROR] Error: Requesting main frame too early!
    at assert (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/util/assert.js:28:15)
    at FrameManager.mainFrame (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:173:32)
    at CDPPage.goto (/Users/cvee/Stream/destreamer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:438:79)
    at doShareLogin (/Users/cvee/Stream/destreamer/build/src/LoginModules.js:121:20)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async downloadShareVideo (/Users/cvee/Stream/destreamer/build/src/Downloaders.js:146:21)
    at async main (/Users/cvee/Stream/destreamer/build/src/destreamer.js:48:9)

[FATAL ERROR] Unhandled error!
Timeout or fatal error, please check your downloads directory and try again

When I launch Chromium by typing chromium on my console, it's the same - it opens then exits very soon after. It does show the following message but I'm not sure if it's related:

[3307:259:0109/204329.397809:ERROR:policy_logger.cc(157)] :components/enterprise/browser/controller/chrome_browser_cloud_management_controller.cc(161) Cloud management controller initialization aborted as CBCM is not enabled. Please use the `--enable-chrome-browser-cloud-management` command line flag to enable it if you are not using the official Google Chrome build.

UPDATE: I got it to work by switching the browser to Brave, but Chromium is still facing the same issue.

[pedro-stanaka]

Worked for me on Chromium (with Macbook Air M2). Steps:

1. Had to install with brew install --cask chromium --no-quarantine.
2. Changed some stuff to make it work with my URLs: https://github.com/pedro-stanaka/destreamer/tree/sharepoint
3. Profit!

[philCryoport]

Two notes:

1. @hakt0-r thank you for the above step-by-step process.

FYI on my M2-Max running Sonoma (MacOS 14), I didn't need to do the xattr commands -- but everything else was taken care of with your suggested fixes.

2. When I copy/paste the Microsoft Stream URL onto my Oh-My-Zsh command line, it's automatically escaping with backslashes the ?, =, and &.

Is anybody else having this issue? I'm half tempted to insert code to look for those escaped characters and remove the escaping...

[pedro-stanaka]

Two notes:

1. @hakt0-r thank you for the above step-by-step process.

FYI on my M2-Max running Sonoma (MacOS 14), I didn't need to do the xattr commands -- but everything else was taken care of with your suggested fixes.

2. When I copy/paste the Microsoft Stream URL onto my Oh-My-Zsh command line, it's automatically escaping with backslashes the ?, =, and &.

Is anybody else having this issue? I'm half tempted to insert code to look for those escaped characters and remove the escaping...

Just paste the url in a text editor with ' (single quotes) wrapping the url. It should work then.

[snowcross]

I had this working for a month or so, now I keep getting an "[ERROR] Invalid url" I think the URLs changed from sharpoint recently. as in the lat week.

[philCryoport]

@pedro-stanaka wrote:

Just paste the url in a text editor with ' (single quotes) wrapping the url. It should work then.

Yep, I figured that out after posted...