kylon / Sharedown

Electron application to download Sharepoint videos (especially meant for students)
GNU General Public License v3.0
376 stars 41 forks source link

This software can't keep credentials stored #51

Closed Elvino00 closed 2 years ago

Elvino00 commented 2 years ago

Basically, even if I select "manual login" as login module and then enable the Chrome user data folder (as you taught me from some previous issue) the software doesn't keep credentials memorized (My university forces 2FA now). Also, after putting credentials, when it's time to play the video, there is always "playback issue" on the page and I have to re-execute at least 3 times (putting in every time the credentials) the software in order to obtain a successful download. Here are the logs. By the way, I am on Linux x64 bit (Fedora 36) at the moment.

sharedownLog.log ytdlp.log

EDIT: I Started another download and these are the errors I always get:

Screenshot from 2022-10-19 16-57-53 Screenshot from 2022-10-19 16-58-00

kylon commented 2 years ago

Logs are clean, something else happening way before due to your uni login changes

You must select yes on first login when it asks to keep credentials or to remember the session otherwise it will not survive the next download

About the error, should be just a missing null check but again i cant reproduce with my uni, just finished 40 tests and no error so i expect to be something specific

I ll post a test version

Elvino00 commented 2 years ago

Logs are clean, something else happening way before due to your uni login changes

You must select yes on first login when it asks to keep credentials or to remember the session otherwise it will not survive the next download

About the error, should be just a missing null check but again i cant reproduce with my uni, just finished 40 tests and no error so i expect to be something specific

I ll post a test version

I always select "yes" when it asks to keep credentials and tick the box "don't show this message again"

kylon commented 2 years ago

It is possible it is now clearing data when you close the browser, in this case you need to use the login module + remember login credentials

It uses the system keyring, tested with gnome keyring but it should support other most common ones

kylon commented 2 years ago

Ah 2fa, prevous reply is wrong

Mh if it is clearing data on close i think a way would be to keep the browser open unless there is a puppeteer option for this I dont see any other way atm for cases like this

Elvino00 commented 2 years ago

Ah 2fa, prevous reply is wrong

Mh if it is clearing data on close i think a way would be to keep the browser open unless there is a puppeteer option for this I dont see any other way atm for cases like this

How do i keep the test browser open if it automatically closes when download starts?

kylon commented 2 years ago

you can't, i need to add a setting for this, i ll add it to the test version

Elvino00 commented 2 years ago

you can't, i need to add a setting for this, i ll add it to the test version

Oh, ok then. I'm ready to test when you release it.

kylon commented 2 years ago

Sharedownt1.zip

Elvino00 commented 2 years ago

Sharedownt1.zip

I've tested the version. I've enabled the option to keep the browser windows opened and at the first download that i did, it asked me credentials (ok, that's fine) and I've downloaded a video (i want to mention that I've clicked on "yes" when it asked me to remember credentials). I've reopened the browser and started two new downloads and it still asks me credentials.

These are the logs:

sharedownLog.log ytdlp.log

This is a screenshot of the settings I've enabled: image

kylon commented 2 years ago

you say reopened the browser, does it mean you close its window while downloading? the option is meant to keep the window open so session is not cleared

Elvino00 commented 2 years ago

you say reopened the browser, does it mean you close its window while downloading? the option is meant to keep the window open so session is not cleared

No, not while downloading. I closed the browser when the downloads were completed obviously, otherwise the option you've made would've been pointless.

kylon commented 2 years ago

ok, if you close the browser window, since your login session does not survive, when a new browser is launched, it will ask for credentials again so the workaround is to keep the browser open while you are using sharedown, don't close it on first batch completed, close it when you exit sharedown (before or after)

data folder is probably not needed in this case, you can just use login module, since session is lost on close

this is not an ideal solution but i have no way to test and no 2fa settings if the same happens with a standard browser, not launched by sharedown, ie you login, 2fa etc.. and then close the browser and once reopened you must redo the login process, it is the way set by your uni, so probably this is the only? workaround, otherwise i would need help from someone that is willing to work on this and send pr

Elvino00 commented 2 years ago

ok, if you close the browser window, since your login session does not survive, when a new browser is launched, it will ask for credentials again so the workaround is to keep the browser open while you are using sharedown, don't close it on first batch completed, close it when you exit sharedown (before or after)

data folder is probably not needed in this case, you can just use login module, since session is lost on close

this is not an ideal solution but i have no way to test and no 2fa settings if the same happens with a standard browser, not launched by sharedown, ie you login, 2fa etc.. and then close the browser and once reopened you must redo the login process, it is the way set by your uni, so probably this is the only? workaround, otherwise i would need help from someone that is willing to work on this and send pr

First, I have changed the executable path because the chromium based internal browser gives me an error about "send input" or something like that (I'll send the logs). I am using google-chrome-stable now, using Unina login module and I've disabled the setting about chrome data folder and selected the "remeber login credentials option". I'll keep you posted on what happens.

Elvino00 commented 2 years ago

ok, if you close the browser window, since your login session does not survive, when a new browser is launched, it will ask for credentials again so the workaround is to keep the browser open while you are using sharedown, don't close it on first batch completed, close it when you exit sharedown (before or after)

data folder is probably not needed in this case, you can just use login module, since session is lost on close

this is not an ideal solution but i have no way to test and no 2fa settings if the same happens with a standard browser, not launched by sharedown, ie you login, 2fa etc.. and then close the browser and once reopened you must redo the login process, it is the way set by your uni, so probably this is the only? workaround, otherwise i would need help from someone that is willing to work on this and send pr

Ok, after my tests, I think there's nothing that can be done. My university 2fa is a pain in the ass. However, I'm not able to do simultaneously more downloads and every time I have to close sharedown and reopen in order to start over with the next download (after one download is complete, I keep the browser always opened). I am using chrome and here are the logs:

sharedownLog.log ytdlp.log

kylon commented 2 years ago

it gets stuck after one download for every download? and it still asks for 2fa even if the browser is open?

i ll add lot more log prints and see if it can catch something useful

Elvino00 commented 2 years ago

it gets stuck after one download for every download? and it still asks for 2fa even if the browser is open?

i ll add lot more log prints and see if it can catch something useful

I start one download, the browser opens and asks for 2fa, the video is played and the download starts. Then, when this download is completed and it's time for the second one, I get an error I can't remember but all downloads stop and I have to close the browser, and close sharedown too cause if I try to start manually the second download I get an error that it can't find a valid link.

Elvino00 commented 2 years ago

it gets stuck after one download for every download? and it still asks for 2fa even if the browser is open?

i ll add lot more log prints and see if it can catch something useful

i guess you didn't catch anything, right?

kylon commented 2 years ago

my uni setup is different, i ll post a version with more logs but had no time, probably this week

Elvino00 commented 2 years ago

my uni setup is different, i ll post a version with more logs but had no time, probably this week

oh sorry, I'll wait then.

kylon commented 2 years ago

Sharedownt2.zip

Elvino00 commented 2 years ago

Sharedownt2.zip

I've put three items in download, so I put my credentials and sms (click "yes" to the pop up asking if i wanna stay signed in) and the first video is played. The download starts smoothly. When the first one is done, another tab is opened where the second video is opened but suddendly i get these two errors:

first: https://imgur.com/WHKo0Ly

second (after clicking on "ok" ) : https://imgur.com/zRT0ssZ

These are the logs: sharedownLog.log ytdlp.log

Basically I'm unable to download automatically more videos (and I keep the browser opened because of 2FA. Obviously I ticked the option in the sharedown)

EDIT: I closed the software and I had to do everything from the beginning. When the second download was completed, it gave me the exact errors I sent you the screenshot above

kylon commented 2 years ago

use manual login

Elvino00 commented 2 years ago

use manual login

Just tried using the manual login. It still gives me the same errors in the pictures. I can't do more than one download at the same time

sharedownLog.log ytdlp.log

kylon commented 2 years ago

Sharedownt4.zip

kylon commented 2 years ago

Sharedownt6.zip

try this with bundled chromium, not custom, it should now work and hopefully solves this issue

Elvino00 commented 2 years ago

Sharedownt4.zip

I tried this and it doesn't work but I've just seen this "Sharedown6.zip" file so I'll check that instead

kylon commented 2 years ago

you can delete sett file, in File -> open logs folder, go to up folder and delete the file

Elvino00 commented 2 years ago

you can delete sett file, in File -> open logs folder, go to up folder and delete the file

The software says it can't find the chromium executable and I've set the program to use the built-in one

sharedownLog.log ytdlp.log

image

kylon commented 2 years ago

is the folder node_modules/puppeteer/chrome present? if not, the modules install failed or you are using an old node_modules folder please delete your node_modules folder and .lock file and run yarn install or npm i to install modules

kylon commented 2 years ago

checked the zip, a file was missing Sharedownt61.zip

Elvino00 commented 2 years ago

checked the zip, a file was missing Sharedownt61.zip

ok, I'm retrying with this. Also, just to say, i compile with npm.

EDIT: Just tried and it gives me the following errors:

first error and second error

sharedownLog.log ytdlp.log

kylon commented 2 years ago

are you setting a ms id in sharedown settings? is your login page changed recently?

try with no auto login, leave ms id empty in settings

kylon commented 2 years ago

oh wait, that is the 2nd download?

EDIT:

if keep browser open is enabled, please disable it and retry

Elvino00 commented 2 years ago

oh wait, that is the 2nd download?

EDIT:

if keep browser open is enabled, please disable it and retry

yes, it was the second. Should I also try with no auto login and ms id empty in settings?

kylon commented 2 years ago

try without the new experimental setting and enable auto login

Elvino00 commented 2 years ago

try without the new experimental setting and enable auto login

What happens now is that when the second download starts I get these two errors: first second

sharedownLog.log ytdlp.log

kylon commented 2 years ago

Sharedownt7.zip

Elvino00 commented 2 years ago

Sharedownt7.zip

I tried to start the first download and i get this error.

sharedownLog.log ytdlp.log

kylon commented 2 years ago

Sharedownt8.zip

Elvino00 commented 2 years ago

Sharedownt8.zip

I tried this build and i was able to do three consecutive downloads. Here's what I've done: I used UNINA authentication module so the only thing to input was the 2FA code. I've left the option to keep the browser window opened (the experimental setting you were referring to ?) off, as you suggested.

EDIT: Added Logs

sharedownLog.log ytdlp.log

kylon commented 2 years ago

ok, so it should be fixed now

about the original issue, credentials, can you try again to enable chrome data folder and disable auto login to keep credentials?

if again it forgets them, i ll post a new test with fixed keep browser open option

Elvino00 commented 2 years ago

ok, so it should be fixed now

about the original issue, credentials, can you try again to enable chrome data folder and disable auto login to keep credentials?

if again it forgets them, i ll post a new test with fixed keep browser open option

I didn't understand if I have to keep enabled this option

kylon commented 2 years ago

no, that one is disabled

Elvino00 commented 2 years ago

ok, so it should be fixed now

about the original issue, credentials, can you try again to enable chrome data folder and disable auto login to keep credentials?

if again it forgets them, i ll post a new test with fixed keep browser open option

I tried to do as you told but I got this error

The following settings are enabled:

Image 1 Image 2 Image 3

These are the logs:

sharedownLog.log ytdlp.log

kylon commented 2 years ago

uh, though that was fixed, so it only works if you enable auto login and no chrome folder?

kylon commented 2 years ago

Sharedownt9.zip

keep browser open should be working here

disable auto login disable chrome data folder enable keep browser open

on first download it will ask for credentials, and the browser window will never close

Elvino00 commented 2 years ago

after downloading the first one I got this error but I am connnected to the network. I refreshed the page and then I got puppeteer error. I only have enabled "keep the browser windows opened". No chrome folder or remember login credential and login module is manual

sharedownLog.log ytdlp.log

kylon commented 2 years ago

Sharedownt10.zip

Elvino00 commented 2 years ago

I've authenticated and I've been able to do 3 consecutive downloads. Basic authentication, haven't tested Unina yet. All of this with the option to keep the browser window opened and i had to authenticate only once, for the rest, seems that the downloads went all smoothly without the need to intervene.

If you want to check the logs, here they are:

sharedownLog.log ytdlp.log

kylon commented 2 years ago

nothing really changed, just added logs previous log was fine too, it got the data but for unknown reasons it failed to grab the video title

basically, it is working fine, but it fails randomly for unknown reasons

i ll add the new option as a workaround for cases like yours and close this

for the errors, enable "retry on fail" so that it will try again when it randomly fails