lossless1024 / StreaMonitor

Adult live stream downloader for advanced people. I could have chosen a better name.
GNU General Public License v3.0
172 stars 42 forks source link

I want to change the saved address. #140

Open myh66 opened 3 months ago

myh66 commented 3 months ago

I want to change the save location on Windows, or move all recorded videos to a folder, can I do that without setting up folders for each ID? What files do I need to configure to modify them?

DerBunteBall commented 3 months ago

I'm not quite sure if I understand the question correctly.

The download directory can be changed via the parameters.py.

The code uses os.path so when setting the path you have to consider that you either have to escape or pass a raw string.

For example, to set C:\Downloads\StreaMonitor as the folder, the following variants are possible:

DOWNLOADS_DIR = 'C:\\Downloads\\StreaMonitor'

or as Raw-String:

DOWNLOADS_DIR = r'C:\Downloads\StreaMonitor'

It is possible that problems may arise from the fact that os.path.join is used. I don't have Windows to test at the moment.

myh66 commented 3 months ago

Ok, I'll test it tomorrow, you've understood half of it, I'd like to change the save directory, hopefully so it won't be a case of each author having a separate folder.

DerBunteBall commented 3 months ago

To prevent this, the code would have to be changed.

Bot.py from line 265:

@property
def outputFolder(self):
    return os.path.join(DOWNLOADS_DIR, self.username + ' [' + self.siteslug + ']')

def genOutFilename(self, create_dir=True):
    folder = self.outputFolder
    if create_dir:
        os.makedirs(folder, exist_ok=True)
    now = datetime.now()
    filename = os.path.join(folder, self.username + '-' + str(now.strftime("%Y%m%d-%H%M%S")) + '.mp4')
    return filename  

As you can see, the path is built statically. This should be prevented. To do this, you would simply have to adjust outputFolder and throw out the creation of the folder in genOutFilename.

myh66 commented 3 months ago

To prevent this, the code would have to be changed.

Bot.py from line 265:

@property
def outputFolder(self):
    return os.path.join(DOWNLOADS_DIR, self.username + ' [' + self.siteslug + ']')

def genOutFilename(self, create_dir=True):
    folder = self.outputFolder
    if create_dir:
        os.makedirs(folder, exist_ok=True)
        now = datetime.now()
        filename = os.path.join(folder, self.username + '-' + str(now.strftime("%Y%m%d-%H%M%S")) + '.mp4')
        return filename  

As you can see, the path is built statically. This should be prevented. To do this, you would simply have to adjust outputFolder and throw out the creation of the folder in genOutFilename.

thanks

DerBunteBall commented 3 months ago

Could look like this:

@property
def outputFolder(self):
    return DOWNLOADS_DIR

def genOutFilename(self, create_dir=False):
    folder = self.outputFolder
    if create_dir:
        os.makedirs(folder, exist_ok=True)
    now = datetime.now()
    filename = os.path.join(folder, self.username + '-' + str(now.strftime("%Y%m%d-%H%M%S")) + '.mp4')
    return filename  

Make sure the directory you specify in parameters.py exists. All files should go as files into that directory with this snippet (not tested).

myh66 commented 3 months ago

I'm not quite sure if I understand the question correctly.

The download directory can be changed via the parameters.py.

The code uses os.path so when setting the path you have to consider that you either have to escape or pass a raw string.

For example, to set C:\Downloads\StreaMonitor as the folder, the following variants are possible:

DOWNLOADS_DIR = 'C:\\Downloads\\StreaMonitor'

or as Raw-String:

DOWNLOADS_DIR = r'C:\Downloads\StreaMonitor'

It is possible that problems may arise from the fact that os.path.join is used. I don't have Windows to test at the moment.

I just changed the download path using "DOWNLOADS_DIR = r'C:\Downloads\StreaMonitor" and now I'm trying to change the save method to just the file. image

myh66 commented 3 months ago

Could look like this:可以是这样的

@property
def outputFolder(self):
    return DOWNLOADS_DIR

def genOutFilename(self, create_dir=False):
    folder = self.outputFolder
    if create_dir:
        os.makedirs(folder, exist_ok=True)
    now = datetime.now()
    filename = os.path.join(folder, self.username + '-' + str(now.strftime("%Y%m%d-%H%M%S")) + '.mp4')
    return filename  

Make sure the directory you specify in parameters.py exists. All files should go as files into that directory with this snippet (not tested).确保在 parameters.py 中指定的目录存在。使用此代码段,所有文件都应以文件形式进入该目录(未经测试)。

This one's right, too.One other change I'd like to make, I'd like to split the videos and limit each video to 2G.

DerBunteBall commented 3 months ago

You could try -fs 2G.

However, you will have to change the code again.

This should stop ffmpeg when 2GB is reached.

This should result in a short loss but the bot should continue to work.

The ffmpeg.py would have to be changed like this:

def getVideoFfmpeg(self, url, filename):
    cmd = [
        'ffmpeg',
        '-user_agent', self.headers['User-Agent']
    ]

    if type(self.cookies) is requests.cookies.RequestsCookieJar:
        cookies_text = ''
        for cookie in self.cookies:
            cookies_text += cookie.name + "=" + cookie.value + "; path=" + cookie.path + '; domain=' + cookie.domain + '\n'
        if len(cookies_text) > 10:
            cookies_text = cookies_text[:-1]
        cmd.extend([
            '-cookies', cookies_text
        ])

    cmd.extend([
        '-i', url,
        '-c:a', 'copy',
        '-c:v', 'copy',
        '-fs', '2G', # Addition inserted here
        filename
    ])

Not tested could lead to problems.

myh66 commented 3 months ago

You could try -fs 2G.

However, you will have to change the code again.

This should stop ffmpeg when 2GB is reached.

This should result in a short loss but the bot should continue to work.

The ffmpeg.py would have to be changed like this:

def getVideoFfmpeg(self, url, filename):
    cmd = [
        'ffmpeg',
        '-user_agent', self.headers['User-Agent']
    ]

    if type(self.cookies) is requests.cookies.RequestsCookieJar:
        cookies_text = ''
        for cookie in self.cookies:
            cookies_text += cookie.name + "=" + cookie.value + "; path=" + cookie.path + '; domain=' + cookie.domain + '\n'
        if len(cookies_text) > 10:
            cookies_text = cookies_text[:-1]
        cmd.extend([
            '-cookies', cookies_text
        ])

    cmd.extend([
        '-i', url,
        '-c:a', 'copy',
        '-c:v', 'copy',
        '-fs', '2G', # Addition inserted here
        filename
    ])

Not tested could lead to problems.

Ok, I just updated these codes and it works. I've limited uploads to nas to 2G and installed the Live Scan Uploader in the DOWNLOADS_DIR folder, if I'm recording and have the upload script open, will this affect the recording? I will test this later.

DerBunteBall commented 3 months ago

I have no idea what the Live Scan Uplaoder is.

In general:

A tool that only uploads files from the folder - i.e. only reads data - should not cause any problems. However, the data can of course be corrupted at the destination as long as the script does not ensure that the file is not still open and is therefore being processed.

Tools that perform other operations should also ensure that the file they are processing is no longer open.

myh66 commented 3 months ago

DerBunteBall

They should be conflicting, where can I view the logs? image It's a sync disk so I can move the video to where I want it after the recording is done so there are no conflicts. For example mv = 'C:\backup\

DerBunteBall commented 3 months ago

DEBUG must be set to True in the parameters.py file.

Logs of the ffmpeg output should then be created. Otherwise you can only check whether the CLI produces exceptions or other errors occur.

myh66 commented 3 months ago

DEBUG must be set to True in the parameters.py file.必须在 parameters.py 文件中将 DEBUG 设置为 True。

Logs of the ffmpeg output should then be created. Otherwise you can only check whether the CLI produces exceptions or other errors occur.然后创建 ffmpeg 输出日志。否则只能检查 CLI 是否产生异常或出现其他错误。

OK,Is it possible to move the video to the folder I want after recording? That way there is no conflict.

DerBunteBall commented 3 months ago

Not at the moment.

It would certainly be possible to incorporate this. But that would be a little more complex.

myh66 commented 3 months ago

Not at the moment.目前没有。

It would certainly be possible to incorporate this. But that would be a little more complex.当然,也有可能将其纳入其中。但这样做会比较复杂。

Looking forward to it, this could be an awesome feature!

DerBunteBall commented 3 months ago

See this

Not tested use at own risk.

myh66 commented 3 months ago

See this 见此

Not tested use at own risk.未经测试,使用风险自负。

ok,I downloaded it.What variables do I need to change?

DerBunteBall commented 3 months ago

parameters.py now knows the variable MOVE_DIR

This is the target folder. If everything works, the file should be moved to this folder when ffmpeg ends.

Before that it is in the folder that is set in DOWNLOADS_DIR.

MOVE_DIR should be created if it does not exist the first time.

I have included the previous changes.

Example:

If C:\Downloads is the DOWNLOADS_DIR and C:\completed\ is the MOVE_DIR, C:\Downloads\foo should be moved to C:\completed\foo after ffmpeg has finished.

DerBunteBall commented 3 months ago

This needs at least Python 3.4

I recommand using 3.11.5 in general for StreaMonitor.

Edit: Only FFmpeg Downloader is patched. There is actually no general Downloader Interface or Super Class.

myh66 commented 3 months ago

This needs at least Python 3.4至少需要 Python 3.4 版本

I recommand using 3.11.5 in general for StreaMonitor.我建议 StreaMonitor 一般使用 3.11.5。

I've been using 3.11.5 and this design is fantastic, I'm not very good at python, thanks a lot!I'll run the test right away.

myh66 commented 3 months ago

This needs at least Python 3.4

I recommand using 3.11.5 in general for StreaMonitor.

Edit: Only FFmpeg Downloader is patched. There is actually no general Downloader Interface or Super Class.

DOWNLOADS_DIR = 'downloads' folder The downloads folder is not automatically created by default, right?

DerBunteBall commented 3 months ago

In this case yes.

Because of the situation that you have a new setup I guess.

Because we set False at the genOutFilename function for create_folder before. This is included in the my patched branch.

Edit: Patched this in my branch. So the folder now gets created if it don't exists. Could be cleaner but it's just the function as it was before.

myh66 commented 3 months ago

In this case yes.在这种情况下,是的。

Because of the situation that you have a new setup I guess.我猜是因为你有了新的设置。

Because we set False at the genOutFilename function for create_folder before. This is included in the my patched branch.因为我们之前在 create_folder 的 genOutFilename 函数中设置了 False。这已包含在我的补丁分支中。

There seems to be a bug. I'll show you a screenshot. D:\Python\python.exe D:\Projectme\Live\StreaMonitor-01042024\Downloader.py

  • Serving Flask app 'streamonitor.managers.httpmanager'
    • Debug mode: off WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
    • Running on http://127.0.0.1:5000 Press CTRL+C to quit 2024-04-01 17:16:29,598 - INFO - [CB] quennacindy: Channel online 2024-04-01 17:16:29,619 - INFO - [CB] lilla_cutie: Channel online

2024-04-01 17:16:29,710 - INFO - manager_cli: Unknown command

2024-04-01 17:16:29,900 - INFO - manager_cli: Unknown command

2024-04-01 17:16:30,636 - INFO - [CB] lilla_cutie: Selected 1920x1080 resolution 2024-04-01 17:16:30,636 - INFO - [CB] lilla_cutie: Started downloading show 2024-04-01 17:16:30,636 - ERROR - [CB] lilla_cutie: 'WindowsPath' object has no attribute 'join' Traceback (most recent call last): File "D:\Projectme\Live\StreaMonitor-01042024\streamonitor\bot.py", line 155, in run ret = self.getVideo(self, video_url, self.genOutFilename()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Projectme\Live\StreaMonitor-01042024\streamonitor\downloaders\ffmpeg.py", line 13, in getVideoFfmpeg dest_file = Path(MOVE_DIR).join(dl_file) ^^^^^^^^^^^^^^^^^^^ AttributeError: 'WindowsPath' object has no attribute 'join' 2024-04-01 17:16:30,637 - INFO - [CB] lilla_cutie: Channel online 2024-04-01 17:16:30,737 - INFO - [CB] quennacindy: Selected 1920x1080 resolution 2024-04-01 17:16:30,737 - INFO - [CB] quennacindy: Started downloading show 2024-04-01 17:16:30,737 - ERROR - [CB] quennacindy: 'WindowsPath' object has no attribute 'join' Traceback (most recent call last): File "D:\Projectme\Live\StreaMonitor-01042024\streamonitor\bot.py", line 155, in run ret = self.getVideo(self, video_url, self.genOutFilename()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Projectme\Live\StreaMonitor-01042024\streamonitor\downloaders\ffmpeg.py", line 13, in getVideoFfmpeg dest_file = Path(MOVE_DIR).join(dl_file) ^^^^^^^^^^^^^^^^^^^ AttributeError: 'WindowsPath' object has no attribute 'join' 2024-04-01 17:16:30,737 - INFO - [CB] quennacindy: Channel online image image I modified parameters.py and ffmpeg.py and set a test limit of 0.1G. image

DerBunteBall commented 3 months ago

Fixed in my branch. My mistake. It's .joinpath() not .join().

myh66 commented 3 months ago

Fixed in my branch. My mistake. It's .joinpath() not .join().已在我的分支中修复。我的错误。是 .joinpath() 不是 .join()。

ok,i downloader

myh66 commented 3 months ago

Fixed in my branch. My mistake. It's .joinpath() not .join().

I just ran a test and moving the video doesn't seem to work,The best scenario is to move all downloaded videos after the streamer has finished broadcasting. image

DerBunteBall commented 3 months ago

Are the directories on the same drive?

Path.rename() is restricted to the same windows drive. If this should work over multiple drives we would need to try shutil. And eventally again check the code.

myh66 commented 3 months ago

Are the directories on the same drive?目录是否在同一驱动器上?

Path.rename() is restricted to the same windows drive. If this should work over multiple drives we would need to try shutil. And eventally again check the code.Path.rename() 仅限于同一 Windows 驱动器。如果要在多个驱动器上运行,我们需要尝试 shutil。最后再检查一遍代码。

Yes, it's on the same hard drive, but no moving video

DerBunteBall commented 3 months ago

Also in sense of Windows Drive? So it's nos something like C: and D:?

Another problem could be the system locale hm......

myh66 commented 3 months ago

hm......

No, it's the same hard drive.

DerBunteBall commented 3 months ago

Ok. I will need to test this. The issue will be solveable. It just need to be checked how it's done best.

Writing code that runs on a completely different OS is always a bit problematic

myh66 commented 3 months ago

Ok. I will need to test this. The issue will be solveable. It just need to be checked how it's done best.好的。我需要测试一下。这个问题是可以解决的。只需要查看最佳解决方案。

Writing code that runs on a completely different OS is always a bit problematic编写在完全不同操作系统上运行的代码始终都有些问题。

OK

ready2r commented 1 month ago

How would i go about changing path on Linux. I've tried:

DOWNLOADS_DIR = '/home/name/YTDL-Ubuntu/ DOWNLOADS_DIR = '~/YTDL-Ubuntu/ DOWNLOADS_DIR = r'/home/name/YTDL-Ubuntu/ DOWNLOADS_DIR = r'~/YTDL-Ubuntu/

All of them still outputs to the download folder

DerBunteBall commented 1 month ago

First things like ~, .. and . confuse os.mkdir and os.mkdirs.

DOWNLOADS_DIR = '/my/path/to/dl/dir/'

Should work fine.

Make sure totally stop the Downlaoder and start it again.