[BUG] Video with non-ASCII title character fails during ffmpeg concatenation

[rlatm]

Describe the bug

Downloading a video with a non-ASCII title character fails in the standalone downloader.

Using a binary release of SCrawler, it silently fails.

If debugging SCrawler from Visual Studio:

The output argument in the ffmpeg command-line appears to show up as ?????????????????????????????.mp4

This produces an error: ?????????????????????????????.mp4: Invalid argument

I also enabled the "Replace file by name by date" download setting but it had no effect, perhaps because that final filename is not used for the invocation of ffmpeg or because that setting is not applied in the standalone downloader.

It appears that the issue is within PersonalUtilities. It appears that source code for PersonalUtilities is not available? From decompiling it and walking through the source:

• It appears that SCrawler generates the destination filename correctly
• PersonalUtilities.Tools.BatchExecutor.Execute displays the non-ASCII characters as ?

To Reproduce

Steps to reproduce the behavior:

1. From the main window select 'Download'
2. Select 'Standalone downloader'
3. Select 'Add (Ins)'
4. Choose a web source that has foreign characters in it
5. Select 'Download (F5)'

Log data

The in-app "LOG" remains blank.

If debugging SCrawler from Visual Studio: -The output argument in the ffmpeg command-line appears to show up as ?????????????????????????????.mp4

• This produces an error: ?????????????????????????????.mp4: Invalid argument

Expected behavior

SCrawler saves the file with a valid filename.

Release information (please complete the following information):

• OS Windows 11
• Architecture x64
• Version 2023.1.2.0

Additional context

Thank you for releasing SCrawler (and making much of it open-source).

[AAndyProgram]

I need an example url.

It appears that source code for PersonalUtilities is not available?

No, it is not available.

From decompiling it and walking through the source, it appears that the destination filename is generated correctly, before outputting incorrectly as part of PersonalUtilities.Tools.BatchExecutor.Execute

If cmd doesn't allow ASCII characters, I think we need to find another way to rename files.

Thank you for releasing SCrawler (and making much of it open-source).

You are welcome but SCrawler is completely open-source. PersonalUtilities.dll is my personal library that contains my functions (mainly functions for working with collections, converters and some objects).

How to build from source

[rlatm]

Based on further debugging and testing:

• Debug.WriteLine("ж") displays correctly in the Visual Studio > Output > Debug window
• Starting a "cmd" process then calling .WriteLine("cd ж") on it, where a subfolder with that name exists, successfully changes into the specified subfolder. However, the non-ASCII character displays as ? in the Visual Studio > Output > Debug window. I am unsure why.
• As such, cmd seems to support the use of non-ASCII characters, but in debugging they display as ?, making troubleshooting more difficult.
• Copy/pasting the command-line generated by SCrawler into Windows Terminal completes successfully. As such, ffmpeg also supports non-ASCII characters.

In the end, I was able to workaround this by directly running Process.Start on ffmpeg, instead of indirectly executing ffmpeg within a cmd process. You can see this at https://github.com/rlatm/SCrawler/commit/b3bf63aa2baf94c0d26844148ceaad905b521af7#diff-a54d011fb17c2377b4c489bddb5f34f7b01f94f0e1087d94f5322a617548acc8L53-R72

My apologies for being unable to provide a sufficient issue submission. Closing this as I have a workaround. Thank you.

[AAndyProgram]

Okay, I'm glad if you solved the problem. But still, you didn't post an example URL (video with non-ASCII characters). If the problem persists in SCrawler, it should be fixed. Otherwise, you will be forced to recompile a new release every time I publish a new release.

If your URL is a porn URL, that's fine. But if you don't want to post this URL on GitHub, you can send it to me via Discord (AndyProgram#3804).

[AAndyProgram]

FYI For future issues: any issue without a problematic URL will be closed without attention.