[latexindent.exe] The -w option does not support Chinese filename

fengzyf commented 8 months ago

Please provide the following when posting an issue:

新建.tex code

test.

actual/given output

D:\test>latexindent -w "新建.tex" FATAL Could not write to backup file ./�½�.bak0. Please check permissions. Exiting, no indentation done.

cmhughes commented 7 months ago

Thanks for this.

I am unable to reproduce the problem.

On my Ubuntu machine, when I start with your file, 新建.tex and I run

latexindent.pl -w 新建.tex -g issue-505.log -y 'onlyOneBackUp:1'

then the log file is

INFO:  latexindent.pl version 3.23.4, 2023-11-19, a script to indent .tex files
       latexindent.pl lives here: /home/cmhughes/projects/latexindent/
       Wed Dec 27 10:43:53 2023
       Filename: 新建.tex
INFO:  Processing switches:
       -w|--overwrite: Overwrite mode active, will make a back up before overwriting
       -y|--yaml: YAML settings specified via command line
       -g|--logfile: logfile name
INFO:  Directory for backup files and issue-505.log:
       .
INFO:  Perl modules are being loaded from the following directories:
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/FindBin.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/site_perl/5.36.0/YAML/Tiny.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/File/Copy.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/File/Basename.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/Getopt/Long.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/site_perl/5.36.0/File/HomeDir.pm
INFO:  LatexIndent perl modules are being loaded from, for example:
       /home/cmhughes/projects/latexindent/LatexIndent/Document.pm
INFO:  YAML settings read: defaultSettings.yaml
       Reading defaultSettings.yaml from /home/cmhughes/projects/latexindent/defaultSettings.yaml
INFO:  YAML reading settings
       Home directory is /home/cmhughes
       latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
       see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO:  YAML settings read: -y switch
       YAML setting: onlyOneBackUp:1
       Updating mainSettings with onlyOneBackUp: 1
INFO:  Backup procedure (-w flag active):
       ./新建.bak will be overwritten (see onlyOneBackUp)
       Backing up 新建.tex to ./新建.bak...
       新建.tex will be overwritten after indentation
INFO:  Phase 1: searching for objects
       No objects found.
INFO:  Phase 2: finding surrounding indentation
INFO:  Phase 3: indenting objects
INFO:  Phase 4: final indentation check
INFO:  Output routine:
       Overwriting file 新建.tex
       --------------
INFO:  Please direct all communication/issues to:
        https://github.com/cmhughes/latexindent.pl

Please can you provide more details?

fengzyf commented 7 months ago

My OS: Windows 10 22H2 19045.3803 (Chinese version) My latexindent.exe version: 3.23.4, 2023-11-19

When I run

latexindent -w 新建.tex -g issue-505.log -y 'onlyOneBackUp:1'

then the log file is

INFO:  latexindent.exe version 3.23.4, 2023-11-19, a script to indent .tex files
       latexindent.exe lives here: D:/test/
       Wed Dec 27 22:32:48 2023
       Filename: �½�.tex
INFO:  Processing switches:
       -w|--overwrite: Overwrite mode active, will make a back up before overwriting
       -y|--yaml: YAML settings specified via command line
       -g|--logfile: logfile name
INFO:  Directory for backup files and issue-505.log:
       .
INFO:  YAML settings read: defaultSettings.yaml
       Reading defaultSettings.yaml from D:/test/defaultSettings.yaml
       Reading defaultSettings.yaml (2nd attempt) from D:/test/../../texmf-dist/scripts/latexindent/defaultSettings.yaml
       and then, if necessary, D:/test/LatexIndent/defaultSettings.yaml
INFO:  YAML reading settings
       Home directory is C:\Users\lenovo
       latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
       see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO:  YAML settings read: -y switch
       YAML setting: 'onlyOneBackUp:1'
       Updating mainSettings with 'onlyOneBackUp: 1'
INFO:  Backup procedure (-w flag active):
       Backing up �½�.tex to ./�½�.bak0...
       �½�.tex will be overwritten after indentation
FATAL Could not write to backup file ./�½�.bak0. Please check permissions.
      Exiting, no indentation done.
       --------------
INFO:  Please direct all communication/issues to:
        https://github.com/cmhughes/latexindent.pl

cmhughes commented 7 months ago

What happens if your file is named aaaa.tex?

fengzyf commented 7 months ago

The process completed successfully, generating aaaa.bak0.

cmhughes commented 7 months ago

How does the o switch behave?

Can you post the output from

latexindent.pl 新建.tex -g issue-505.log -o 新建.tex

fengzyf commented 7 months ago

INFO: latexindent.pl version 3.23.5, 2024-01-01, a script to indent .tex files latexindent.pl lives here: C:/CTEX/MiKTeX/scripts/latexindent/ Sun Jan 7 22:52:12 2024 Filename: �½�.tex INFO: Processing switches: -o|--outputfile: output to file -g|--logfile: logfile name INFO: Directory for backup files and issue-505.log: . INFO: Perl modules are being loaded from the following directories: E:/program/Strawberry/perl/lib/FindBin.pm E:/program/Strawberry/perl/vendor/lib/YAML/Tiny.pm E:/program/Strawberry/perl/lib/File/Copy.pm E:/program/Strawberry/perl/lib/File/Basename.pm E:/program/Strawberry/perl/lib/Getopt/Long.pm E:/program/Strawberry/perl/vendor/lib/File/HomeDir.pm INFO: LatexIndent perl modules are being loaded from, for example: C:/CTEX/MiKTeX/scripts/latexindent/LatexIndent/Document.pm INFO: YAML settings read: defaultSettings.yaml Reading defaultSettings.yaml from C:/CTEX/MiKTeX/scripts/latexindent/defaultSettings.yaml INFO: YAML reading settings Home directory is C:\Users\lenovo latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options) INFO: -o switch active: output file check INFO: Phase 1: searching for objects No objects found. INFO: Phase 2: finding surrounding indentation INFO: Phase 3: indenting objects INFO: Phase 4: final indentation check INFO: Output routine: Outputting to file �½�.tex

INFO: Please direct all communication/issues to: https://github.com/cmhughes/latexindent.pl

fengzyf commented 7 months ago

According to the code in https://github.com/cmhughes/latexindent.pl/issues/263#issuecomment-830560942

#!perl -w
use autodie;
use Encode;

my $file ="./测试.txt";
# We have to first decode utf-8 because perl do not use utf-8 in default.
# We can `use utf8;` to set utf-8 encoding into default.
$file = encode("gb2312", decode("utf-8", $file));
open my $in,'<',"$file";

while (<$in>){
print;
}

In the Chinese version of Windows, it appears that perl receives commands from cmd encoded in GB2312 instead of utf-8. Therefore, you need to use a command like this to convert the encoding:

decode("GB2312", ${$self}{fileName} )

And when it comes to file operations, the string needs to be converted back to GB2312 encoding. Such as:

copy( encode("GB2312", $fileName), encode("GB2312", $backupFile)

The following example confirms this.

If BackUpFileProcedure.pm is changed to the following file BackUpFileProcedure.txt

When I run

latexindent -w 新建.tex -g issue-505.log

the bak file is generated as normal and the log file is

INFO:  latexindent.pl version 3.23.5, 2024-01-01, a script to indent .tex files
       latexindent.pl lives here: C:/CTEX/MiKTeX/scripts/latexindent/
       Sun Jan  7 23:33:34 2024
       Filename: �½�.tex
INFO:  Processing switches:
       -w|--overwrite: Overwrite mode active, will make a back up before overwriting
       -g|--logfile: logfile name
INFO:  Directory for backup files and issue-505.log:
       .
INFO:  Perl modules are being loaded from the following directories:
       E:/program/Strawberry/perl/lib/FindBin.pm
       E:/program/Strawberry/perl/vendor/lib/YAML/Tiny.pm
       E:/program/Strawberry/perl/lib/File/Copy.pm
       E:/program/Strawberry/perl/lib/File/Basename.pm
       E:/program/Strawberry/perl/lib/Getopt/Long.pm
       E:/program/Strawberry/perl/vendor/lib/File/HomeDir.pm
INFO:  LatexIndent perl modules are being loaded from, for example:
       C:/CTEX/MiKTeX/scripts/latexindent/LatexIndent/Document.pm
INFO:  YAML settings read: defaultSettings.yaml
       Reading defaultSettings.yaml from C:/CTEX/MiKTeX/scripts/latexindent/defaultSettings.yaml
INFO:  YAML reading settings
       Home directory is C:\Users\lenovo
       latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
       see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO:  Backup procedure (-w flag active):
       Backing up 新建.tex to ./新建.bak0...
       新建.tex will be overwritten after indentation
INFO:  Phase 1: searching for objects
       No objects found.
INFO:  Phase 2: finding surrounding indentation
INFO:  Phase 3: indenting objects
INFO:  Phase 4: final indentation check
INFO:  Output routine:
       Overwriting file �½�.tex
       --------------
INFO:  Please direct all communication/issues to:
        https://github.com/cmhughes/latexindent.pl

Therefore, I believe that you can refer to https://github.com/cmhughes/latexindent.pl/issues/263 and https://github.com/cmhughes/latexindent.pl/pull/273 to add a switch to the user to specify their encoding.

cmhughes commented 7 months ago

That's helpful many thanks.

cmhughes commented 7 months ago

For my reference, this file will need updating

https://github.com/cmhughes/latexindent.pl/blob/main/.github/workflows/batch-check.yaml

cmhughes commented 5 months ago

Implemented as of https://github.com/cmhughes/latexindent.pl/commit/f75a818690121a0691665a1600bb9e18875e5205

Please leave this open until I have released it.

cmhughes commented 5 months ago

Thanks again, released at https://github.com/cmhughes/latexindent.pl/releases/tag/V3.23.7

cmhughes / latexindent.pl

[latexindent.exe] The -w option does not support Chinese filename #505

新建.tex code

actual/given output