Character encoding issue

[lyda]

I'm not seeing this in the current version. Not sure what's happening. Need to relearn perl encoding issues.

% git clone dokuwiki::http://dokuwiki.site/ dokuwiki-site     
Cloning into 'dokuwiki-site'...
Read configuration...
Dokuwiki server version: Release 2013-12-08 "Binky"
Connected to dokuwiki...
Fetching from beginning.
Getting pages (from Thu Jan  1 01:00:00 1970)...
Getting revisions (280 pages)...
Generating implied deletions (0 pages)...
Getting attachments (431 attachments)...
Generating implied deletions (0 attachments)...
Getting content (2214 revisions)...
Getting wiki/dokuwiki-128.png
Wide character in subroutine entry at /home/lyda/bin/git-remote-dokuwiki line 116.
fatal: Expected 'data n' command, found: 
fast-import: dumping crash report to /home/lyda/src/dokuwiki-site/.git/fast_import_crash_28979
fatal: Error while running fast-import

[lyda]

Using the version in debug/wide-char it generates this...

: lyda@salthill on pts/9; git clone dokuwiki::http://dokuwiki.example.com/ dokuwiki-site 
Cloning into 'dokuwiki-site'...
Read configuration...
Dokuwiki server version: Release 2013-12-08 "Binky"
Connected to dokuwiki...
Fetching from beginning.
Getting pages (from Thu Jan  1 01:00:00 1970)...
Getting revisions (280 pages)...
Generating implied deletions (0 pages)...
Getting attachments (431 attachments)...
Generating implied deletions (0 attachments)...
Getting content (2214 revisions)...
Getting wiki/dokuwiki-128.png (size: 33615, length: 33615, bytes::length: 33615)
Wide character in print at /home/lyda/bin/git-remote-dokuwiki line 117.
Wide character in print at /home/lyda/bin/git-remote-dokuwiki line 117.
Wide character in print at /home/lyda/bin/git-remote-dokuwiki line 117.
Wide character in print at /home/lyda/bin/git-remote-dokuwiki line 117.
Wide character in print at /home/lyda/bin/git-remote-dokuwiki line 117.
Wide character in print at /home/lyda/bin/git-remote-dokuwiki line 117.
[...]
fatal: Unsupported command: refs/dokuwiki/origin/master
fast-import: dumping crash report to /home/lyda/src/test/dokuwiki-site/.git/fast_import_crash_8400
fatal: Error while running fast-import

And the resulting 1.wiki:dokuwiki-128.png is fine.

[lyda]

#! /usr/bin/env perl

use strict;
use warnings;

binmode STDERR, ':encoding(UTF-8)';
binmode STDOUT, ':encoding(UTF-8)';

my $infile = "1.wiki:dokuwiki-128.png";
open my $in, '<:raw', $infile or die;
my $data = "";
while (1) {
  my $success = read $in, $data, 100, length($data);
  die $! if not defined $success;
  last if not $success;
}
close $in;

# print_data_raw...
utf8::downgrade($data);
binmode STDOUT, ":raw";
print STDERR length($data). "\n";
print $data;

And it's this version of the file...

https://github.com/splitbrain/dokuwiki/blob/f62ea8a1d1cf10eddeae777b11420624e111b7ea/data/media/wiki/dokuwiki-128.png

[zzkt]

similarly...

Dokuwiki server version: Release 2018-04-22b "Greebo" [...] Wide character in print at git-remote-dokuwiki line 117. [...] fatal: Unsupported command: ommit refs/dokuwiki/origin/master

[lyda]

Yep. The reason for the 2013 issue I was seeing was that attachment (which comes with dokuwiki) changed in 2015. If I put the version from 2013 into my 2018 dokuwiki I get the same issue.

I'm trying to figure out the encoding issues, but haven't done perl in a while so it will take me a bit. Suggestions welcome! My test program above isn't reproducing the issue annoyingly.

[lyda]

OK, new commit that might fix some encoding issues but it's still crashing out somewhere.

To better test this I've found I need to split clone into it's constituent parts because it deletes the dir on failure:

mkdir dokuwiki-test
git init
git remote add origin dokuwiki::http://dokuwiki.example.com/
git fetch

Failures from fast-import with logs will stick around.

[lyda]

Right. It should now work. I've cloned a few things and it looks correct. Let me know if I'm wrong.