search

hanw / git-repo

Automatically exported from code.google.com/p/git-repo
Apache License 2.0
0 stars 0 forks source link

mirror of mirror and UTF-8 encoded tag name #167

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi,

I was surprised (positively) to be allowed to execute "repo init --mirror 
--reference <directory> -u ... -b ..." to create a mirror based on another 
mirror.

Unfortunately, when I run "repo sync", it fails:

    $ repo init --mirror -u ssh://gitolite@example.co.de.com/foo/FOO_manifests.git --reference $PWD/../../mirrors/master-default -b master -m default.xml

    $ repo sync
    [...]
    Fetching projects:  33% (20/60)  Fetching project xxxx/XXXX_YYYYYYYYY_zzz
    Traceback (most recent call last):
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/main.py", line 500, in <module>
        _Main(sys.argv[1:])
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/main.py", line 476, in _Main
        result = repo._Run(argv) or 0
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/main.py", line 155, in _Run
        result = cmd.Execute(copts, cargs)
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/subcmds/sync.py", line 635, in Execute
        fetched = self._Fetch(to_fetch, opt)
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/subcmds/sync.py", line 339, in _Fetch
        self._FetchProjectList(**kwargs)
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/subcmds/sync.py", line 234, in _FetchProjectList
        success = self._FetchHelper(opt, project, *args, **kwargs)
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/subcmds/sync.py", line 275, in _FetchHelper
        no_tags=opt.no_tags, archive=self.manifest.IsArchive)
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/project.py", line 1115, in Sync_NetworkHalf
        no_tags=no_tags):
      File "/home/ydroneaud/src/mirrors/test/.repo/repo/project.py", line 1784, in _RemoteFetch
        tmp_packed += line
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 51: ordinal not in range(128)

Indeed, at this stage, repo is processing a tag named "–b" (which is *not* 
"-b") ... probably a tag created as a result of a typo.

This tag was never a problem for my day to day usage of git-repo: the issue 
reported here arise only when running the first "repo sync" after initializing 
a workspace to be a mirror, using another mirror.

I was able to "work around" this issue with the following temporary patch: 
expecting the packed ref tag names being always encoded in UTF-8 is probably 
wrong and cannot apply to every environment.

Regards.

Original issue reported on code.google.com by yann@droneaud.fr on 2 May 2014 at 11:33

Attachments:

GoogleCodeExporter commented 9 years ago 
Testing tag creation with locale fr_FR.UTF-8 and fr_FR.ISO-8859-15 prove that 
git (version 1.9.0) use current encoding to record a tag and do not perform any 
kind of canonicalisation (no conversion to UTF-8):

.git/packed-refs (showed as UTF-8 content):

    $ cat packed-refs 
    # pack-refs with: peeled fully-peeled 
    9dec613b6860dc0bf5cbe23e98abd5f6602e527c refs/heads/master
    9dec613b6860dc0bf5cbe23e98abd5f6602e527c refs/tags/é
    9dec613b6860dc0bf5cbe23e98abd5f6602e527c refs/tags/

.git/packed-refs (showed as ISO-8859-15 content):

    $ cat packed-refs 
    # pack-refs with: peeled fully-peeled 
    9dec613b6860dc0bf5cbe23e98abd5f6602e527c refs/heads/master
    9dec613b6860dc0bf5cbe23e98abd5f6602e527c refs/tags/é
    9dec613b6860dc0bf5cbe23e98abd5f6602e527c refs/tags/é

Binary dump:

    $ hexdump -C packed-refs 
    00000000  23 20 70 61 63 6b 2d 72  65 66 73 20 77 69 74 68  |# pack-refs with|
    00000010  3a 20 70 65 65 6c 65 64  20 66 75 6c 6c 79 2d 70  |: peeled fully-p|
    00000020  65 65 6c 65 64 20 0a 39  64 65 63 36 31 33 62 36  |eeled .9dec613b6|
    00000030  38 36 30 64 63 30 62 66  35 63 62 65 32 33 65 39  |860dc0bf5cbe23e9|
    00000040  38 61 62 64 35 66 36 36  30 32 65 35 32 37 63 20  |8abd5f6602e527c |
    00000050  72 65 66 73 2f 68 65 61  64 73 2f 6d 61 73 74 65  |refs/heads/maste|
    00000060  72 0a 39 64 65 63 36 31  33 62 36 38 36 30 64 63  |r.9dec613b6860dc|
    00000070  30 62 66 35 63 62 65 32  33 65 39 38 61 62 64 35  |0bf5cbe23e98abd5|
    00000080  66 36 36 30 32 65 35 32  37 63 20 72 65 66 73 2f  |f6602e527c refs/|
    00000090  74 61 67 73 2f c3 a9 0a  39 64 65 63 36 31 33 62  |tags/...9dec613b|
    000000a0  36 38 36 30 64 63 30 62  66 35 63 62 65 32 33 65  |6860dc0bf5cbe23e|
    000000b0  39 38 61 62 64 35 66 36  36 30 32 65 35 32 37 63  |98abd5f6602e527c|
    000000c0  20 72 65 66 73 2f 74 61  67 73 2f e9 0a           | refs/tags/..|
    000000cd

This confirms that my temporary patch is just a workaround for my case and 
cannot be enabled for everyone.

Regards.

Original comment by yann@droneaud.fr on 2 May 2014 at 12:01