NixOS / hydra

Hydra, the Nix-based continuous build system
http://nixos.org/hydra
GNU General Public License v3.0
1.15k stars 298 forks source link

<hash>.ls.xz contain improperly encoded non-unicode characters: #450

Open bennofs opened 7 years ago

bennofs commented 7 years ago

For example:

$ curl https://cache.nixos.org/4srlwrphq4z6iz354am0rf3vis84s8kw.ls.xz | xz -d | xxd
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   212  100   212    0     0    632      0 --:--:-- --:--:-- --:--:--   632
...
000000b0: 6de5 6c2e 616c 6961 7322 3a7b 2274 7970  m.l.alias":{"typ
...

Note the e5 byte, which is not a valid unicode character and thus should be escaped.

bennofs commented 7 years ago

Perhaps the file names should just be base64-encoded?

domenkozar commented 7 years ago

There are two packages indeed :)

@bennofs curious, what are you working on?

bennofs commented 7 years ago

@domenkozar

I'm working on nixpkgs-locate. Example:

d-cube:/c/nixpkgs-locate (master ⚡=) result/bin/locate -r -d out.fr 'locate$'
armagetronad.out:/share/games/armagetronad/scripts/relocate
expect.out:/bin/dislocate
findutils.out:/bin/locate
freeipmi.out:/bin/ipmi-locate
emscripten.out:/share/emscripten/system/include/libcxx/__undef___deallocate
libcxx.out:/include/c++/v1/__undef___deallocate
mlocate.out:/bin/locate
eject.bin:/bin/fallocate
eject.out:/share/bash-completion/completions/fallocate
libuuid.bin:/bin/fallocate
domenkozar commented 7 years ago

Oh me too! Currently have a restful api at https://github.com/Enlambda/nix-channelstatus

edolstra commented 7 years ago

Given the rarity of such paths, it might be easiest to just filter them out.

This is implemented in Nix BTW, not Hydra (https://github.com/NixOS/nix/blob/master/src/libstore/binary-cache-store.cc#L174).