NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.21k stars 14.21k forks source link

Packaging Request: Veloren #95226

Closed JonathanILevi closed 2 years ago

JonathanILevi commented 4 years ago

Project description Veloren is a multiplayer voxel RPG written in Rust. Veloren takes inspiration from games such as Cube World, Minecraft and Dwarf Fortress. The game is currently under heavy development, but is playable.

kira-bruneau commented 4 years ago

Veloren actually already has embedded nix files: https://github.com/veloren/veloren/tree/master/nix.

Hey @AngelOnFira! I guess you might not recognize me, but I remember you telling me about the nix build for Veloren when I was visiting Colorado last year. Is there a reason why this hasn't been submitted to nixpkgs yet?

AngelOnFira commented 4 years ago

@MetaDark Hey :wave: I haven't looked too much into nix for Veloren, I know we had a contributor that added nix configs, but I don't think we've looked at packaging it. I think we officially build for the AUR, flatpak, and maybe a few others, not 100% sure though. I don't think there are any issues with doing this, but I'm not sure if there is anything special this might entail. Also, is it possible to package from a Gitlab repo? Our Github mirror isn't doing too hot since we're running into LFS restrictions. Github currently shows that the last commit was over a month ago :grimacing:

AngelOnFira commented 4 years ago

I'll include @yusdacra on this thread as they were the one to get our Nix stuff set up :100:

kira-bruneau commented 4 years ago

@AngelOnFira Yep, there's a helper called fetchFromGitLab which can be used. I'm not really sure how it integrates with LFS though since I think the archive it grabs doesn't include files from LFS.

kira-bruneau commented 4 years ago

It looks like it might be better to just use fetchgit and run the git lfs commands as part of postFetch. See https://discourse.nixos.org/t/how-to-fetch-lfs-enabled-repo-with-fetchfromgithub/5890/3.

lboklin commented 3 years ago

I tried to write a derivation to add to my flake-based system:

{ 
    veloren =
      let
        velorenSrc = pkgs.fetchgit {
          url = "https://gitlab.com/veloren/veloren";
          rev = "v0.8.0";
          sha256 = "sha256-Y/z9pLVknTJbz0QkZFsXtgBkMlaYAmKcGS4dqy9ApSQ=";
          leaveDotGit = true;
        };
        system = "x86_64-linux";
        sources = import "${velorenSrc}/nix/sources.nix" { inherit system pkgs; };
      in
        import "${velorenSrc}/nix" { inherit system sources; };
    #...
}

The git-lfs stuff is handled already in there, but I get stuck on this error:

error: --- Error -------------------------------------------------------------------------------- nix
in pure evaluation mode, 'fetchurl' requires a 'sha256' argument

which originates from /nix/store/fghm13lvj2l83bxradhxkjvn2x0h8n7k-nixpkgsMoz-src/rust-overlay.nix (meaning I don't know what I can do about this). This is getting pulled in from here: https://gitlab.com/veloren/veloren/-/blob/bce6e1c6e953c4531519ee4c4138ea0a7e34bb78/nix/nixpkgs.nix

yusdacra commented 3 years ago

Hello! That looks like an issue with nixpkgs-mozilla, which is what we use to pull in the Rust toolchain version we use (which is declared in the rust-toolchain file, in the root of the repo). I will try to reproduce that (i didn't try flakes much yet). For now you can try the yusdacra/fix-nix-build-tag-bug branch of the Veloren repository and see if it fixes it, or use nix-env to install it.

EDIT: The branch is merged into master now, so you can use master branch :)

lboklin commented 3 years ago

Hello! That looks like an issue with nixpkgs-mozilla, which is what we use to pull in the Rust toolchain version we use (which is declared in the rust-toolchain file, in the root of the repo). I will try to reproduce that (i didn't try flakes much yet). For now you can try the yusdacra/fix-nix-build-tag-bug branch of the Veloren repository and see if it fixes it, or use nix-env to install it.

EDIT: The branch is merged into master now, so you can use master branch :)

It seems to have fixed the issue. I'm now getting a failure from /nix/store/h42zql6d38gwglnb8q0dabsp3ypxkl5p-gitLfsCheck.drv: Full output.

Using same expression as before but with latest rev from master.

yusdacra commented 3 years ago

Hello! That looks like an issue with nixpkgs-mozilla, which is what we use to pull in the Rust toolchain version we use (which is declared in the rust-toolchain file, in the root of the repo). I will try to reproduce that (i didn't try flakes much yet). For now you can try the yusdacra/fix-nix-build-tag-bug branch of the Veloren repository and see if it fixes it, or use nix-env to install it.

EDIT: The branch is merged into master now, so you can use master branch :)

It seems to have fixed the issue. I'm now getting a failure from /nix/store/h42zql6d38gwglnb8q0dabsp3ypxkl5p-gitLfsCheck.drv: stderr.

Using same expression as before but with latest rev from master.

Yikes, that's my fault. It should've given you a message that says "git-lfs isn't setup, bla bla". fetchgit doesn't handle git-lfs so you'll need to setup git-lfs with the postFetch hook.

lboklin commented 3 years ago

I'm not having any luck getting LFS to work.

I wrote this wrapper for fetchgit:

let
  fetchgitLFS = args:
    let
      args' = args // {
        fetchSubmodules = true;
        leaveDotGit = true;
        postFetch = ''
          git lfs install --local
          git lfs fetch
          git lfs checkout
        '';
      };

    in
      (pkgs.fetchgit args').overrideAttrs (oldAttrs: {
        nativeBuildInputs = oldAttrs.nativeBuildInputs or [] ++ [ pkgs.git-lfs ];
      });
in  # ...

and just substituted my usage of fetchgit for it, but when those git lfs ... commands are run, the current directory is /build, which neither is nor contains the repo. I figured this out by setting postFetch to

''
pwd
ls -A
exit 1
''
lboklin commented 3 years ago

Aha, I had to cd $out before running the git lfs ... commands. Seems weird to me that it wouldn't put me in the repo in postFetch though.

yusdacra commented 3 years ago

Aha, I had to cd $out before running the git lfs ... commands. Seems weird to me that it wouldn't put me in the repo in postFetch though.

So does it work now? If so, nice!

lboklin commented 3 years ago

I now get another error:

[...]
Nothing new to pack.
Updated git hooks.
Git LFS initialized.
fetch: Fetching reference refs/heads/fetchgit
Downloading LFS objects:   0% (0/1), 0 B | 0 B/s^Mbatch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
batch request: missing protocol: ""
error: failed to fetch some objects from ''

but at least the commands are run.

Edit: git remote -v returns nothing, so that's probably the culprit. Edit2: I'm doing an ugly git remote add origin ${args.url}. It's currently doing something. Edit3: Not there yet:

[...]
Total 74583 (delta 49981), reused 22648 (delta 0), pack-reused 0
Enumerating objects: 74583, done.
Nothing new to pack.
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

Edit4: Oops, I think that last one was me accidentally removing cd $out from postFetch.

lboklin commented 3 years ago

Ok, so far I have this:

{ pkgs, ... }:

let
  fetchgitLFS = args:
    let
      args' = args // {
        fetchSubmodules = true;
        leaveDotGit = true;
        # deepClone = true;
        postFetch = ''
          cd $out
          git remote add origin ${args.url}
          git lfs install --local
          git lfs fetch
          git lfs checkout
        '';
      };

    in
      (pkgs.fetchgit args').overrideAttrs (oldAttrs: {
        nativeBuildInputs = oldAttrs.nativeBuildInputs or [] ++ [ pkgs.git-lfs ];
      });

  velorenSrc = fetchgitLFS {
    url = "https://gitlab.com/veloren/veloren";
    rev = "30563f59f3d09115b8b6a7d9fddb6cfb1f842e6a";
    sha256 = "sha256-vMl6FmpK7DLa7wtNh+n1H0MS0t6wcFqpDdqbjbFRj2Q=";
  };

  system = "x86_64-linux";

in
  import "${velorenSrc}/nix" { inherit system; }

but the problem is that every time I get a different hash, so there's always a mismatch.

Edit: I forgot to check the log. There's this:

error: assets/voxygen/audio/ambient/wind.ogg: cannot add to the index - missing --add option?
fatal: Unable to process path assets/voxygen/audio/ambient/wind.ogg

A bit silly that the reported error is hash mismatch when something went wrong, but whatever.

lboklin commented 3 years ago

I don't know what it means by error: assets/voxygen/audio/ambient/wind.ogg: cannot add to the index - missing --add option?

There's no --add flag for git lfs checkout, but that's the command producing the error. I don't really know how this works, tbh.

I went through the process manually with a local clone of the repo and it worked fine.

yusdacra commented 3 years ago

I don't know what it means by error: assets/voxygen/audio/ambient/wind.ogg: cannot add to the index - missing --add option?

There's no --add flag for git lfs checkout, but that's the command producing the error. I don't really know how this works, tbh.

I went through the process manually with a local clone of the repo and it worked fine.

That sounds like the index file is missing in the repo, which git-lfs can't operate without. Can you try putting git checkout ${args.rev} after adding the remote? TBH I don't see how it will help since fetchgit should already do that... But maybe that will work?

I can add a way to override the version & git-lfs command so that git and git-lfs commands won't run (we use git to determine what to put in the version string, both in game and in Nix derivation) and then you should be able to fetch it via fetchFromGitLab, which should have the assets too (downloading an archive from GitLab does at least). It would work, but I feel that's kinda dirty.

lboklin commented 3 years ago

Progress! In a narrow sense of the word..

Adding ${args.rev} helped. Now we're back to

error: --- Error --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
builder for '/nix/store/h42zql6d38gwglnb8q0dabsp3ypxkl5p-gitLfsCheck.drv' failed with exit code 1
------------------------------------------------------------------------------------------------ show-trace ------------------------------------------------------------------------------------------------
trace: while evaluating the attribute 'DISABLE_GIT_LFS_CHECK' of the derivation 'rust_veloren-common-0.8.0'
yusdacra commented 3 years ago

Progress! In a narrow sense of the word..

Adding ${args.rev} helped. Now we're back to

error: --- Error --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
builder for '/nix/store/h42zql6d38gwglnb8q0dabsp3ypxkl5p-gitLfsCheck.drv' failed with exit code 1
------------------------------------------------------------------------------------------------ show-trace ------------------------------------------------------------------------------------------------
trace: while evaluating the attribute 'DISABLE_GIT_LFS_CHECK' of the derivation 'rust_veloren-common-0.8.0'

Can you try with the latest commit on branch yusdacra/override-git-lfs, and adding disableGitLfsCheck = true; to last import "${velorenSrc}/nix" { ... }? That should disable the git-lfs check. I want to see if the assets are actually in place since the git-lfs check fails. If the game builds and can't load one of the assets it'll panic and abort.

lboklin commented 3 years ago

When I tried to build my system flake I got this:

error: --- Error --- nix-daemon
builder for '/nix/store/dcq4i2vxf95p745k1g8rr8j14x6zn5k8-iced-f464316.drv' failed with exit code 1; last 5 log lines:
exporting https://github.com/hecrj/iced (rev f46431600cb61d4e83e0ded1ca79525478436be3) into /nix/store/qn43ka6287ih8932kc252wrap4ngmiw1-iced-f464316
Initialized empty Git repository in /nix/store/qn43ka6287ih8932kc252wrap4ngmiw1-iced-f464316/.git/
error: RPC failed; curl 18 transfer closed with outstanding read data remaining
fatal: expected flush after ref listing
Unable to checkout f46431600cb61d4e83e0ded1ca79525478436be3 from https://github.com/hecrj/iced.
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/2phv70v90z0ja1zs2mn02ipaq4x6civk-rust_iced_core-0.2.1.drv' failed to build
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/cqvdfdz6lc6vb9mq8k12mbqx2xlcvmbq-veloren-voxygen_e7eb51ce-2020-12-04-18-28.drv' failed to build

I'm currently building it with nix-build --arg pkgs '(builtins.getFlake "nixpkgs").outputs.legacyPackages.x86_64-linux'. I'll let you know what comes out of it after it finished compiling. I have no idea how long it'll take.

lboklin commented 3 years ago

The assets are indeed not where they should be.

Also, the error from my nixos-rebuild build was a fluke. At least after building it once with nix-build.

yusdacra commented 3 years ago

The assets are indeed not where they should be.

Also, the error from my nixos-rebuild build was a fluke. At least after building it once with nix-build.

Hmm, then that means the git-lfs commands don't do anything? I'm quite confused here. Perhaps check if there are anything under the assets directory before running each git command in postFetch hook? Also what happens if you make deepClone true?

lboklin commented 3 years ago

It's very strange. I appended postFetch with [ -f assets/voxygen/audio/ambient/wind.ogg ] || exit 1 just to make sure.

Then I did import (pkgs.lib.traceVal "${velorenSrc}/nix") # ... so that I can manually check the contents, which does indeed have assets (wind.ogg at least). Still the build panics with:

Panic Payload: "Failed loading essential asset: voxygen.texture.noise (error=ParseError(Unsupported(UnsupportedError { format: Unknown, kind: Format(Unknown) })))"
PanicInfo: panicked at 'Failed loading essential asset: voxygen.texture.noise (error=ParseError(Unsupported(UnsupportedError { format: Unknown, kind: Format(Unknown) })))', /source/src/assets/mod.rs:220:1
3
Game version: e7eb51ce [2020-12-04]

Backtrace:
0: veloren_voxygen::main::{{closure}}
1: std::panicking::rust_panic_with_hook
2: std::panicking::begin_panic_handler::{{closure}}
3: std::sys_common::backtrace::__rust_end_short_backtrace
4: rust_begin_unwind
5: std::panicking::begin_panic_fmt
6: veloren_common::assets::Asset::load_expect::{{closure}}
7: veloren_voxygen::render::renderer::Renderer::new
8: veloren_voxygen::window::Window::new
9: veloren_voxygen::main
10: std::sys_common::backtrace::__rust_begin_short_backtrace
11: std::rt::lang_start::{{closure}}
12: std::rt::lang_start_internal
13: main
14: __libc_start_main
15: _start

The contents of /nix/store/...-veloren-e7eb51c/assets/voxygen/texture/ (the fetched source) are these two files:

I'm trying with deepClone = true;. After that I'll insert recursive ls or something for the assets dir before each git command as you suggest.

lboklin commented 3 years ago

Wait, where does the game look for the assets?

Edit:

$ head -n 2 result-2/bin/veloren-voxygen
#! /nix/store/516z50fm1jbpcl32qnzy7kynrh0vl22w-bash-4.4-p23/bin/bash -e
export VELOREN_ASSETS='/nix/store/xwdgn945fjikr5d8hicz1wpn8l3l8w1h-makeAssetsDir'

$ ls -lA /nix/store/xwdgn945fjikr5d8hicz1wpn8l3l8w1h-makeAssetsDir/assets/voxygen/audio/ambient/
total 4
-r--r--r-- 1 root root 131  1 jan  1970 wind.ogg

$ ls /nix/store/gq5lyzlkjpwlr3q6dhsyk5dwizbilnwn-veloren-e7eb51c/assets/voxygen/texture/
total 8
-r--r--r-- 1 root root 129  1 jan  1970 noise.png
-r--r--r-- 1 root root 130  1 jan  1970 waves.png
yusdacra commented 3 years ago

Wait, where does the game look for the assets?

If you check the default.nix file, you'll see a veloren-assets variable, which is basically created like this:

  veloren-assets = pkgs.runCommand "makeAssetsDir" { } ''
    mkdir $out
    ln -sf ${../assets} $out/assets
  '';

Then I wrap the binary and set VELOREN_ASSETS to veloren-assets, so that's where the game looks for the assets directory.

lboklin commented 3 years ago

Updated my comment. I don't get the problem really.

yusdacra commented 3 years ago

Updated my comment. I don't get the problem really.

Those errors basically mean that git-lfs doesn't work properly. I'm at a loss here too.

I will add a "override version" option so that we can also skip the version checks and directly use fetchFromGitLab and see if it works.

lboklin commented 3 years ago

Ohh..

$ file /nix/store/xwdgn945fjikr5d8hicz1wpn8l3l8w1h-makeAssetsDir/assets/voxygen/audio/ambient/wind.ogg
/nix/store/xwdgn945fjikr5d8hicz1wpn8l3l8w1h-makeAssetsDir/assets/voxygen/audio/ambient/wind.ogg: ASCII text
yusdacra commented 3 years ago

Ok, so I added a way to override the version. Try the latest commit of the same branch. Now instead of using fetchgit, try using fetchFromGitLab, without any postFetch hooks or whatever. Just make sure to pass the disableGitLfsCheck = true; and overrideVersion = <revision of commit used to fetch source> while importing.

lboklin commented 3 years ago

I seem to be stuck on

trying https://gitlab.com/api/v4/projects/veloren%2Fveloren/repository/archive.tar.gz?sha=96d68d9e542735c2596b4fa7e0ef600a7359c839

There's no reported progress, but maybe it is progressing and it just takes a reeeaally long time. My network activity is mostly idle too, so I don't know what it's doing. I cancelled two or three times but I'll leave it for an hour or two this time.

Edit: Ok it finished. It's on to compiling now.

Edit2:

$ file /nix/store/cddfcxdhfh5xp3xfb6xcdpr4nq3pysik-source/assets/voxygen/audio/ambient/wind.ogg
/nix/store/cddfcxdhfh5xp3xfb6xcdpr4nq3pysik-source/assets/voxygen/audio/ambient/wind.ogg: Ogg data, Vorbis audio, stereo, 44100 Hz, ~160000 bps, created by: Xiph.Org libVorbis I (1.3.3)

Still compiling, but that part's looking good.

lboklin commented 3 years ago

It runs!

The window isn't actually displaying anything (it's just a frozen image of my screen), but no other bad things are happening.

lboklin commented 3 years ago

The server does not run, however:

$ RUST_BACKTRACE=1 ./result/bin/veloren-server-cli                                                                                                                                      home  ✭ ✈ ✱
thread 'main' panicked at 'failed to retrieve git_datetime!', src/util/mod.rs:13:75
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::option::expect_failed
3: std::sync::once::Once::call_once::{{closure}}
4: std::sync::once::Once::call_inner
5: std::sync::once::Once::call_once::{{closure}}
6: std::sync::once::Once::call_inner
7: std::sync::once::Once::call_once::{{closure}}
8: std::sync::once::Once::call_inner
9: std::sync::once::Once::call_once::{{closure}}
10: std::sync::once::Once::call_inner
11: <veloren_common::util::DISPLAY_VERSION_LONG as core::ops::deref::Deref>::deref
12: veloren_server_cli::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
yusdacra commented 3 years ago

The server does not run, however:

$ RUST_BACKTRACE=1 ./result/bin/veloren-server-cli                                                                                                                                      home  ✭ ✈ ✱
thread 'main' panicked at 'failed to retrieve git_datetime!', src/util/mod.rs:13:75
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::option::expect_failed
3: std::sync::once::Once::call_once::{{closure}}
4: std::sync::once::Once::call_inner
5: std::sync::once::Once::call_once::{{closure}}
6: std::sync::once::Once::call_inner
7: std::sync::once::Once::call_once::{{closure}}
8: std::sync::once::Once::call_inner
9: std::sync::once::Once::call_once::{{closure}}
10: std::sync::once::Once::call_inner
11: <veloren_common::util::DISPLAY_VERSION_LONG as core::ops::deref::Deref>::deref
12: veloren_server_cli::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Both freeze and crash is caused since the game expects a proper formatted gitHash, which we aren't giving here. So I want to fix the git-lfs issue, but in the mean time I will upstream an assets argument, which will point to assets of the game. This will fix the issue, but you'll need to use fetchgit (for version detection) and fetchFromGitLab (to get the assets), which is a dirty workaround.

Thanks for your time :)

lboklin commented 3 years ago

I too would really like to get fetchgit to work with git-lfs. Anything hackier than that would be too embarrassing for a PR, hehe.

Thanks for being accommodating on the other end :)

yusdacra commented 3 years ago

@lboklin can you try the branch yusdacra/nix-assets? You need to pass in the assets using the assets argument as I explained above but use the fetchgit source for importing default.nix.

nixos-discourse commented 3 years ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-fetch-lfs-enabled-repo-with-fetchfromgithub/5890/9

lboklin commented 3 years ago

It works perfectly.

This is cleaned up from what worked for me just now. I'll update in a bit with the real hash when I've confirmed it works verbatim.

{ pkgs, ... }:

let
  rev = "a5c285a13d8a0d174a2bb397dededdc94b40f544";

  velorenAssets = pkgs.fetchFromGitLab {
    owner = "veloren";
    repo = "veloren";
    inherit rev;
    sha256 = "sha256-n1EqxxYDuDM9Cz10csf2EJoeOKaWCwXvkL7LX5IP+Yo=";
  };

  velorenSrc = pkgs.fetchgit {
    url = "https://gitlab.com/veloren/veloren";
    branchName = "yusdacra/nix-assets";
    inherit rev;
    leaveDotGit = true;
    sha256 = "sha256-moPhJ1+y5fuWtg/Osk+wO6gIErxEfbzRzFox0CTYW1Q=";
  };

  system = "x86_64-linux";

in
  import "${velorenSrc}/nix" {
    inherit system;
    assets = "${velorenAssets}/assets";
  }

Edit: Hashed and confirmed.

yusdacra commented 3 years ago

It works perfectly.

This is cleaned up from what worked for me just now. I'll update in a bit with the real hash when I've confirmed it works verbatim.

{ pkgs, ... }:

let
  rev = "a5c285a13d8a0d174a2bb397dededdc94b40f544";

  velorenAssets = pkgs.fetchFromGitLab {
    owner = "veloren";
    repo = "veloren";
    inherit rev;
    sha256 = "sha256-n1EqxxYDuDM9Cz10csf2EJoeOKaWCwXvkL7LX5IP+Yo=";
  };

  velorenSrc = pkgs.fetchgit {
    url = "https://gitlab.com/veloren/veloren";
    branchName = "yusdacra/nix-assets";
    inherit rev;
    leaveDotGit = true;
    sha256 = "sha256-moPhJ1+y5fuWtg/Osk+wO6gIErxEfbzRzFox0CTYW1Q=";
  };

  system = "x86_64-linux";

in
  import "${velorenSrc}/nix" {
    inherit system;
    assets = "${velorenAssets}/assets";
  }

Edit: Hashed and confirmed.

That's nice! I'll try and see if I can get git-lfs to work properly. I decided to not merge the workaround branch, but I'll rebase it daily so don't worry :P

lboklin commented 3 years ago

Alright; thanks to user jonringer, who figured out that the issue was with git-lfs needing some configuring, I have managed to write a working fetchgitLFS helper. This expression will create the derivation attributes veloren-voxygen and veloren-server-cli from latest master:

{ pkgs, ... }:

let
  fetchgitLFS = args:
    let
      gitconfig = pkgs.writeText "gitconfig" ''
        [filter "lfs"]
          clean = "git-lfs clean -- %f"
          process = "git-lfs filter-process"
          required = true
          smudge = "git-lfs smudge -- %f"
      '';

    in
      (pkgs.fetchgit (args // { leaveDotGit = true; })).overrideAttrs
        (oldAttrs: {
          fetcher = pkgs.writeText "git-lfs.sh" ''
            #! /usr/bin/env bash

            export HOME=$TMPDIR
            mkdir -p $HOME/.config/git/
            cp ${gitconfig} $HOME/.config/git/config

            bash ${oldAttrs.fetcher} $@
          '';

          nativeBuildInputs =
            oldAttrs.nativeBuildInputs or [] ++ [
              pkgs.git-lfs
            ];
        });

  velorenSrc = fetchgitLFS {
    url = "https://gitlab.com/veloren/veloren";
    rev = "20b45a12023739cb9c303f90092868ed526bb931";
    sha256 = "sha256-ncubJuUpGNQGUV0jrQGg0GlLbZGAiIVpvqQ3hUl2zHw=";
  };

in
  import "${velorenSrc}/nix" { system = "x86_64-linux"; }

Edit: well it seems master doesn't compile, but that's a separate issue. I'm compiling v0.8.0 now:

  {
    url = "https://gitlab.com/veloren/veloren";
    rev = "v0.8.0";
    sha256 = "sha256-+hUnIvonRH2f3hLc8+JyVTfBnCmKRTwCXsJXA3krWOU=";
  }

Edit2: v0.8.0 doesn't work but it's also missing some commits to the nix files. I'll try a recent commit from master which hopefully compiles.

Edit3: Apologies to anyone reading my comments via email for using my comments as live documents. I just don't want to spam new ones for every little update, but I also want to spare anyone that's helping from running into obsolete problems.

Anyway, here's a fully working revision (latest commit pre-merge of the campfire regen feature):

  {
    url = "https://gitlab.com/veloren/veloren";
    rev = "30563f59f3d09115b8b6a7d9fddb6cfb1f842e6a";
    sha256 = "sha256-CgFxZP1aktO4tt9UWMz9NswoBP9Y8NXakHjXQ880Mdw=";
  }
nixos-discourse commented 3 years ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-fetch-lfs-enabled-repo-with-fetchfromgithub/5890/14

lboklin commented 3 years ago

Now that it everything appears to be working well I suppose as the next step the repo-committed nix files should be locally included for an upstream PR into nixpkgs, as I seem to recall that nixpkgs doesn't accept reliance on remote nix expressions.

lboklin commented 3 years ago

The niv stuff has to be ported. crate2nix has to be in the same nixpkgs and all extensions to nixpkgs in nixpkgsMoz needs to be upstreamed or otherwise included in the PR.

yusdacra commented 3 years ago

Still need to make the game work on Wayland as that is currently broken. It fallbacks to X fine but I'd rather have it work properly.

lboklin commented 3 years ago
jonringer commented 3 years ago

I have a PR to add support for LFS here: https://github.com/NixOS/nixpkgs/pull/105998

lboklin commented 3 years ago

I have a WIP branch for adding Veloren to nixpkgs here: https://github.com/lboklin/nixpkgs/tree/veloren

lboklin commented 3 years ago

Orthogonal to this issue, but maybe of interest to some is that I've opened a PR for a flake, which at least makes life easier for us flakers. Edit: Probably good to include the link: https://gitlab.com/veloren/veloren/-/merge_requests/1607 Edit2: It's now merged, so you can add the repository as a flake now. It contains the apps & packages veloren-voxygen and veloren-server-cli.

jansol commented 3 years ago

Any news on this? Looks like the fetchgit LFS support PR got merged as well.

jonringer commented 3 years ago

it's already packaged within the upstream repo. However, I'm fine with including it in nixpkgs. We just probably wont have it built in hydra as it has sever GBs worth of assests.

jansol commented 3 years ago

Would it be feasible to split the assets off into their own package that isn't built on hydra while still building the binaries?

RPGHank commented 3 years ago

Maybe Airshipper could be packaged instead?