adding it to nixpkgs

[happysalada]

I see that you have a custom nur, feel free to ping me when you want to add this to nixpkgs.

[dustinblackman]

Hey there! Sure, I'd be down. I've never attempted submitting directly to their package manager. What's the usual process for initial submission, and submitting updates?

[happysalada]

you make a PR to https://github.com/nixos/nixpkgs the format of the commit matter a bit, people are a little sensitive about that. it should be something like "oatmeal: init at 0.5.5" you just need to find someone willing to help you merge the PR (I'm happy to do it, oatmeal looks really cool, just tag me as a reviewer). Whenever you want to make an update, you submit a new PR with the title of e.g. "oatmeal: 0.5.5 -> 0.5.6" That's about it really. Unless you are maintaining a package relied upon by many there shouldn't be any more to do. There might be some nixpkgs particular stuff, but nothing comes off the top of my head right now. I'll be happy to point those if I see anything. let me know if you are looking for more info.

[dustinblackman]

Great, thank you! Once I hit a point where I'm not releasing a version every two days, I'd be down to do this. :D

[dustinblackman]

https://github.com/NixOS/nixpkgs/pull/277622

[dustinblackman]

@onemoresuza Thanks for the assist in the nixpkgs PR!

The patch you put together is quite large, I think I can refactor to get rid of it! If I remember correctly, Nix can consume something like a JSON file with all the theme and syntax dependencies as long as there is a hash included for the archives, right? I can move those all out of build.rs and in to a separate file. And then additional in build.rs, don't attempt to download archives if environment BUILDING_ON_NIX (or anything that is common) is set.

Watcha think?

[onemoresuza]

@onemoresuza Thanks for the assist in the nixpkgs PR!

The patch you put together is quite large, I think I can refactor to get rid of it! If I remember correctly, Nix can consume something like a JSON file with all the theme and syntax dependencies as long as there is a hash included for the archives, right? I can move those all out of build.rs and in to a separate file. And then additional in build.rs, don't attempt to download archives if environment BUILDING_ON_NIX (or anything that is common) is set.

Watcha think?

You're welcome!, @dustinblackman.

Regarding the theme and syntax dependencies, yes, nix can consume data from either a JSON or TOML file. By having the urls and their respective hashes, we would be able to fetch them with nix.

[onemoresuza]

Two more things:

1. I've saw how the manpages are generated on the tools/release.sh script. To achieve the same in nix, we would have to enable the manpages feature; would there be any downsides in the binary having such feature, besides increasing its size?
2. By having a git fetched dependency on Cargo.toml --- dirs-sys ---, we must bundle the Cargo.lock file with the nix recipe, since nix is not able to fetch it. Is not possible to use one of its versions on crates.io?

[dustinblackman]

Regarding the theme and syntax dependencies, yes, nix can consume data from either a JSON or TOML file. By having the urls and their respective hashes, we would be able to fetch them with nix.

Awesome! I have a branch up that introduces an assets.toml file. My nix experience is pretty minimal, let me know if you feel it will work!

I've saw how the manpages are generated on the tools/release.sh script. To achieve the same in nix, we would have to enable the manpages feature; would there be any downsides in the binary having such feature, besides increasing its size?

Fixed in https://github.com/dustinblackman/oatmeal/commit/f5aab048841b91a3db4e749e74b4652848b7f5e1

By having a git fetched dependency on Cargo.toml --- dirs-sys ---, we must bundle the Cargo.lock file with the nix recipe, since nix is not able to fetch it. Is not possible to use one of its versions on crates.io?

I had forked it due to a licensing issue. Looking deeper it was only with the most recent version, so I've rolled back to an older one. Fixed in https://github.com/dustinblackman/oatmeal/commit/5c2359be48b9ed1c2b45b936063eddd867aa578f

[onemoresuza]

Tested the changes on my branch and the package compiles and generates the manpages and the shell completions --- the Cargo.lock file is still bundled because I'm applying your PR as patch; once it's merged, nix will be able to use the Cargo.lock file from the project.

With the assets.toml solution, we can automate downloading the themes and the syntaxes. Although that file must still be bundled on nixpkgs, there's no problem, since a passthru.updateScript may be able to fetch it when updating the package.

Regarding the themes and syntaxes change, are these the expected directory trees for the OATMEAL_BUILD_DOWNLOADED_*_DIR directories?

[user@hostname:~] $ ls -lh $OATMEAL_BUILD_DOWNLOADED_THEMES_DIR
dr-xr-xr-x 5 nixbld nixbld 160 Dec 31 14:37 base16-textmate

[user@hostname:~] $ ls -lh $OATMEAL_BUILD_DOWNLOADED_SYNTAXES_DIR
dr-xr-xr-x  3 nixbld nixbld  220 Dec 31 14:37 GraphQL-SublimeText3
dr-xr-xr-x 49 nixbld nixbld 1.1K Dec 31 14:37 Packages
dr-xr-xr-x  4 nixbld nixbld  340 Dec 31 14:37 Terraform.tmLanguage
dr-xr-xr-x 10 nixbld nixbld  420 Dec 31 14:37 bat
dr-xr-xr-x  4 nixbld nixbld  240 Dec 31 14:37 elixir-sublime-syntax
dr-xr-xr-x  4 nixbld nixbld  240 Dec 31 14:37 protobuf-syntax-highlighting
dr-xr-xr-x  4 nixbld nixbld  120 Dec 31 14:37 sublime-text-gleam
dr-xr-xr-x 11 nixbld nixbld  380 Dec 31 14:37 sublime-zig-language
dr-xr-xr-x  2 nixbld nixbld  200 Dec 31 14:37 sublime_toml_highlighting

[dustinblackman]

Regarding the themes and syntaxes change, are these the expected directory trees for the OATMEAL_BUILDDOWNLOADED*_DIR directories?

Yep! Though they matter less than they did before as I walk the directory tree to find assets.

Although that file must still be bundled on nixpkgs

Does it? I've been working on removing any bundled assets in to nixpkgs, and so far I've gotten this to download all that's needed, but it fails due to Nixpkgs having an old version of Rust (1.69.0), where Oatmeal expects 1.75.0. How did you overcome it?

{ lib
, stdenv
, fetchFromGitHub
, rustPlatform
, makeRustPlatform
, installShellFiles
}:
let
  version = "0.12.1";

  oatmeal = fetchFromGitHub {
    name = "oatmeal";
    owner = "dustinblackman";
    repo = "oatmeal";
    rev = "622fd5e7d374c9b3434efceeaf3b47c27664523a"; #todo
    hash = "sha256-Ps7b2ZKE9K8HxLB+4DZporIpF0JKSuOBjqS95SoLVWY=";
  };

  assets = lib.importTOML(oatmeal + "/assets.toml");

  arrayToSrc = arr: lib.forEach arr (e:
    fetchFromGitHub {
      inherit (e) rev repo owner;
      hash = e.nix-hash;
      name = e.repo;
    }
  );

  syntaxes = arrayToSrc assets.syntaxes;
  themes = arrayToSrc assets.themes;

  syntaxDirs = lib.concatStringsSep " " (lib.forEach (syntaxes) (e: "${e}"));
  themeDirs = lib.concatStringsSep " " (lib.forEach (themes) (e: "${e}"));

  buildSyntaxes = "${srcs.oatmeal.name}/.cache/syntaxes";
  buildThemes = "${srcs.oatmeal.name}/.cache/themes";
in
rustPlatform.buildRustPackage {
  pname = "oatmeal";
  inherit version;
  srcs = [ oatmeal ] ++ syntaxes ++ themes;
  sourceRoot = oatmeal.name;

  cargoLock.lockFile = oatmeal + "/Cargo.lock";

  nativeBuildInputs = [ installShellFiles ];

  env = {
    OATMEAL_BUILD_DOWNLOADED_SYNTAXES_DIR = buildSyntaxes;
    OATMEAL_BUILD_DOWNLOADED_THEMES_DIR = buildThemes;
    VERGEN_IDEMPOTENT = 1;
  };

  preBuild = ''
    mkdir -p ${buildSyntaxes} ${buildThemes}
    cp -r ${syntaxDirs} ${buildSyntaxes}
    cp -r ${themeDirs} ${buildThemes}
  '';

  postInstall = lib.optionalString (stdenv.hostPlatform.canExecute stdenv.buildPlatform) ''
    $out/bin/oatmeal manpages 1>oatmeal.1
    installManPage oatmeal.1

    installShellCompletion --cmd oatmeal \
      --bash <($out/bin/oatmeal completions -s bash) \
      --fish <($out/bin/oatmeal completions -s fish) \
      --zsh <($out/bin/oatmeal completions -s zsh)
  '';

  meta = {
    description = "Terminal UI to chat with large language models (LLM)";
    longDescription = ''
      Oatmeal is a terminal UI chat application that speaks with LLMs, complete with
      slash commands and fancy chat bubbles. It features agnostic backends to allow
      switching between the powerhouse of ChatGPT, or keeping things private with
      Ollama. While Oatmeal works great as a stand alone terminal application, it
      works even better paired with an editor like Neovim!
    '';
    homepage = "https://github.com/dustinblackman/oatmeal/";
    changelog = "https://github.com/dustinblackman/oatmeal/blob/main/CHANGELOG.md";
    downloadPage = "https://github.com/dustinblackman/oatmeal/releases/tag/v${version}";
    license = lib.licenses.mit;
    maintainers = with lib.maintainers; [ dustinblackman ];
    mainProgram = "oatmeal";
  };
}

[onemoresuza]

Although that file must still be bundled on nixpkgs

Does it? I'll been working on removing any bundled assets in to nixpkgs

I'm believe so. I don't know of way of using one of the sources --- in this case, the oatmeal repo --- to gather the info to download other sources --- the themes and syntaxes from the assets.toml file.

so far I've gotten this to download all that's needed, but it fails due to Nixpkgs having an old version of Rust (1.69.0), where Oatmeal expects 1.75.0. How did you overcome it?

That's odd. I've checked your branch and the Rust version is 1.74.0, the same one against which I've tested earlier. How are you building the package? By "cding" into your fork of the repo and running nix-build -A oatmeal?