`yaml-to-dhall` is not included in `dhall-json`

[samuela]

Describe the bug

Dhall claims to include an executable yaml-to-dhall (source). I'm able to find dhall-to-yaml, but not yaml-to-dhall:

❯ nix-shell -p dhall dhall-json
these paths will be fetched (1.84 MiB download, 11.15 MiB unpacked):
  /nix/store/5c2dzhv8bz8rabqkg12wvviwwc6ybpvx-dhall-1.39.0
copying path '/nix/store/5c2dzhv8bz8rabqkg12wvviwwc6ybpvx-dhall-1.39.0' from 'https://cache.nixos.org'...

[nix-shell:~/dev/nixpkgs-upkeep]$ dhall-to-yaml --version
1.7.7

[nix-shell:~/dev/nixpkgs-upkeep]$ yaml-to-dhall
yaml-to-dhall: command not found

Steps To Reproduce

Illustrated above.

Expected behavior

A clear and concise description of what you expected to happen.

Screenshots

n/a

Additional context

n/a

Notify maintainers

@Gabriel439 (I had a hard time tracking down how dhall/dhall-json are packaged, and was not able to find a meta.maintainers, but this is my best guess 🤷)

Metadata

❯ nix-shell -p nix-info --run "nix-info -m"                                       
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.91, NixOS, 21.11 (Porcupine)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.16`
 - channels(root): `"nixos-21.11.335376.610d4ea2750, nix-ld"`
 - channels(skainswo): `"home-manager, nixpkgs-unstable-22.05pre343295.adf7f03d3bf"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[samuela]

(Side note: why are yaml related tools included in dhall-json?)

[Gabriella439]

Short answer

The yaml-to-dhall executable is now provided by a separate dhall-yaml package.

Long answer

You can find some explanation of this here: https://github.com/dhall-lang/dhall-haskell/tree/master/dhall-yaml#introduction

… but I'll also expand upon that in more detail.

The organization behind the two packages is an artifact of the Haskell ecosystem's YAML support. There are two Haskell packages that support YAML:

• The yaml package is a wrapper around the C libyaml library with a BSD 3-clause license
• The HsYAML package is a more recent pure Haskell implementation of YAML support, but with a GPLv3 license

Originally, all of the tools (dhall-to-{json,yaml} and {yaml,json}-to-dhall) were provided by the dhall-json package back when there was only the yaml package. The main reason for bundling both JSON and YAML tools within the same package is because Haskell's aeson package (which provided JSON support) and Haskell's yaml package (which provided YAML support) shared the same underlying representation, so when we implemented JSON support we got YAML support essentially for free.

However, there were a few downsides of using the yaml package:

• It depended on a C library

  … so it was incompatible with non-native backends such as GHCJS

• It reused the Value type from the aeson package

  In some ways this was a feature (again, JSON support meant YAML support for free), but in other ways this was a downside, because the Value type from the aeson package does not correctly model some YAML features

So we decided to split out a separate dhall-yaml package which would depend on the newer HsYAML package, that way the dhall-json package could keep its BSD 3-clause license.

However, the next question was how to avoid name conflicts between the dhall-to-yaml and yaml-to-dhall executables provided by the dhall-json and dhall-yaml packages.

For dhall-to-yaml, we decided to name the one from the dhall-yaml package dhall-to-yaml-ng to avoid the conflict

For yaml-to-dhall, we decided to move it entirely from the dhall-json package to the dhall-yaml package because correctly handling YAML eventually required some functionality that was only available from the HsYAML package.

There was a related discussion about that which you can find here:

https://github.com/dhall-lang/dhall-haskell/issues/1435

[samuela]

Thank you for such a detailed explanation @Gabriel439! I'll try moving forward with the dhall-yaml package.

[samuela]

Oops, I closed too soon! I'm not able to find the dhall-yaml package:

❯ nix-shell -p dhall-yaml
error: undefined variable 'dhall-yaml' at (string):1:94
(use '--show-trace' to show detailed location information)

I'm finding this to be the case even as recent as commit 5efc8ca954272c4376ac929f4c5ffefcc20551d5.

[Gabriella439]

Try haskellPackages.dhall-yaml

What's happening here is that some Haskell packages provide top-level attributes where pkgs.${foo} = pkgs.haskell.lib.justStaticExecutables pkgs.haskellPackages.${foo}. For example, that's the case for dhall-json, but nobody has wrapped dhall-yaml in a similar way, yet