Switch from `mkYarnPackage` to `yarn{Config,Build}Hook`.

doronbehar commented 5 months ago

~~After https://github.com/NixOS/nixpkgs/pull/318015 will be~~ Now that https://github.com/NixOS/nixpkgs/pull/318015 is merged, we should start deprecate the usage of mkYarnPackage, because it is too complex, and hard to maintain. It also requires to vendor into Nixpkgs a package.json file for each such package (to avoid IFD, see https://github.com/NixOS/nixpkgs/issues/296856 ).

I'm opening this issue before #318015 is ready / merged to refer to this issue in a small documentation I will perform there.

The following list was generated with:

env NIXPKGS_ALLOW_INSECURE=1 nix eval --json -L --impure --expr '
  let
    pkgs = (builtins.getFlake "github:nixos/nixpkgs").legacyPackages.${builtins.currentSystem}; 
  in builtins.map (pPath: let
      p = builtins.baseNameOf (builtins.dirOf pPath);
    in if builtins.hasAttr p pkgs then
      if builtins.hasAttr "maintainers" pkgs.${p}.meta then 
        "- [ ] `${p}`: ${builtins.concatStringsSep " " (builtins.map (m: "@${m.github}") pkgs.${p}.meta.maintainers)}"
      else 
        "- [ ] `${p}`: no maintainers found"
    else
      "- [ ] `${p}`: did not found attribute automatically, file is `${pPath}`"
  ) [
    '"$(git grep -l mkYarnPackage pkgs | while read p; do echo \"$p\"; done)"'
  ]' | jq --raw-output '.[]'

[x] xplorer: @dit7ya
[ ] kuro: @LostAttractor
[ ] zammad: @n0emis @Radvendii @NetaliDev
[x] powerdns-admin: @Flakebi @zhaofengli
[x] micropad: @rhysmdnz
[x] auto-changelog: @pyrox0
[x] codefresh: @takac
[x] diagnostic-languageserver: @pyrox0
[x] dotenv-cli: @pyrox0
[x] element-call: @kilimnik
[x] get-graphql-schema: @pyrox0
[x] gramma: @pyrox0
[x] koodo-reader: @TomaSajt
[x] listmonk: @RaitoBezarius
[x] postlight-parser: @viraptor
[x] spectral-language-server: @momeemt
[x] treedome: @tengkuizdihar
[x] typst-preview: @berberman
[x] vim-language-server: @doronbehar
[x] your_spotify: @PatrickDaG
[x] your_spotify: @PatrickDaG
[ ] zitadel:
[ ] devpod: @maxbrunet
[ ] yaml-language-server:
[ ] prettierd: @NotAShelf @n3oney
[ ] react-static:
[ ] yarn2nix: no maintainers found
[ ] apache-airflow: @bhipple @gbpdt @ingenieroariel
[ ] button-card: did not found attribute automatically, file is pkgs/servers/home-assistant/custom-lovelace-modules/button-card/default.nix
[ ] multiple-entity-row: did not found attribute automatically, file is pkgs/servers/home-assistant/custom-lovelace-modules/multiple-entity-row/default.nix
[ ] zigbee2mqtt-networkmap: did not found attribute automatically, file is pkgs/servers/home-assistant/custom-lovelace-modules/zigbee2mqtt-networkmap/default.nix
[ ] pomerium: @lukegb @devusb
[ ] jellyseerr: @camillemndn
[ ] matrix-appservice-discord: @pacien
[ ] matrix-appservice-slack: @beardhatcode @chvp
[ ] matrix-hookshot: @chvp
[ ] mjolnir: @jojosch
[ ] grafana-image-renderer: @Ma27
[ ] lemmy: did not found attribute automatically, file is pkgs/servers/web-apps/lemmy/ui.nix
[ ] meshcentral: @Ma27
[ ] pgadmin: @gador
[ ] synapse-admin: @mkg20001 @Ma27
[ ] asar: @xvapx
[ ] near-cli: @Ekleog
[ ] v2raya: @LostAttractor
[ ] top-level: did not found attribute automatically, file is pkgs/top-level/all-packages.nix

gador commented 5 months ago

Is there a form of migration guide to use the new hooks?

I am using some scripted hacks to get the upstream yarn file, which also needs to be converted to the older v1 file to be used in nixpkgs. There are some dirty hacks involved, specific to pgadmin4, which would have to be used in the main derivation and not in the seperate update.sh script which right now will commit the fixed and ready to use yarn.lock file. (https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/pgadmin/update.sh#L8-L16)

Also mkYarnModules did not work for pgadmin4 which needed special treatment to build the frontend and also uses the yarn webpacker. (https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/pgadmin/default.nix#L110-L122)

Is there a way to not use the newly introduced hooks? Is this even desired? Or will I have to find some way to run my specially crafted update.sh and preBuild yarn magic with the hooks?

doronbehar commented 5 months ago

Thanks for willing to cooperate @gador . Indeed this can become complex... The new hooks I added indeed usually makes things a bit simpler, but in such a complex case as you describe, you will have to play with it yourself, and there is no migration guide. However, there are many migration examples in #318015 .

The non-trivial update scripts were discussed a bit in https://github.com/NixOS/nixpkgs/pull/318015 . I believe that with the new hooks you should be able to reach a point where you don't vendor any file (neither yarn.lock or package.json), and you should also be able to update your package with nix-update which is capable of updating the hash of the offlineCache by itself.

gador commented 5 months ago

Thanks for all your work @doronbehar ! I looked through your examples. AFAI can see I could rely on my update script and build script as it is now and not use the newly introduced hooks and everything should continue to work fine. I don't use mkYarnPackage anyway. However I will look into it if there is a way to use the hooks and maybe get rid of the packaged yarn.lock file in the future!