Closed sdake closed 11 months ago
(And also implemented). I don't want to go overboard on the distribution channels. I have built brew packages before, it takes a few minutes. @MostAwesomeDude would you mind adding a PR after this merges, that updates the docs with Nix instructions?
You are free to use your time as you wish, although if you could time one of these, that would help me understand our overall distribution costs.
Thank you! -steve
To clarify, implementing brew packaging for this would take me about 30 minutes. I don't have the time to spend, and the dependencies would be difficult to deal with. I am seeking a signal on the initial cost of packaging workloads using Nix. I am super interested in the dependency cost. Flatpak seems hot too, although I don't have time to eval.
Thank you< -steve
I'm giving this a swing before lunch. I'm not using a speedrunning timer, just a wall clock.
nix flake init --template github:nix-community/poetry2nix
git add flake.nix
to enable the flake, and then nix flake show
error: getting status of '/nix/store/gapzxdimkhmgvwkyjblcf6c46qndsjps-source/poetry.lock': No such file or directory
projectDir
to point to this interior directory? In flake.nix
, I try projectDir = ./.;
default: package 'python3.10-llama2pi-0.1.0'
nix build
and maybe it's now taco time?[0/374 built, 4/38/341 copied (145.7/3239.9 MiB), 25.6/586.3 MiB DL]
So far I've been at the keyboard for just about 10min. And I'll return later.
Okay, more Nix time. This is the hard part.
ModuleNotFoundError: No module named 'hatchling'
while building einops
hatchling
for einops
nix build
, :taco:, :phone: hatchling
is a special case; see einops upstream config. Reading another few override examples...einops
is packaged; in fact, torch
is packaged. So let's just slap together a nixpkgs environment for now.pykoi
dependencies, but not at the versions that pykoi
demandsOkay, so I've been here for an hour. This is a sort of good sign, I guess; it means that nobody else has tried to package pykoi
yet. I need to take a heat break, and then I'll figure out the rest of the details.
This demonstration inferences a llama2 model on our hardware. The personal intelligence is (not particularly) intelligent, just yet.
A note to reviewers. Please take a look at the packaging model. I really do want to stay near and dear to the Python community vibe. I am not opposed to alternate packaging approaches. Any software within the runtime really does need to be packaged using whatever approach the ecosystem we integrate with typically uses. For this example, the code is Python, so we need to use "Pythonic" packaging. Brew, Nix, Portage, etc, these are all alternatives. We need to pick our Pythonic packaging and ensure any workloads use that packaging model.
My proposal for Pythonic packaging is realized in this PR. I also propose we do not hedge.
Cage match?
Alternative packaging may be a distraction.
Alternative packaging may also generate demand...
Therefore, I propose Nix (or Brew) should be documented, if the overhead vs demand generation is in our favor.