search

danielmiessler / fabric

fabric is an open-source framework for augmenting humans using AI. It provides a modular framework for solving specific problems using a crowdsourced set of AI prompts that can be used anywhere.
https://danielmiessler.com/p/fabric-origin-story
MIT License
19.88k stars 2.05k forks source link

[Bug]: pipx installation issues #359

Closed gwpl closed 3 months ago

gwpl commented 3 months ago

What happened?

form: https://github.com/danielmiessler/fabric/issues/272#issuecomment-2028275846

Our installation issues have dropped massively since switching to pipx

Still does not install on ArchLinux:

Version check

Relevant log output

$ pipx install https://github.com/danielmiessler/fabric.git
  ERROR: Cannot unpack file /tmp/pip-unpack-4mbdoai4/fabric.git (downloaded from /tmp/pip-req-build-969i88q9, content-type: text/html; charset=utf-8); cannot detect archive format
ERROR: Cannot determine archive format of /tmp/pip-req-build-969i88q9
Cannot determine package name from spec 'https://github.com/danielmiessler/fabric.git'. Check package
spec for errors.


### Relevant screenshots (optional)

_No response_
xssdoctor commented 3 months ago

Git clone first. Then go into the directory and type "pipx install ."

ksylvan commented 3 months ago 
PS C:\Users\kayvan\src\fabric> docker run --rm -it -v ${PWD):/src -w /src archlinux/archlinux:multilib-devel /bin/sh
Unable to find image 'archlinux/archlinux:multilib-devel' locally
multilib-devel: Pulling from archlinux/archlinux
ab61d9570ed9: Pull complete
4504f2a9fafb: Pull complete
Digest: sha256:cbc7aa9e3ac74726e7eab22b66c10bf88952ec38b8bf5124189fd078721d297e
Status: Downloaded newer image for archlinux/archlinux:multilib-devel
sh-5.2# pwd
/src

In my archlinux container:

sh-5.2# pacman -Syu
:: Synchronizing package databases...
 core downloading...
 extra downloading...
 multilib downloading...
:: Starting full system upgrade...
 there is nothing to do

sh-5.2# pacman -S python
resolving dependencies...
looking for conflicting packages...

Package (1)  New Version  Net Change  Download Size

core/python  3.11.8-1      74.64 MiB      13.85 MiB

Total Download Size:   13.85 MiB
Total Installed Size:  74.64 MiB

:: Proceed with installation? [Y/n] Y
:: Retrieving packages...
 python-3.11.8-1-x86_64 downloading...
checking keyring...
checking package integrity...
loading package files...
checking for file conflicts...
:: Processing package changes...
installing python...
Optional dependencies for python
    python-setuptools: for building Python packages using tooling that is usually bundled with Python
    python-pip: for installing Python packages using tooling that is usually bundled with Python
    python-pipx: for installing Python software not packaged on Arch Linux
    sqlite: for a default database integration [installed]
    mpdecimal: for decimal
    xz: for lzma [installed]
    tk: for tkinter
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
sh-5.2# python -V
Python 3.11.8

Install git and pipx:

sh-5.2# pacman -S git python-pipx
resolving dependencies...
looking for conflicting packages...

Package (11)               New Version  Net Change  Download Size

extra/perl-error           0.17029-5      0.04 MiB       0.02 MiB
extra/perl-mailtools       2.21-7         0.10 MiB       0.06 MiB
extra/perl-timedate        2.33-5         0.08 MiB       0.03 MiB
extra/python-argcomplete   3.1.1-1        0.28 MiB       0.06 MiB
extra/python-click         8.1.7-1        1.24 MiB       0.20 MiB
extra/python-distro        1.9.0-1        0.20 MiB       0.04 MiB
extra/python-packaging     23.2-1         0.53 MiB       0.10 MiB
extra/python-platformdirs  4.2.0-1        0.24 MiB       0.03 MiB
extra/python-userpath      1.9.2-1        0.09 MiB       0.02 MiB
extra/git                  2.44.0-1      26.35 MiB       6.19 MiB
extra/python-pipx          1.5.0-1        0.76 MiB       0.14 MiB

Total Download Size:    6.90 MiB
Total Installed Size:  29.91 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
 git-2.44.0-1-x86_64 downloading...
 python-click-8.1.7-1-any downloading...
 python-pipx-1.5.0-1-any downloading...
 python-packaging-23.2-1-any downloading...
 python-argcomplete-3.1.1-1-any downloading...
 perl-mailtools-2.21-7-any downloading...
 python-distro-1.9.0-1-any downloading...
 perl-timedate-2.33-5-any downloading...
 python-platformdirs-4.2.0-1-any downloading...
 python-userpath-1.9.2-1-any downloading...
 perl-error-0.17029-5-any downloading...
checking keyring...
checking package integrity...
loading package files...
checking for file conflicts...
:: Processing package changes...
installing perl-error...
installing perl-timedate...
installing perl-mailtools...
installing git...
Optional dependencies for git
    tk: gitk and git gui
    openssh: ssh transport and crypto
    perl-libwww: git svn
    perl-term-readkey: git svn and interactive.singlekey setting
    perl-io-socket-ssl: git send-email TLS support
    perl-authen-sasl: git send-email TLS support
    perl-mediawiki-api: git mediawiki support
    perl-datetime-format-iso8601: git mediawiki support
    perl-lwp-protocol-https: git mediawiki https support
    perl-cgi: gitweb (web interface) support
    python: git svn & git p4 [installed]
    subversion: git svn
    org.freedesktop.secrets: keyring credential helper
    libsecret: libsecret credential helper [installed]
installing python-click...
installing python-distro...
installing python-userpath...
installing python-argcomplete...
installing python-packaging...
installing python-platformdirs...
installing python-pipx...
:: Running post-transaction hooks...
(1/4) Creating system user accounts...
Creating group 'git' with GID 973.
Creating user 'git' (git daemon user) with UID 973 and GID 973.
(2/4) Reloading system manager configuration...
  Skipped: Current root is not booted.
(3/4) Arming ConditionNeedsUpdate...
(4/4) Warn about old perl modules

Now clone the repo and follow the instructions:

sh-5.2# git clone https://github.com/danielmiessler/fabric
Cloning into 'fabric'...
remote: Enumerating objects: 7138, done.
remote: Counting objects: 100% (1337/1337), done.
remote: Compressing objects: 100% (353/353), done.
remote: Total 7138 (delta 1036), reused 1153 (delta 948), pack-reused 5801
Receiving objects: 100% (7138/7138), 150.79 MiB | 3.43 MiB/s, done.
Resolving deltas: 100% (3290/3290), done.
Updating files: 100% (187/187), done.

sh-5.2# pipx install .
  installed package fabric 1.2.0, installed using Python 3.11.8
  These apps are now globally available
    - fabric
    - fabric-api
    - fabric-webui
    - save
    - ts
    - yt
⚠️  Note: '/root/.local/bin' is not on your PATH environment variable. These apps will not be globally accessible
    until your PATH is updated. Run `pipx ensurepath` to automatically add it, or manually modify your PATH in your
    shell's config file (i.e. ~/.bashrc).

Verify it's running (used pcman to instll ffmpeg and copied over my .env file with the API keys into the container):

sh-5.2# mv ../.env ~/.config/fabric/
sh-5.2# export OLLAMA_HOST=host.docker.internal
sh-5.2# fabric --listmodels
GPT Models:
gpt-3.5-turbo
gpt-3.5-turbo-0125
gpt-3.5-turbo-0301
gpt-3.5-turbo-0613
gpt-3.5-turbo-1106
gpt-3.5-turbo-16k
gpt-3.5-turbo-16k-0613
gpt-3.5-turbo-instruct
gpt-3.5-turbo-instruct-0914
gpt-4
gpt-4-0125-preview
gpt-4-0613
gpt-4-1106-preview
gpt-4-1106-vision-preview
gpt-4-turbo
gpt-4-turbo-2024-04-09
gpt-4-turbo-preview
gpt-4-vision-preview

Local Models:
codellama:13b
codellama:latest
dolphin-mistral:latest
dolphincoder:latest
llama2:13b
llama2:latest
llama3:latest
mistral:latest
mxbai-embed-large:latest
starcoder2:15b

Claude Models:
claude-3-opus-20240229
claude-3-sonnet-20240229
claude-3-haiku-20240307
claude-2.1
gwpl commented 3 months ago

I've found the problem!

I've forgotten that pipx install will not recognize git repo URL automatically and one needs to prefix with git+ , thereore:

# DOES NOT WORK:
# pipx install https://github.com/danielmiessler/fabric.git
# THIS WORKS:
pipx install git+https://github.com/danielmiessler/fabric.git