search

danielmiessler / fabric

fabric is an open-source framework for augmenting humans using AI. It provides a modular framework for solving specific problems using a crowdsourced set of AI prompts that can be used anywhere.
https://danielmiessler.com/p/fabric-origin-story
MIT License
25.57k stars 2.72k forks source link

[Bug]: Fabric Dependency Issues with gRPC and Google Cloud Libraries #747

Closed joshuafuller closed 3 months ago

joshuafuller commented 4 months ago

What happened?

What happened?

I was using the fabric package for various commands, such as changing the default model and summarizing YouTube videos. Despite multiple attempts to reinstall fabric and its dependencies, I am still encountering warning messages related to grpcio and other related libraries. The specific warnings are:

These warnings appear consistently during different fabric operations, not just when changing the default model.

Environment:

Steps Taken to Resolve the Issue:

  1. Full Uninstall and Reinstallation:

    • Uninstalled fabric using pipx: pipx uninstall fabric
    • Deleted the local fabric repository: rm -rf ~/github/fabric
    • Re-cloned the fabric repository from GitHub: git clone https://github.com/danielmiessler/fabric.git
    • Reinstalled fabric using pipx: pipx install fabric
    • Re-setup fabric as a new installation: fabric --setup
    • I performed these steps twice to ensure a clean installation.

  2. Dependency Verification:

    • Checked installed versions of key dependencies using: pipx runpip fabric show grpcio
    • Noted the installed versions:
      • grpcio==1.65.1
      • google-cloud-aiplatform==1.59.0
      • google-cloud-storage==2.17.0
    • Upgraded dependencies: pipx runpip fabric install --upgrade grpcio google-cloud-aiplatform google-cloud-storage

  3. Example Commands Producing Warnings:

    • Changing the default model:

      fabric --changeDefaultModel gpt-4o-mini
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721329796.587000   48349 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721329796.596775   48349 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.
Default model changed to gpt-4o-mini. Please restart your terminal to use it.

    • Summarizing a YouTube video (snipped for brevity):

      yt https://www.youtube.com/watch?v=_CwGUt0CpvU | summarize -s
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721330542.882595   52151 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721330542.888972   52151 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.
[SNIPPED FOR BREVITY - Normal Output]

      The command still provided the expected output after the warning messages.

Version check

Relevant log output

fabric --changeDefaultModel gpt-4o-mini
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721329796.587000   48349 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721329796.596775   48349 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.
Default model changed to gpt-4o-mini. Please restart your terminal to use it.

yt https://www.youtube.com/watch?v=_CwGUt0CpvU | summarize -s
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721330542.882595   52151 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721330542.888972   52151 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.

Relevant screenshots (optional)

WindowsTerminal_LJlCqEOP3q

manojkollam commented 4 months ago

I am getting the same type of behavior on a fresh install on ubuntu on WSL

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1721348702.029898 172323 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache I0000 00:00:1721348702.035481 172323 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.

joshuafuller commented 4 months ago

I found a temporary fix for the issue:

To mitigate the problem, run the following commands:

pipx runpip fabric uninstall grpcio
pipx runpip fabric install grpcio==1.64.0

This resolves the warning messages related to grpcio and other related libraries in my environment.

For more details on the grpc bug, see:

As for @07pepa's comments, they were completely useless and off-topic. They didn't help at all and only added noise to the conversation. If you can't contribute constructively, please refrain from commenting.

I hope this helps others facing the same issue.

manojkollam commented 4 months ago

I found a temporary fix for the issue:

To mitigate the problem, run the following commands:

pipx runpip fabric uninstall grpcio
pipx runpip fabric install grpcio==1.64.0

This resolves the warning messages related to grpcio and other related libraries in my environment.

For more details on the grpc bug, see:

As for @07pepa's comments, they were completely useless and off-topic. They didn't help at all and only added noise to the conversation. If you can't contribute constructively, please refrain from commenting.

I hope this helps others facing the same issue.

I can confirm that this fix downgrading grpcio to 1.64 is working for me as well

andeeisaacs commented 4 months ago

Also confirming that the uninstall and reinstall of grpcio worked and resolved the warning messages.

Thanks, @joshuafuller.

joshuafuller commented 4 months ago

@07pepa,

I felt compelled to share your recent emails with the community here to ensure transparency and address your concerns publicly.

Email 1:

Subject: "Never attribute to malice that which is adequately explained by stupidity."

From: Josef Havránek jhpepa@gmail.com

Hello i am 07pepa from https://github.com/danielmiessler/fabric/issues/747

i do not want to drag you throu the mud there publicly but next time please use
halons razor https://en.wikipedia.org/wiki/Hanlon%27s_razor
sometime people do stupid things after all we are all humans. and your coments are partily unvaranted and go too much into passive agresive (by your github profile i can guess its caused by stress)

also beffore your comment 2 people <3 my response.... it was clear as it was helpfull profesional and foccused and you indirectly accused me that i am not.

I sugest for you to take terapy and counseling since it is painfully obvious to me as someone who also came through some shit as well you need it.

Only think i require from you is to take honest resto-instrospective and question yoursel if you had to write such snarky passive coments.

Josef

Email 2:

Subject: heads up by 07pepa

From: Josef Havránek jhpepa@gmail.com

If i am proved right i will be pressing compensation for damages that you caused me by defamation.

i am open to go out of court way but that depends on your cooperation.

i already taken screenshots and i am just warning you not to delete anything retroactively it is a evidence and i already took screenshots

I don't know what to say other than "Thank you". This has made my day.

To commemorate your unique contributions, I’ve crafted a special certificate for you. You can find it here.

Master of Unsolicited Advice and Legal Threats

This will always be in my heart. Thank you, Peepee. Feel free to print it out and frame it. You’ve earned it.

Best regards, ❤️Josh❤️

fearby commented 4 months ago

MacOS same problem and fix doesn't work

ph3ll3r commented 4 months ago

[[Bug]: Fabric Dependency Issues with gRPC and Google Cloud Libraries] https://github.com/danielmiessler/fabric/issues/747#top)

747

I only want to use Google's models/gemini-1.5-pro and ollama models. I want exclude for the time being OpenAI (fabric default) and Claude.ai.

After a reboot, I get this error "Error: Client error '401 Unauthorized' for url 'https://api.openai.com/v1/models' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401"

I followed the suggested format a similar way described by Joshua Fuller

Full Uninstall and Reinstallation:

  1. Uninstalled fabric using pipx: pipx uninstall fabric

  2. Deleted the local fabric repository: rm -rf ~/github/fabric

  3. Re-cloned the fabric repository from GitHub: git clone https://github.com/danielmiessler/fabric.git

  4. Reinstalled fabric using pipx: pipx install fabric

  5. Re-setup fabric as a new installation: fabric --setup

    • Bypassed the openai key and claude.ai key
    • Inserted Google generative ai api key and youtube api key

  6. I performed these steps twice to ensure a clean installation.

I tested that fabric was installed properly:

  1. fabric --listmodels and found models/gemini-1.5-pro
  2. fabric --changeDefaultModel models/gemini-1.5-pro
  3. At this point the gRPC error, was still occurring but I was getting answers from fabric

Once, I reboot OpenAI took precedence and returned the error "Error: Client error '401 Unauthorized' for url 'https://api.openai.com/v1/models'". This reminds me of the "Harry Belafonte & Odetta - A Hole in the Bucket: "https://www.youtube.com/watch?v=xVAvMIhvqfk"

There needs to be a way to persist fabric --changeDefaultModel.

Also, add as many keys as you like but have a way to reset a combination of keys which are dynamically persisted but can be changed and restart the shell and happy fabric-ing to handle as an example (Fabric: This OPENSOURCE AI Framework can AUTOMATE YOUR LIFE & ANY TASK (Setup with Ollama & Groq) : "https://www.youtube.com/watch?v=Iup3-Jvkrhg" ) as a permutation of the ability to reset a combination of keys

joshuafuller commented 4 months ago

@ph3ll3r, it looks like your issue differs from the one I originally reported. You might want to start a new thread to address your specific problem.

The error I encountered affected all Fabric operations and involved warnings related to grpcio and other libraries, such as:

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721329796.596775   48349 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.

These warnings occurred during various commands, including changing the default model and summarizing YouTube videos.

Your error seems to be a 401 Unauthorized error related to OpenAI's API after a reboot, which suggests a different underlying issue.

Please refer to the photo I uploaded to see the differences in our issues.

th3f001 commented 4 months ago

I found a temporary fix for the issue:

To mitigate the problem, run the following commands:

pipx runpip fabric uninstall grpcio
pipx runpip fabric install grpcio==1.64.0

This resolves the warning messages related to grpcio and other related libraries in my environment.

For more details on the grpc bug, see:

As for @07pepa's comments, they were completely useless and off-topic. They didn't help at all and only added noise to the conversation. If you can't contribute constructively, please refrain from commenting.

I hope this helps others facing the same issue.

On MacOS Sonoma 14.5.1 the reported fix works. Thanks!

ph3ll3r commented 3 months ago

I used this and it did not work.

Dependency Verification:

Checked installed versions of key dependencies using: pipx runpip fabric show grpcio

Noted the installed versions:

th3f001 commented 20 hours ago

To mitigate the problem, run the following commands:

worked perfectly

fcalazans commented 3 months ago

Having the same problem with the initial message of the thread:

❯ fabric --listmodels
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1722717785.434341  880089 check_gcp_environment_no_op.cc:29] ALTS: Platforms other than Linux and Windows are not supported
I0000 00:00:1722717786.424544  880089 check_gcp_environment_no_op.cc:29] ALTS: Platforms other than Linux and Windows are not supported

Device: MacBook Pro 14-inch, 2023 Chip: Apple M2 Pro Memory: 16 GB macOS: 14.6 (23G80)

worked perfectly