Installation Guide: Required Dependencies for Supported Operating Systems and Distributions

stanford-crfm / helm

Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in HEIM (https://arxiv.org/abs/2311.04287) and vision-language models in VHELM (https://arxiv.org/abs/2410.07112).

https://crfm.stanford.edu/helm

Apache License 2.0

1.95k stars 253 forks source link

Installation Guide: Required Dependencies for Supported Operating Systems and Distributions #3160

Open shakatoday opened 1 week ago

shakatoday commented 1 week ago

The installation guide currently lacks detailed information on the dependencies required for various supported operating systems and distributions. To help users set up the project smoothly, we should add a section in docs/installation.md that outlines the dependencies for each supported OS and distro (e.g., Ubuntu, Debian, macOS, etc.). This will ensure that users have a clear understanding of the prerequisites and can avoid common setup issues.

yifanmai commented 1 week ago

Are you running into an issue with a specific distro? I believe HELM should work on Ubuntu and MacOS; Debian hasn't been tested but I don't see why it wouldn't work.

There is a known issue that HELM doesn't work on Windows; see #2907.

shakatoday commented 1 week ago

Oh, sorry it's for Developer setup, so it should be docs/developer_setup.md

For Ubuntu, it's quite trivial such as build-essential and unzip. Even though they are trivial, some open-source projects include git in their setup dependencies if a distro needs it.

This issue should be a very low priority I think.

On the other hand, do you think that the setup of cuda should be mentioned in both docs/installation.md and docs/developer_setup.md?

yifanmai commented 3 days ago

That's a good point. Do you have an example of documentation from another open source project that has done something similar?

Yes, CUDA setup is quite complicated, and it's worth discussing this in the documentation.

shakatoday commented 3 days ago

Sorry, I should have given some examples. Here are three.

Electron. There are build-essential curl for Ubuntu. They also have different setup prerequisites for other distros.
Android. There are build-essential zip unzip curl for Ubuntu
OpenWrt. There are build-essential git unzip wget for Debian/Ubuntu/Mint. They also have different setup prerequisites for other distros.

Thanks for your time!