EvolvingLMMs-Lab / lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval
https://lmms-lab.github.io/
Other
1.03k stars 53 forks source link

Local dataasets config didn't work #29

Open lucasjinreal opened 3 months ago

lucasjinreal commented 3 months ago

I have using this script downloaded dataset to a specific path:

import os
from datasets import load_dataset

os.environ['HF_HOME'] = '/bigdata/hub'

dataset = load_dataset("lmms-lab/CMMMU")
dataset = load_dataset("lmms-lab/MMMU")
dataset = load_dataset("lmms-lab/MMBench_CN")
dataset = load_dataset("lmms-lab/MMBench", 'en')

print(next(dataset))

and the result:

(base) ➜  hub ll datasets                                                                         
total 12K
drwxr-xr-x 2 root root 8.0K Mar 25 00:07 downloads
drwxr-xr-x 3 root root   28 Mar 24 23:38 lmms-lab___cmmmu
drwxr-xr-x 3 root root   23 Mar 25 00:07 lmms-lab___mm_bench
drwxr-xr-x 3 root root   28 Mar 25 00:04 lmms-lab___mm_bench_cn
drwxr-xr-x 3 root root   28 Mar 24 23:47 lmms-lab___mmmu

but when run eval I set envoriemnt to: /bigdata/hub for HF_HOME,

it still try load:

ConnectionError: Couldn't reach 'lmms-lab/CMMMU' on the Hub (ProxyError)

My server can not connect to internet, any one knows why/