baidu / puck

Puck is a high-performance ANN search engine
Apache License 2.0
329 stars 37 forks source link

关于ann- benchmark的运行的问题 #27

Open Autumn1998 opened 6 months ago

Autumn1998 commented 6 months ago

在成功构建docker之后,运行deep-100M的benchmark出现stat error:

E0218 09:37:10.278347     1 hierarchical_cluster_index.cpp:641] model file data/deep- 
100M.C3000_F3000_FN16_Flat.puckindex/index.dat stat error

并且后面出现了file not found的错误:

E0218 09:37:10.278366     1 py_api_wrapper.cpp:97] load index Faild
Traceback (most recent call last):
  File "run_algorithm.py", line 3, in <module>
    run_from_cmdline()
  File "/home/app/benchmark/runner.py", line 245, in run_from_cmdline
    args.private_query)
  File "/home/app/benchmark/runner.py", line 105, in run
    algo.fit(dataset)
  File "/home/app/benchmark/algorithms/puck_inmem.py", line 79, in fit
    for xblock in ds.get_dataset_iterator(bs=add_part):
  File "/home/app/benchmark/datasets.py", line 332, in get_dataset_iterator
    x = xbin_mmap(filename, dtype=self.dtype, maxn=self.nb)
  File "/home/app/benchmark/datasets.py", line 96, in xbin_mmap
    n, d = map(int, np.fromfile(fname, dtype="uint32", count=2))
FileNotFoundError: [Errno 2] No such file or directory: 'data/deep1b/base.1B.fbin.crop_nb_100000000'

奇怪的是,对应目录下是有对应的文件的:

/puck/ann-benchmarks> ls data/deep1b/base.1B.fbin.crop_nb_100000000
data/deep1b/base.1B.fbin.crop_nb_100000000

另外,能够给出deep-100M或者在benchmark页面中对应的benchmark的数据?现在只有折线图,无法进行详细的比较。

nk2014yj commented 6 months ago

1.在成功构建docker之后,运行deep-100M的benchmark出现stat error:

不加rebuild参数时候,会先调用load接口加载索引。首次执行时候,是没有索引文件的,在load加载索引时候报错该信息,是正常的。load失败,会自动调用fit接口进行训练&建库,可以忽略首次执行时候该提示。

2.并且后面出现了file not found的错误:

docker实例启动时候,会挂载data目录。data目录下文件不要使用软链,会报错(找不到文件)。