Why "SlowFS.find(URL)" returns "nil"?

twitter-archive / torch-dataset

An extensible and high performance method of reading, sampling and processing data for Torch

Apache License 2.0

76 stars 24 forks source link

Why "SlowFS.find(URL)" returns "nil"? #31

Closed chienlinhuang1116 closed 8 years ago

chienlinhuang1116 commented 8 years ago

Hi,

I have 100 files (0001.t7 , .., 0100.t7) and each is about 10 GB. To process them, I need to use SlowFS and files can be loaded dynamically as the training/sampler progresses through them. However, "SlowFS.find(URL)" returns "nil". Do you have any idea?

local dataset = Dataset('slowfs:///home/chienh/big/files')
local getBatch, numBatches = dataset.sampledBatcher({
   batchSize = 1,
   inputDims = {bsize},
   samplerKind = 'part-linear'
})

Thank you

chienlinhuang1116 commented 8 years ago

I think this question is related to What is the difference between 'viewfs://' and 'slowfs://' ?