search

isetbio / isetbio

Tools for modeling image systems engineering in the human visual system front end
MIT License
100 stars 42 forks source link

Cost of storing precomputed dynamic stimuli on archiva #121

Open jamesgolden1 opened 8 years ago

jamesgolden1 commented 8 years ago

Brian and I have been working on a script to precompute the scene, oi and sensor for a drifting Gabor patch over a range of parameters like spatial frequency and orientation. We thought this could be useful in terms of pulling a precomputed stimulus down with the RDT, which is much faster than computing it on the fly.

The only issue is the cost of storing these rather big .mat files. How much is too much? This set of Gabors was 12 GB and we have five or six other types of stimuli in mind. Would it not be worth it to be pay for storage of 100 GB of precomputed stimuli? We don't know the rates on data storage, but we believe @DavidBrainard and @benjamin-heasly do.

chichilnisky commented 8 years ago

When I looked into commercial storage rates last year it was roughly $0.01/GB/month.

I think this brings up the overall issue of efficiency which I was hoping to ignore for a long time but I wonder if we can continue to do so as we do all these RGC computations over space. Particularly if users come on board, they will be less patient than we are ...

ej

On May 27, 2016, at 5:35 PM, jamesgolden1 notifications@github.com<mailto:notifications@github.com> wrote:

Brian and I have been working on a script to precompute the scene, oi and sensor for a drifting Gabor patch over a range of parameters like spatial frequency and orientation. We thought this could be useful in terms of pulling a precomputed stimulus down with the RDT, which is much faster than computing it on the fly.

The only issue is the cost of storing these rather big .mat files. How much is too much? This set of Gabors was 12 GB and we have five or six other types of stimuli in mind. Would it not be worth it to be pay for storage of 100 GB of precomputed stimuli? We don't know the rates on data storage, but we believe @DavidBrainardhttps://github.com/DavidBrainard and @benjamin-heaslyhttps://github.com/benjamin-heasly do.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/isetbio/isetbio/issues/121, or mute the threadhttps://github.com/notifications/unsubscribe/AEHBrVdbXW7s8a4JCUdlfgho0XFvq2khks5qF43ogaJpZM4Io9Dd.

wandell commented 8 years ago

That's awesome. We were worrying about 100G, which amounts to 12 dollars a year. It's a go for our stuff.

Thanks for the info.

Brian

Sent from a mobile device

On May 27, 2016, at 7:45 PM, E.J. Chichilnisky notifications@github.com<mailto:notifications@github.com> wrote:

When I looked into commercial storage rates last year it was roughly $0.01/GB/month.

I think this brings up the overall issue of efficiency which I was hoping to ignore for a long time but I wonder if we can continue to do so as we do all these RGC computations over space. Particularly if users come on board, they will be less patient than we are ...

ej

On May 27, 2016, at 5:35 PM, jamesgolden1 notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Brian and I have been working on a script to precompute the scene, oi and sensor for a drifting Gabor patch over a range of parameters like spatial frequency and orientation. We thought this could be useful in terms of pulling a precomputed stimulus down with the RDT, which is much faster than computing it on the fly.

The only issue is the cost of storing these rather big .mat files. How much is too much? This set of Gabors was 12 GB and we have five or six other types of stimuli in mind. Would it not be worth it to be pay for storage of 100 GB of precomputed stimuli? We don't know the rates on data storage, but we believe @DavidBrainardhttps://github.com/DavidBrainard and @benjamin-heaslyhttps://github.com/benjamin-heasly do.

You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/isetbio/isetbio/issues/121, or mute the threadhttps://github.com/notifications/unsubscribe/AEHBrVdbXW7s8a4JCUdlfgho0XFvq2khks5qF43ogaJpZM4Io9Dd.

You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/isetbio/isetbio/issues/121#issuecomment-222286123, or mute the threadhttps://github.com/notifications/unsubscribe/ABwIWVig3pz1SZ8xnf1mQ5OyKe8d1xyCks5qF6xggaJpZM4Io9Dd.