search

iceberg-project / Seals

This repo will be used for ICEBERG-related science: scripts, models, figures, and findings related to ICEBERG and its Seal detection use case.
MIT License
7 stars 3 forks source link

Assemble and transfer full dataset to Bridges #39

Closed bspitzbart closed 5 years ago

iparask commented 6 years ago

This issue blocks the milestone. It is also blocked by ICEBERG Administration issue 15

bspitzbart commented 6 years ago

Get Globus endpoint on SeaWulf

iparask commented 6 years ago

Brad, can you verify the data size of the whole dataset?

Based on the TXT file you provided me the size is 5TBs. I may very well make some kind of mistake, but I would like to avoid asking for space that we will not utilize, because that would reduce our credibility for later.

Thank you, Giannis

bspitzbart commented 6 years ago

Hi Giannis, Which txt file? In any case, the Seals data we have for last season is 12 TB, and it would be easy to run more.

On Tue, Nov 20, 2018 at 11:43 PM Ioannis Paraskevakos < notifications@github.com> wrote:

Brad, can you verify the data size of the whole dataset?

Based on the TXT file you provided me the size is 5TBs. I may very well make some kind of mistake, but I would like to avoid asking for space that we will not utilize, because that would reduce our credibility for later.

Thank you, Giannis

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/iceberg-project/Seals/issues/39#issuecomment-440528413, or mute the thread https://github.com/notifications/unsubscribe-auth/ABvS2yzTEacmYVPRx050fLqogNNavo9xks5uxNnwgaJpZM4YXZVM .

iparask commented 6 years ago

You gave at some point a txt file, called 2017seals_list, that has all the images and their size. I mean that one.

Thank you for verifying.