ethrane / transients

Discussion for the Time Domain and Multi-Messenger Astrophysics Group (1.3)
0 stars 0 forks source link

Access to high-performance computing facilities/supercomputer time (proposed infrastructure priority) #9

Open ethrane opened 8 months ago

ethrane commented 8 months ago

Access to high-performance computing facilities/supercomputer time is required for simulations, multi-wavelength data handling and analysis, and data storage. This would then support the theoretical modelling of the transient and multi-messenger phenomena and the processing of the vast amount of observational data that is associated with these endeavours.

(This is an example posted with @kauchettl.)

adellej commented 8 months ago

I think this point is important both for observations (eg. processing of SKA data and SKA precursor data - the Pawsey supercomputing centre has caused a whole host of delays and issues with processing of observational data that really need to be ironed out before SKA comes online) as well as theoretical simulations.

marcinglowacki commented 6 months ago

Agreed, HPC is hugely important to ensure our observations can be processed and analysed, and running/hosting next-gen simulations

ethrane commented 6 months ago

@adellej, reliability issues with Pawsey came up at the Melbourne-area town hall as well. I'm not sure what the solution is, but many people are expressing concern.

kwwette commented 6 months ago

This should also specifically mention high-throughput/grid-based computing. Offline data processing - certainly for GWs, maybe in other fields - doesn't necessary need a high-performance cluster with fast inter-connect, but can be farmed out as single independent jobs to any cluster, provided there is efficient access to the required data. GW data processing is also increasingly making use of the Open Science Grid. It would be fantastic to have a similar facility in Australia, where one could submit jobs over the grid to any cluster with spare cycles. An alternative might be to buy into the existing Open Science Grid, or at least leverage their technologies. Elements of what's needed are already been done by OzSTAR and through the GW data centre.