search

OpenDrift / opendrift

Open source framework for ocean trajectory modelling
https://opendrift.github.io
GNU General Public License v2.0
245 stars 120 forks source link

Combining input files #1107

Closed IreneNA95 closed 1 year ago

IreneNA95 commented 1 year ago

Hi everyone, I hope you're all doing well. I wanted to reach out to ask for some help with my particle advection experiments using the Opendrift code and the SHYFEM native readers. My challenge is that I need to run the same program for multiple input data files, with each file corresponding to one month of SHYFEM simulation. I've attempted to add multiple SHYFEM readers for each file, but it didn't worked. I'm wondering if there's a way to combine several input files at the beginning of the experiment to get just one output. I'm hoping this approach would allow me to run a long-term simulation more efficiently. If concatenating multiple input files isn't feasible, I'm wondering if there's a way to use the results from one simulation run as the initial conditions for the next simulation run. Thank you for any help or advice you can offer.

knutfrode commented 1 year ago

Hi, Did you try to use wildcards in filename, e.g. r = shyfem.Reader(filename='shyfem_2022*.nc') Then they should me merged to a single dataset.

If you add several independent readers, there might be a gap of e.g. a few hours between them, and OpenDrift will not interpolate in time between different readers.

IreneNA95 commented 1 year ago

Hi, Thank you for your quick reply. I tried what you suggested, but I got the following error message:

opendrift_error

Based on my understanding, it seems that shyfem native readers are unable to combine multiple input files into a single dataset. Therefore, I am considering the second option. Is it possible to use the output of one simulation as the initial conditions for the subsequent simulation? Thank you again.

knutfrode commented 1 year ago

There is not automatic built-in method to start a simulation based on the output of a previous simulation. It should be possible to read output and seed again at final positions, but it would be a little awkward.

It is not clear why var.getncattr fails for you. I see line numbers do not mach latest version; did you do some local modification to shyfem.py? Can you post the output from ncdump of one of these files?

knutfrode commented 1 year ago

I found that there seem to be a bug in netCDF4 so that var.getncattr does not work when using MFDataset: https://stackoverflow.com/questions/34331005/getting-variable-attributes-from-netcdf-mfdataset-in-python/34334833#34334833

Thus I made a new commit to circumvent this. Please see if it works for you after updating OpenDrift. https://github.com/OpenDrift/opendrift/pull/1108/files

IreneNA95 commented 1 year ago

Hi, Yes, actually I made some minor modifications to the code to enable the shyfem native reader to read environmental properties such as temperature and salinity, but those lines were commented when I obtained the error.

Anyway, I tried to update the OpenDrift code as you suggested, and now it's working perfectly. Thank you for your help and quick responses to all my questions! :)

knutfrode commented 1 year ago

Very good. Thanks for contributing to improvement! Then closing this issue.