search

OpenDrift / opendrift

Open source framework for ocean trajectory modelling
https://opendrift.github.io
GNU General Public License v2.0
245 stars 120 forks source link

Issue while seeding the elements at south hemisphere #418

Closed GuiCruz closed 3 years ago

GuiCruz commented 3 years ago

Hello. I'm trying to use the Ocean Drift module with a netcdf file that I created. However, when I try to use the command o.seed_elements the program says that all particles are outside the domain. My domain is at the south hemisphere and Lat and Lon values are negative, but the error message says that is at the north ( All 100 particles (-32.11--32.11E, -52.14--52.14N) are outside domain of)

The variables in my netcdf file are as follows:

{'latitude_longitude': <class 'netCDF4._netCDF4.Variable'>
 int32 latitude_longitude()
     grid_mapping_name: latitude_longitude
 unlimited dimensions: 
 current shape = ()
 filling on, default _FillValue of -2147483647 used,
 'X': <class 'netCDF4._netCDF4.Variable'>
 float32 X(X)
     units: m
     axis: X
     standard_name: projection_x_coordinate
     long_name: x-coordinate in projected coordinate system
 unlimited dimensions: 
 current shape = (130,)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'Y': <class 'netCDF4._netCDF4.Variable'>
 float32 Y(Y)
     units: m
     axis: Y
     standard_name: projection_y_coordinate
     long_name: y-coordinate in projected coordinate system
 unlimited dimensions: 
 current shape = (558,)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'depth': <class 'netCDF4._netCDF4.Variable'>
 float32 depth(depth)
     units: m
     axis: Z
     standard_name: depth
 unlimited dimensions: 
 current shape = (1,)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'time': <class 'netCDF4._netCDF4.Variable'>
 int32 time(time)
     long_name: time since initialization
     units: seconds since 2020-07-01 00:00:00
     calendar: gregorian
     field: time, scalar, series
     axis: T
     standard_name: time
 unlimited dimensions: 
 current shape = (97,)
 filling on, default _FillValue of -2147483647 used,
 'u': <class 'netCDF4._netCDF4.Variable'>
 float32 u(time, depth, Y, X)
     units: meter second-1
     standard_name: x_sea_water_velocity
     long_name: Sea water x velocity
     coordinates: lon lat
 unlimited dimensions: 
 current shape = (97, 1, 558, 130)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'v': <class 'netCDF4._netCDF4.Variable'>
 float32 v(time, depth, Y, X)
     units: meter second-1
     standard_name: y_sea_water_velocity
     long_name: Sea water y velocity
     coordinates: lon lat
 unlimited dimensions: 
 current shape = (97, 1, 558, 130)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'lon': <class 'netCDF4._netCDF4.Variable'>
 float64 lon(Y, X)
     units: 
     standard_name: lon
     long_name: lon
 unlimited dimensions: 
 current shape = (558, 130)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'lat': <class 'netCDF4._netCDF4.Variable'>
 float64 lat(Y, X)
     units: 
     standard_name: lat
     long_name: lat
 unlimited dimensions: 
 current shape = (558, 130)
 filling on, default _FillValue of 9.969209968386869e+36 used}
o = OceanDrift(loglevel=0)
reader = reader_netCDF_CF_unstructured.Reader('./OD_25x100_Wind05ms_opendrift.nc')
o.add_reader([reader])
o.seed_elements(lat=-52.13804129, lon=-32.11012351, number=100,time=reader.start_time)
o.run()

Exception: 01:20:18 INFO: All 100 particles (-32.11--32.11E, -52.14--52.14N) are outside domain of ./OD_25x100_Wind05ms_opendrift.nc (-32.31--32.31E, -52.19--52.18N) 01:20:18 DEBUG: Traceback (most recent call last): File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\models\basemodel.py", line 990, in get_environment z[missing_indices], self.use_block, self.proj) File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 478, in get_variables_interpolated lat.max(), self.name, self.coverage_string())) ValueError: All 100 particles (-32.11--32.11E, -52.14--52.14N) are outside domain of ./OD_25x100_Wind05ms_opendrift.nc (-32.31--32.31E, -52.19--52.18N)

GuiCruz commented 3 years ago

I've managed to solve the problem with the location of the particles, but now I am getting a diferent type of problem with the shape of the variables.

Thanks for your help!

03:33:07 DEBUG: Calculation SRS set to: None
03:33:07 INFO: OpenDriftSimulation initialised (version 1.3.3)
03:33:07 INFO: Opening dataset: ./OD_25x100_Wind05ms_opendrift.nc
03:33:07 INFO: Opening file with Dataset
03:33:07 DEBUG: Finding coordinate variables.
03:33:07 DEBUG: Calculation SRS set to: +proj=longlat +ellps=WGS84 +no_defs
03:33:07 DEBUG: Using srs for common grid: +proj=latlong
03:33:07 DEBUG: Added reader ./OD_25x100_Wind05ms_opendrift.nc
03:33:07 INFO: Setting config: drift:vertical_advection -> False
03:33:07 INFO: Setting config: drift:vertical_mixing -> False
03:33:07 DEBUG: 
------------------------------------------------------
Software and hardware:
  OpenDrift version 1.3.3
  Platform: Windows, 10
  unknown GB memory
  4 processors (Intel64 Family 6 Model 142 Stepping 9, GenuineIntel)
  NumPy version 1.19.2
  SciPy version 1.5.2
  Matplotlib version 3.1.3
  NetCDF4 version 1.5.4
  Python version 3.7.8 | packaged by conda-forge | (default, Jul 31 2020, 01:53:57) [MSC v.1916 64 bit (AMD64)]
------------------------------------------------------

03:33:07 INFO: Config validation OK
03:33:07 DEBUG: No output file is specified, neglecting export_buffer_length
03:33:07 INFO: Fallback values will be used for the following variables which have no readers: 
03:33:07 INFO:  x_wind: 0.000000
03:33:07 INFO:  y_wind: 0.000000
03:33:07 INFO:  upward_sea_water_velocity: 0.000000
03:33:07 INFO:  ocean_vertical_diffusivity: 0.000000
03:33:07 INFO:  sea_surface_wave_significant_height: 0.000000
03:33:07 INFO:  sea_surface_wave_stokes_drift_x_velocity: 0.000000
03:33:07 INFO:  sea_surface_wave_stokes_drift_y_velocity: 0.000000
03:33:07 INFO:  sea_surface_wave_period_at_variance_spectral_density_maximum: 0.000000
03:33:07 INFO:  sea_surface_wave_mean_period_from_variance_spectral_density_second_frequency_moment: 0.000000
03:33:07 INFO:  surface_downward_x_stress: 0.000000
03:33:07 INFO:  surface_downward_y_stress: 0.000000
03:33:07 INFO:  turbulent_kinetic_energy: 0.000000
03:33:07 INFO:  turbulent_generic_length_scale: 0.000000
03:33:07 INFO:  sea_floor_depth_below_sea_level: 10000.000000
03:33:07 INFO: Duration, steps or end time not specified, running until end of first reader: 2020-07-05 00:00:00
03:33:07 DEBUG: Preparing readers for simulation coverage ([-37.390489399670834, -55.29800371737093, -27.23355692601276, -49.07056088834196]) and time (2020-07-01 00:00:00 to 2020-07-05 00:00:00)
03:33:07 DEBUG:     Preparing ./OD_25x100_Wind05ms_opendrift.nc
03:33:07 INFO: Adding a dynamical landmask with max. priority based on assumed maximum speed of 1 m/s. Adding a customised landmask may be faster...
ERROR:root:could not verify read permissions for group and others on landmask.
Traceback (most recent call last):
  File "C:\Users\Cliente\anaconda3\envs\opendrift\lib\site-packages\opendrift_landmask_data-0.6-py3.8.egg\opendrift_landmask_data\mask.py", line 77, in __check_permissions__
    if not os.stat(self.lockf).st_mode & 0o777 == 0o777:
FileNotFoundError: [WinError 2] O sistema não pode encontrar o arquivo especificado: 'C:\\Users\\Public\\Documents\\Wondershare\\CreatorTemp\\landmask\\.mask.dat.lock'
03:33:10 DEBUG: Added reader global_landmask
03:33:10 INFO: Using existing reader for land_binary_mask
03:33:10 DEBUG: ----------------------------------------
03:33:10 DEBUG: Variable group ['land_binary_mask']
03:33:10 DEBUG: ----------------------------------------
03:33:10 DEBUG: Calling reader global_landmask
03:33:10 DEBUG: ----------------------------------------
03:33:10 DEBUG: Data needed for 100 elements
03:33:10 DEBUG: Reader time:
        None (before)
        None (after)
03:33:10 DEBUG: Fetching variables from global_landmask
03:33:10 DEBUG: Fetched env-before
03:33:10 DEBUG: No time interpolation needed - right on time.
03:33:10 DEBUG: Obtained data for all elements.
03:33:10 DEBUG: ---------------------------------------
03:33:10 DEBUG: Finished processing all variable groups
03:33:10 DEBUG: ------------ SUMMARY -------------
03:33:10 DEBUG:     land_binary_mask: 0 (min) 0 (max)
03:33:10 DEBUG: ---------------------------------
03:33:10 DEBUG:         0 active elements
03:33:10 INFO: All points are in ocean
03:33:10 DEBUG: to be seeded: 100, already seeded 0
03:33:10 DEBUG: Released 100 new elements.
03:33:10 DEBUG: ======================================================================
03:33:10 INFO: 2020-07-01 00:00:00 - step 1 of 96 - 100 active elements (0 deactivated)
03:33:10 DEBUG: 0 elements scheduled.
03:33:10 DEBUG: ======================================================================
03:33:10 DEBUG: ----------------------------------------
03:33:10 DEBUG: Variable group ['x_sea_water_velocity', 'y_sea_water_velocity']
03:33:10 DEBUG: ----------------------------------------
03:33:10 DEBUG: Calling reader ./OD_25x100_Wind05ms_opendrift.nc
03:33:10 DEBUG: ----------------------------------------
03:33:10 DEBUG: Data needed for 100 elements
03:33:10 DEBUG: Reader time:
        2020-07-01 00:00:00 (before)
        2020-07-01 01:00:00 (after)
03:33:10 DEBUG: Fetching variables from ./OD_25x100_Wind05ms_opendrift.nc
03:33:10 INFO: ========================
03:33:10 INFO: Exception:
03:33:10 INFO: operands could not be broadcast together with shapes (130,) (558,) 
03:33:10 DEBUG: Traceback (most recent call last):
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\models\basemodel.py", line 990, in get_environment
    z[missing_indices], self.use_block, self.proj)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 545, in get_variables_interpolated
    block=block)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 374, in _get_variables
    env = self.get_variables_derived(variables, time, x, y, z, block)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 350, in get_variables_derived
    env = self.get_variables(variables, *args, **kwargs)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\reader_netCDF_CF_unstructured.py", line 210, in get_variables
    (self.lat > latmin) &
ValueError: operands could not be broadcast together with shapes (130,) (558,)
knutfrode commented 3 years ago

It seems that the following file is not found, for some reason: C:\\Users\\Public\\Documents\\Wondershare\\CreatorTemp\\landmask\\.mask.dat.lock

Could you try to delete this folder and try agan C:\\Users\\Public\\Documents\\Wondershare\\CreatorTemp\\landmask ?

Otherwise, perhaps @gauteh has a suggestion here?

GuiCruz commented 3 years ago

@knutfrode Thanks for the suggestion.

I deleted the folder, but the problems with the shape of the operands still persist

04:46:57 INFO: OpenDriftSimulation initialised (version 1.3.3)
04:46:57 INFO: Opening dataset: ./OD_25x100_Wind05ms_opendrift.nc
04:46:57 INFO: Opening file with Dataset
04:46:57 DEBUG: Finding coordinate variables.
04:46:57 DEBUG: Calculation SRS set to: +proj=longlat +ellps=WGS84 +no_defs
04:46:57 DEBUG: Using srs for common grid: +proj=latlong
04:46:57 DEBUG: Added reader ./OD_25x100_Wind05ms_opendrift.nc
04:46:57 DEBUG: 
------------------------------------------------------
Software and hardware:
  OpenDrift version 1.3.3
  Platform: Windows, 10
  unknown GB memory
  4 processors (Intel64 Family 6 Model 142 Stepping 9, GenuineIntel)
  NumPy version 1.19.2
  SciPy version 1.5.2
  Matplotlib version 3.1.3
  NetCDF4 version 1.5.4
  Python version 3.7.8 | packaged by conda-forge | (default, Jul 31 2020, 01:53:57) [MSC v.1916 64 bit (AMD64)]
------------------------------------------------------

04:46:57 INFO: Config validation OK
04:46:57 DEBUG: No output file is specified, neglecting export_buffer_length
04:46:57 INFO: Fallback values will be used for the following variables which have no readers: 
04:46:57 INFO:  x_wind: 0.000000
04:46:57 INFO:  y_wind: 0.000000
04:46:57 INFO:  upward_sea_water_velocity: 0.000000
04:46:57 INFO:  ocean_vertical_diffusivity: 0.000000
04:46:57 INFO:  sea_surface_wave_significant_height: 0.000000
04:46:57 INFO:  sea_surface_wave_stokes_drift_x_velocity: 0.000000
04:46:57 INFO:  sea_surface_wave_stokes_drift_y_velocity: 0.000000
04:46:57 INFO:  sea_surface_wave_period_at_variance_spectral_density_maximum: 0.000000
04:46:57 INFO:  sea_surface_wave_mean_period_from_variance_spectral_density_second_frequency_moment: 0.000000
04:46:57 INFO:  surface_downward_x_stress: 0.000000
04:46:57 INFO:  surface_downward_y_stress: 0.000000
04:46:57 INFO:  turbulent_kinetic_energy: 0.000000
04:46:57 INFO:  turbulent_generic_length_scale: 0.000000
04:46:57 INFO:  sea_floor_depth_below_sea_level: 10000.000000
04:46:57 INFO: Duration, steps or end time not specified, running until end of first reader: 2020-07-05 00:00:00
04:46:57 DEBUG: Preparing readers for simulation coverage ([-37.427359437311885, -55.3177829226932, -27.200780058537724, -49.049358800939615]) and time (2020-07-01 00:00:00 to 2020-07-05 00:00:00)
04:46:57 DEBUG:     Preparing ./OD_25x100_Wind05ms_opendrift.nc
04:46:57 INFO: Adding a dynamical landmask with max. priority based on assumed maximum speed of 1 m/s. Adding a customised landmask may be faster...
ERROR:root:fcntl not available on this platform, concurrent generations of landmask (different threads or processes) might cause failing landmask generation. Make sure only one instance of landmask is running on the system the first time.
04:47:17 DEBUG: Added reader global_landmask
04:47:17 INFO: Using existing reader for land_binary_mask
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Variable group ['land_binary_mask']
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Calling reader global_landmask
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Data needed for 100 elements
04:47:17 DEBUG: Reader time:
        None (before)
        None (after)
04:47:17 DEBUG: Fetching variables from global_landmask
04:47:17 DEBUG: Fetched env-before
04:47:17 DEBUG: No time interpolation needed - right on time.
04:47:17 DEBUG: Obtained data for all elements.
04:47:17 DEBUG: ---------------------------------------
04:47:17 DEBUG: Finished processing all variable groups
04:47:17 DEBUG: ------------ SUMMARY -------------
04:47:17 DEBUG:     land_binary_mask: 0 (min) 0 (max)
04:47:17 DEBUG: ---------------------------------
04:47:17 DEBUG:         0 active elements
04:47:17 INFO: All points are in ocean
04:47:17 DEBUG: to be seeded: 100, already seeded 0
04:47:17 DEBUG: Released 100 new elements.
04:47:17 DEBUG: ======================================================================
04:47:17 INFO: 2020-07-01 00:00:00 - step 1 of 96 - 100 active elements (0 deactivated)
04:47:17 DEBUG: 0 elements scheduled.
04:47:17 DEBUG: ======================================================================
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Variable group ['x_sea_water_velocity', 'y_sea_water_velocity']
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Calling reader ./OD_25x100_Wind05ms_opendrift.nc
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Data needed for 100 elements
04:47:17 DEBUG: Reader time:
        2020-07-01 00:00:00 (before)
        2020-07-01 01:00:00 (after)
04:47:17 DEBUG: Fetching variables from ./OD_25x100_Wind05ms_opendrift.nc
04:47:17 INFO: ========================
04:47:17 INFO: Exception:
04:47:17 INFO: operands could not be broadcast together with shapes (259,) (1116,) 
04:47:17 DEBUG: Traceback (most recent call last):
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\models\basemodel.py", line 990, in get_environment
    z[missing_indices], self.use_block, self.proj)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 545, in get_variables_interpolated
    block=block)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 374, in _get_variables
    env = self.get_variables_derived(variables, time, x, y, z, block)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 350, in get_variables_derived
    env = self.get_variables(variables, *args, **kwargs)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\reader_netCDF_CF_unstructured.py", line 210, in get_variables
    (self.lat > latmin) &
ValueError: operands could not be broadcast together with shapes (259,) (1116,) 

04:47:17 INFO: ========================
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Variable group ['land_binary_mask']
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Calling reader global_landmask
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Data needed for 100 elements
04:47:17 DEBUG: Reader time:
        None (before)
        None (after)
04:47:17 DEBUG: Fetching variables from global_landmask
04:47:17 DEBUG: Fetched env-before
04:47:17 DEBUG: No time interpolation needed - right on time.
04:47:17 DEBUG: Obtained data for all elements.
04:47:17 DEBUG: ---------------------------------------
04:47:17 DEBUG: Finished processing all variable groups
04:47:17 DEBUG: Creating empty dictionary for profiles not profided by any reader: ['ocean_vertical_diffusivity']
04:47:17 DEBUG:       Using fallback value 0 for ocean_vertical_diffusivity for all profiles
04:47:17 DEBUG: ------------ SUMMARY -------------
04:47:17 DEBUG:     x_sea_water_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     y_sea_water_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     x_wind: 0 (min) 0 (max)
04:47:17 DEBUG:     y_wind: 0 (min) 0 (max)
04:47:17 DEBUG:     upward_sea_water_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     ocean_vertical_diffusivity: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_significant_height: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_stokes_drift_x_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_stokes_drift_y_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_period_at_variance_spectral_density_maximum: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_mean_period_from_variance_spectral_density_second_frequency_moment: 0 (min) 0 (max)
04:47:17 DEBUG:     surface_downward_x_stress: 0 (min) 0 (max)
04:47:17 DEBUG:     surface_downward_y_stress: 0 (min) 0 (max)
04:47:17 DEBUG:     turbulent_kinetic_energy: 0 (min) 0 (max)
04:47:17 DEBUG:     turbulent_generic_length_scale: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_floor_depth_below_sea_level: 10000 (min) 10000 (max)
04:47:17 DEBUG:     land_binary_mask: 0 (min) 0 (max)
04:47:17 DEBUG: ---------------------------------
04:47:17 DEBUG:         100 active elements
04:47:17 DEBUG:         -52.20427 <- latitude  -> -52.162872
04:47:17 DEBUG:         -32.349327 <- longitude -> -32.278812
04:47:17 DEBUG:         z = 0.0
04:47:17 DEBUG: ---------------------------------
04:47:17 DEBUG: Calculating Hs from wind, min: 0.000000, mean: 0.000000, max: 0.000000
04:47:17 DEBUG: Calculating wave period Tm02 from wind
04:47:17 DEBUG:    min: 1.256637, mean: 1.256637, max: 1.256637
04:47:17 DEBUG: Calculating wave period from wind, min: 1.256637, mean: 1.256637, max: 1.256637
04:47:17 DEBUG: No elements to deactivate
04:47:17 DEBUG: Calling OceanDrift.update()
04:47:17 DEBUG: No wind for wind-sheared ocean drift
04:47:17 DEBUG: No Stokes drift velocity available
04:47:17 DEBUG: No vertical advection for elements at surface
04:47:17 DEBUG: Horizontal diffusivity is 0, no random walk.
04:47:17 DEBUG: 100 active elements (0 deactivated)
04:47:17 DEBUG: to be seeded: 0, already seeded 100
04:47:17 DEBUG: ======================================================================
04:47:17 INFO: 2020-07-01 01:00:00 - step 2 of 96 - 100 active elements (0 deactivated)
04:47:17 DEBUG: 0 elements scheduled.
04:47:17 DEBUG: ======================================================================
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Variable group ['x_sea_water_velocity', 'y_sea_water_velocity']
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Calling reader ./OD_25x100_Wind05ms_opendrift.nc
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Data needed for 100 elements
04:47:17 DEBUG: Reader time:
        2020-07-01 01:00:00 (before)
        2020-07-01 02:00:00 (after)
04:47:17 DEBUG: Fetching variables from ./OD_25x100_Wind05ms_opendrift.nc
04:47:17 INFO: ========================
04:47:17 INFO: Exception:
04:47:17 INFO: operands could not be broadcast together with shapes (259,) (1116,) 
04:47:17 DEBUG: Traceback (most recent call last):
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\models\basemodel.py", line 990, in get_environment
    z[missing_indices], self.use_block, self.proj)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 545, in get_variables_interpolated
    block=block)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 374, in _get_variables
    env = self.get_variables_derived(variables, time, x, y, z, block)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 350, in get_variables_derived
    env = self.get_variables(variables, *args, **kwargs)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\reader_netCDF_CF_unstructured.py", line 210, in get_variables
    (self.lat > latmin) &
ValueError: operands could not be broadcast together with shapes (259,) (1116,) 

04:47:17 INFO: ========================
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Variable group ['land_binary_mask']
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Calling reader global_landmask
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Data needed for 100 elements
04:47:17 DEBUG: Reader time:
        None (before)
        None (after)
04:47:17 DEBUG: Fetching variables from global_landmask
04:47:17 DEBUG: Fetched env-before
04:47:17 DEBUG: No time interpolation needed - right on time.
04:47:17 DEBUG: Obtained data for all elements.
04:47:17 DEBUG: ---------------------------------------
04:47:17 DEBUG: Finished processing all variable groups
04:47:17 DEBUG: Creating empty dictionary for profiles not profided by any reader: ['ocean_vertical_diffusivity']
04:47:17 DEBUG:       Using fallback value 0 for ocean_vertical_diffusivity for all profiles
04:47:17 DEBUG: ------------ SUMMARY -------------
04:47:17 DEBUG:     x_sea_water_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     y_sea_water_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     x_wind: 0 (min) 0 (max)
04:47:17 DEBUG:     y_wind: 0 (min) 0 (max)
04:47:17 DEBUG:     upward_sea_water_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     ocean_vertical_diffusivity: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_significant_height: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_stokes_drift_x_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_stokes_drift_y_velocity: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_period_at_variance_spectral_density_maximum: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_surface_wave_mean_period_from_variance_spectral_density_second_frequency_moment: 0 (min) 0 (max)
04:47:17 DEBUG:     surface_downward_x_stress: 0 (min) 0 (max)
04:47:17 DEBUG:     surface_downward_y_stress: 0 (min) 0 (max)
04:47:17 DEBUG:     turbulent_kinetic_energy: 0 (min) 0 (max)
04:47:17 DEBUG:     turbulent_generic_length_scale: 0 (min) 0 (max)
04:47:17 DEBUG:     sea_floor_depth_below_sea_level: 10000 (min) 10000 (max)
04:47:17 DEBUG:     land_binary_mask: 0 (min) 0 (max)
04:47:17 DEBUG: ---------------------------------
04:47:17 DEBUG:         100 active elements
04:47:17 DEBUG:         -52.20426940917969 <- latitude  -> -52.162872314453125
04:47:17 DEBUG:         -32.349327087402344 <- longitude -> -32.278812408447266
04:47:17 DEBUG:         z = 0.0
04:47:17 DEBUG: ---------------------------------
04:47:17 DEBUG: Calculating Hs from wind, min: 0.000000, mean: 0.000000, max: 0.000000
04:47:17 DEBUG: Calculating wave period Tm02 from wind
04:47:17 DEBUG:    min: 1.256637, mean: 1.256637, max: 1.256637
04:47:17 DEBUG: Calculating wave period from wind, min: 1.256637, mean: 1.256637, max: 1.256637
04:47:17 DEBUG: No elements to deactivate
04:47:17 DEBUG: Calling OceanDrift.update()
04:47:17 DEBUG: No wind for wind-sheared ocean drift
04:47:17 DEBUG: No Stokes drift velocity available
04:47:17 DEBUG: No vertical advection for elements at surface
04:47:17 DEBUG: Horizontal diffusivity is 0, no random walk.
04:47:17 DEBUG: 100 active elements (0 deactivated)
04:47:17 DEBUG: to be seeded: 0, already seeded 100
04:47:17 DEBUG: ======================================================================
04:47:17 INFO: 2020-07-01 02:00:00 - step 3 of 96 - 100 active elements (0 deactivated)
04:47:17 DEBUG: 0 elements scheduled.
04:47:17 DEBUG: ======================================================================
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Variable group ['x_sea_water_velocity', 'y_sea_water_velocity']
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Calling reader ./OD_25x100_Wind05ms_opendrift.nc
04:47:17 DEBUG: ----------------------------------------
04:47:17 DEBUG: Data needed for 100 elements
04:47:17 DEBUG: Reader time:
        2020-07-01 02:00:00 (before)
        2020-07-01 03:00:00 (after)
04:47:17 DEBUG: Fetching variables from ./OD_25x100_Wind05ms_opendrift.nc
04:47:17 INFO: ========================
04:47:17 INFO: Exception:
04:47:17 INFO: operands could not be broadcast together with shapes (259,) (1116,) 
04:47:17 DEBUG: Traceback (most recent call last):
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\models\basemodel.py", line 990, in get_environment
    z[missing_indices], self.use_block, self.proj)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 545, in get_variables_interpolated
    block=block)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 374, in _get_variables
    env = self.get_variables_derived(variables, time, x, y, z, block)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\basereader.py", line 350, in get_variables_derived
    env = self.get_variables(variables, *args, **kwargs)
  File "D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\reader_netCDF_CF_unstructured.py", line 210, in get_variables
    (self.lat > latmin) &
ValueError: operands could not be broadcast together with shapes (259,) (1116,)
knutfrode commented 3 years ago

I see that you are using reader_netCDF_CF_unstructured, but your ncdump indicates this is a regular, structured netCDF file. So you should instead use reader_netCDF_CF_generic. You can use these lines to check the contents:

>>> from opendrift.readers import reader_netCDF_CF_generic
>>> r = reader_netCDF_CF_generic.Reader(<filename>)
>>> print(r)
>>> r.plot()

Or from commandline: readerinfo.py <filename> -p

GuiCruz commented 3 years ago

I was trying to use the generic reader, but I get the following error:

reader = reader_netCDF_CF_generic.Reader('./OD_25x100_Wind05ms_opendrift.nc')
print(reader) 

05:01:15 INFO: Opening dataset: ./OD_25x100_Wind05ms_opendrift.nc
05:01:15 INFO: Opening file with Dataset
05:01:15 DEBUG: Finding coordinate variables.
05:01:15 DEBUG: Parsing variable: latitude_longitude
05:01:15 DEBUG: Parsing CF grid mapping dictionary: {'grid_mapping_name': 'latitude_longitude'}
05:01:15 INFO: Could not parse CF grid_mapping
05:01:15 DEBUG: Parsing variable: Y
05:01:15 DEBUG: Parsing variable: X
05:01:15 DEBUG: Parsing variable: depth
05:01:15 DEBUG: Parsing variable: time
05:01:15 DEBUG: Parsing variable: u
05:01:15 DEBUG: Parsing variable: v
05:01:15 DEBUG: Parsing variable: lon
05:01:15 DEBUG: Parsing variable: lat
0.33333334
[-0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00001144
 -0.00000763 -0.00001144 -0.00001144 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144
 -0.00000763 -0.00001144 -0.00001144 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00000763 -0.00001144 -0.00001144 -0.00000763 -0.00001144
 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144
 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00000763 -0.00001144
 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00000763 -0.00001144
 -0.00000763 -0.00000763 -0.00001144 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00000763 -0.00000763 -0.00001144 -0.00000763 -0.00001144
 -0.00000763 -0.00001144 -0.00000763 -0.00000763 -0.00001144 -0.00000763
 -0.00001144 -0.00000763 -0.00000763 -0.00001144 -0.00000763 -0.00001144
 -0.00000763 -0.00000763 -0.00001144 -0.00000763 -0.00000763 -0.00001144
 -0.00000763 -0.00001144 -0.00000763 -0.00000763 -0.00001144 -0.00000763
 -0.00000763 -0.00001144 -0.00000763 -0.00000763 -0.00001144 -0.00000763
 -0.00000763 -0.00001144 -0.00000763 -0.00000763 -0.00001144 -0.00000763
 -0.00000763 -0.00001144 -0.00000763]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-e6d586f4ff4b> in <module>()
----> 1 reader = reader_netCDF_CF_generic.Reader('./OD_25x100_Wind05ms_opendrift.nc')
      2 print(reader)

D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\reader_netCDF_CF_generic.py in __init__(self, filename, name, proj4, standard_name_mapping)
    332                 print(rel_delta_x)
    333                 print(x[1::] - x[0:-1])
--> 334                 raise ValueError('delta_x is not constant!')
    335             if rel_delta_y > 0.05:
    336                 print(rel_delta_y)

ValueError: delta_x is not constant!
knutfrode commented 3 years ago

It seems your file contains projection_x_coordinate and projection_y_coordinate in meters, but there is no projection information to tell how to interpret these coordinates. Thus you should add a projection variable according to the netCDF CF-convention, containing an attribute "proj4" with the relevant projection.

But easiest here might instead be to simply remove the x- and y-variables (not the dimensions), as OpenDrift will then use 2D polynomial interpolation using the 2D lon- and lat arrays.

GuiCruz commented 3 years ago

Thanks again for helping!

I deleted the x and y variables from the netcdf file

{'latitude_longitude': <class 'netCDF4._netCDF4.Variable'>
 int32 latitude_longitude()
     grid_mapping_name: latitude_longitude
 unlimited dimensions: 
 current shape = ()
 filling on, default _FillValue of -2147483647 used,
 'depth': <class 'netCDF4._netCDF4.Variable'>
 float32 depth(depth)
     units: m
     positive: down
     axis: Z
     standard_name: depth
 unlimited dimensions: 
 current shape = (1,)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'time': <class 'netCDF4._netCDF4.Variable'>
 int32 time(time)
     long_name: time since initialization
     units: seconds since 2020-07-01 00:00:00
     calendar: gregorian
     field: time, scalar, series
     axis: T
     standard_name: time
 unlimited dimensions: time
 current shape = (97,)
 filling on, default _FillValue of -2147483647 used,
 'u': <class 'netCDF4._netCDF4.Variable'>
 float32 u(time, depth, Y, X)
     units: meter second-1
     standard_name: x_sea_water_velocity
     long_name: Sea water x velocity
     coordinates: lat lon
 unlimited dimensions: time
 current shape = (97, 1, 558, 130)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'v': <class 'netCDF4._netCDF4.Variable'>
 float32 v(time, depth, Y, X)
     units: meter second-1
     standard_name: y_sea_water_velocity
     long_name: Sea water y velocity
     coordinates: lat lon
 unlimited dimensions: time
 current shape = (97, 1, 558, 130)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'lon': <class 'netCDF4._netCDF4.Variable'>
 float32 lon(X)
     units: 
     standard_name: lon
     long_name: longitude
 unlimited dimensions: 
 current shape = (130,)
 filling on, default _FillValue of 9.969209968386869e+36 used,
 'lat': <class 'netCDF4._netCDF4.Variable'>
 float32 lat(Y)
     units: 
     standard_name: lat
     long_name: latitude
 unlimited dimensions: 
 current shape = (558,)
 filling on, default _FillValue of 9.969209968386869e+36 used}

But now both readers (unstructured and generic) are not working. They can't find the variable lat and lon.

reader = reader_netCDF_CF_generic.Reader('./OD_25x100_Wind05ms_opendrift.nc')

05:24:01 INFO: Opening dataset: ./OD_25x100_Wind05ms_opendrift.nc
05:24:01 INFO: Opening file with Dataset
05:24:01 DEBUG: Finding coordinate variables.
05:24:01 DEBUG: Parsing variable: latitude_longitude
05:24:01 DEBUG: Parsing CF grid mapping dictionary: {'grid_mapping_name': 'latitude_longitude'}
05:24:01 INFO: Could not parse CF grid_mapping
05:24:01 DEBUG: Parsing variable: depth
05:24:01 DEBUG: Parsing variable: time
05:24:01 DEBUG: Parsing variable: u
05:24:01 DEBUG: Parsing variable: v
05:24:01 DEBUG: Parsing variable: lon
05:24:01 DEBUG: Parsing variable: lat
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-16-7c5533ccb98d> in <module>()
----> 1 reader = reader_netCDF_CF_generic.Reader('./OD_25x100_Wind05ms_opendrift.nc')

D:\Dropbox\Doutorado\Jupyter\opendrift\opendrift\readers\reader_netCDF_CF_generic.py in __init__(self, filename, name, proj4, standard_name_mapping)
    302 
    303         if 'x' not in locals():
--> 304             if self.lon.ndim == 1:
    305                 x = self.lon[:]
    306                 self.xname = lon_var_name

AttributeError: 'Reader' object has no attribute 'lon'
knutfrode commented 3 years ago

The unstructured reader should not be used here, as this is a structured file.

But the netCDF file seems still not CF-compliant, e.g. standard_name should be longitude / latitude and not lon / lat.

GuiCruz commented 3 years ago

Thank you so much @knutfrode it is working now The name has to be Latitude and longitude as you said, and deleting x and y coordinates solved the problem. The generic reader is working now.

I am still getting some problems with the files at C:\Users\Public\Documents\Wondershare\CreatorTemp\landmask, but deleting this folder solved it.

Thank you

knutfrode commented 3 years ago

Very good.

The problem with the landmask might be a Windows issue, so keeping this issue open until that is eventually solved.

GuiCruz commented 3 years ago

Ok. I got just one more question.

In the tutorial to create a landmask it says that the order is # lonmin, lonmax, latmin, latmax. However, when I follow this sequence, some errors appear when running OpenDrift. Is the order right? or it can change according to the layout of the netcdf file dimensions

reader_landmask = reader_global_landmask.Reader(
                       extent=[2, 8, 59, 63])  # lonmin, lonmax, latmin, latmax
knutfrode commented 3 years ago

Normally you do not need to make an explicit landmask-reader, as this is generated automatically at the start of the run.

However, the correct order is lonmin, latmin, lonmax, latmax, so there is a mistake in the tutotial. Thank you for spotting this, which is now fixed: https://github.com/OpenDrift/opendrift/pull/420/commits/e688ba2c925698d01a5791b0978f6857e287ea41

Note that if you update your code with git pull, there has been several changes since your version 1.3.3: https://opendrift.github.io/history_link.html And after updating with git pull, you would also need to update conda environment with $ conda env update -f environment.yml

gauteh commented 3 years ago

Can this issue be closed?

GuiCruz commented 3 years ago

Yes.

Em seg, 16 de nov de 2020 05:19, Gaute Hope notifications@github.com escreveu:

Can this issue be closed?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenDrift/opendrift/issues/418#issuecomment-727816502, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE45N7MQT2EAC3SW25LOHSDSQDOBVANCNFSM4TK35WXQ .

gauteh commented 3 years ago

Ok, thanks.