HDF5-DIAG: Error

[claytonpbarrows]

julia> sys = PSB.build_system(PSB.SIIPExampleSystems, "5_bus_hydro_uc_sys")
[ Info: Deserializing with InMemoryTimeSeriesStorage is currently not supported. Using HDF
[ Info: Loaded time series from storage file existing=5_bus_hydro_uc_sys_time_series_storage.h5 new=/var/folders/27/2jr8c7gn4j72fvrg4qt81zrw8w_711/T/jl_PM3GKQ compression=CompressionSettings(false, CompressionTypes.DEFLATE = 1, 3, true)
System
======
System Units Base: DEVICE_BASE
Base Power: 100.0
Base Frequency: 60.0

Components
==========
Num components: 35

12×3 DataFrame
 Row │ ConcreteType                SuperTypes                        
     │ String                      String                            
─────┼──────────────────────────────────────────────────────────────────
   1 │ Arc                         Topology <: Component <: Infrast…
   2 │ Area                        AggregationTopology <: Topology …
   3 │ Bus                         Topology <: Component <: Infrast…
   4 │ HydroDispatch               HydroGen <: Generator <: StaticI…
   5 │ HydroEnergyReservoir        HydroGen <: Generator <: StaticI…
   6 │ HydroPumpedStorage          HydroGen <: Generator <: StaticI…
   7 │ Line                        ACBranch <: Branch <: Device <: …
   8 │ LoadZone                    AggregationTopology <: Topology …
   9 │ PowerLoad                   StaticLoad <: ElectricLoad <: St…
  10 │ TapTransformer              ACBranch <: Branch <: Device <: …
  11 │ ThermalStandard             ThermalGen <: Generator <: Stati…
  12 │ VariableReserve{ReserveUp}  Reserve{ReserveUp} <: Service <:…
                                                        1 column omitted

TimeSeriesContainer
===================
Components with time series data: 7
Total StaticTimeSeries: 14
Total Forecasts: 14
Resolution: 60 minutes
First initial time: 2020-01-01T00:00:00
Last initial time: 2020-01-07T00:00:00
Horizon: 24
Interval: 1440 minutes
Forecast window count: 7

julia>  re = RenewableDispatch(
               "WindBusA",
               true,
               get_component(Bus, sys, "bus5"),
               0.0,
               0.0,
               1.200,
               PrimeMovers.WT,
                       (min = 0.0, max = 0.0),
               1.0,
               TwoPartCost(0.220, 0.0),
               100.0,
                   )

┌ Warning: SystemUnitSetting not defined, using NATURAL_UNITS for displaying device specification.
└ @ PowerSystems ~/.julia/packages/PowerSystems/DdZ5W/src/utils/print.jl:69
WindBusA (RenewableDispatch):
   name: WindBusA
   available: true
   bus: bus5 (Bus)
   active_power: 0.0
   reactive_power: 0.0
   rating: 120.0
   prime_mover: PrimeMovers.WT = 22
   reactive_power_limits: (min = 0.0, max = 0.0)
   power_factor: 1.0
   operation_cost: TwoPartCost
   base_power: 100.0
   services: 0-element Vector{Service}
   dynamic_injector: nothing
   ext: Dict{String, Any}()
   time_series_container: InfrastructureSystems.TimeSeriesContainer: 0
   internal: InfrastructureSystems.InfrastructureSystemsInternal

julia> add_component!(sys, re)

julia> copy_time_series!(re, get_component(PowerLoad, sys, "bus2"))
HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
  #000: H5A.c line 312 in H5Acreate2(): unable to create attribute
    major: Attribute
    minor: Unable to initialize object
  #001: H5VLcallback.c line 1023 in H5VL_attr_create(): attribute create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #002: H5VLcallback.c line 989 in H5VL__attr_create(): attribute create failed
    major: Virtual Object Layer
    minor: Unable to create file
  #003: H5VLnative_attr.c line 77 in H5VL__native_attr_create(): unable to create attribute
    major: Attribute
    minor: Unable to initialize object
  #004: H5Aint.c line 166 in H5A__create(): attribute already exists
    major: Attribute
    minor: Object already exists
HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
  #000: H5A.c line 885 in H5Aget_name(): not an attribute
    major: Invalid arguments to routine
    minor: Inappropriate type
ERROR: Error getting attribute name
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] h5a_get_name
    @ ~/.julia/packages/HDF5/VJkAi/src/api.jl:207 [inlined]
  [3] h5a_get_name(attr_id::HDF5.Group)
    @ HDF5 ~/.julia/packages/HDF5/VJkAi/src/api_helpers.jl:27
  [4] h5a_create
    @ ~/.julia/packages/HDF5/VJkAi/src/api.jl:119 [inlined]
  [5] create_attribute(parent::HDF5.Group, name::String, dtype::HDF5.Datatype, dspace::HDF5.Dataspace)
    @ HDF5 ~/.julia/packages/HDF5/VJkAi/src/HDF5.jl:750
  [6] create_attribute(parent::HDF5.Group, name::String, data::Bool; pv::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ HDF5 ~/.julia/packages/HDF5/VJkAi/src/HDF5.jl:1538
  [7] create_attribute
    @ ~/.julia/packages/HDF5/VJkAi/src/HDF5.jl:1535 [inlined]
  [8] write_attribute(parent::HDF5.Group, name::String, data::Bool; pv::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ HDF5 ~/.julia/packages/HDF5/VJkAi/src/HDF5.jl:1560
  [9] write_attribute
    @ ~/.julia/packages/HDF5/VJkAi/src/HDF5.jl:1560 [inlined]
 [10] setindex!
    @ ~/.julia/packages/HDF5/VJkAi/src/HDF5.jl:882 [inlined]
 [11] (::InfrastructureSystems.var"#116#117"{InfrastructureSystems.Hdf5TimeSeriesStorage, Base.UUID, String, String, String})(file::HDF5.File)
    @ InfrastructureSystems ~/.julia/packages/InfrastructureSystems/fSRUY/src/hdf5_time_series_storage.jl:269
 [12] h5open(::InfrastructureSystems.var"#116#117"{InfrastructureSystems.Hdf5TimeSeriesStorage, Base.UUID, String, String, String}, ::String, ::Vararg{String, N} where N; swmr::Bool, pv::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ HDF5 ~/.julia/packages/HDF5/VJkAi/src/HDF5.jl:458
 [13] h5open
    @ ~/.julia/packages/HDF5/VJkAi/src/HDF5.jl:456 [inlined]
 [14] add_time_series_reference!(storage::InfrastructureSystems.Hdf5TimeSeriesStorage, component_uuid::Base.UUID, name::String, ts_uuid::Base.UUID)
    @ InfrastructureSystems ~/.julia/packages/InfrastructureSystems/fSRUY/src/hdf5_time_series_storage.jl:266
 [15] copy_time_series!(dst::RenewableDispatch, src::PowerLoad; name_mapping::Nothing, scaling_factor_multiplier_mapping::Nothing)
    @ InfrastructureSystems ~/.julia/packages/InfrastructureSystems/fSRUY/src/component.jl:444
 [16] #copy_time_series!#156
    @ ~/.julia/packages/PowerSystems/DdZ5W/src/base.jl:1014 [inlined]
 [17] copy_time_series!(dst::RenewableDispatch, src::PowerLoad)
    @ PowerSystems ~/.julia/packages/PowerSystems/DdZ5W/src/base.jl:1014
 [18] top-level scope
    @ REPL[22]:1

[claytonpbarrows]

This appears to only happen when the system has been deserialized from a json.

[daniel-thom]

This bug should happen anytime you call copy_time_series! on a component with SingleTimeSeries that has been transformed into DeterministicSingleTimeSeries. In the copy logic we try to add the same HDF5 attribute twice (buried in the stack trace is attribute already exists). There are two views into the same time array; we don't need to make the second call.

This is actually a long-standing bug. I recently optimized some of our logic accessing HDF5 files and found that we were allowing duplicate references. The consequence of that bug was that removing a component would not have properly deleted the time array. I fixed that bug last week, so that is why you are seeing this now.

I don't think that this has anything to do with serialization. The system you are creating with PSB uses in-memory time series. There is different copy logic for that implementation. We currently have a limitation in that if you deserialize a system that was created with in-memory time series you will get back HDF5 time series.