Apollo3zehn / PureHDF

A pure .NET library that makes reading and writing of HDF5 files (groups, datasets, attributes, ...) very easy.
MIT License
47 stars 16 forks source link

Source Generator for Writing #77

Open Blackclaws opened 2 months ago

Blackclaws commented 2 months ago

We're using PureHDF as our main framework for writing HDF files from C# and I'm pretty happy with it.

We've internally written a source generator that allows us to more easily write/read HDF5 files via Properties (without relying on a sample file), would you be interested in a contribution of that functionality back to PureHDF? I still need to also clear this with the appropriate people within the company but asking generally if this is something worth contributing back :)

Apollo3zehn commented 2 months ago

That sound definitely interesting! Do you have a small sample so I could better understand how one would use that in practice? I am happy to accept contributions to PureHDF :-)

Blackclaws commented 2 months ago

In general you'd use it like this:

[ReportProperty<double>("duration")]
[ReportProperty<string>("run_started")]
[ReportProperty<string>("run_ended")]
[ReportProperty<string>("Pattern", TargetStorage.Attribute)]
[ReportProperty<int>("PatternLength", TargetStorage.Attribute)]
[ReportProperty<int>("Waveforms", TargetStorage.Attribute)]
[ReportProperty<int>("AveragedWaveforms", TargetStorage.Attribute)]
[ReportProperty<ChannelResult>("ReferenceMeasurement")]
public sealed partial class TestResult : ReportGroup
{
}

and then later:

 var      results  = new TestResult();
 results.run_started       = DateTime.UtcNow.ToString("s");
 results.PatternLength     = 100;
 results.Waveforms         = 20;
 results.AveragedWaveforms = 20;

The generated code looks like this:

 partial class TestResult
 {
        public string? Pattern  {
             get
             {
                 if (this.Attributes.TryGetValue("Pattern", out var value) && value is string dt)
                 {
                     return dt;
                 }
                 return null;
             }
             set {
                 if (value is null)
                 {
                     this.Attributes.Remove("Pattern");
                     return;
                 }
                 this.Attributes["Pattern"] = value;
             }
         }
 }

The basic idea is that it allows easy hardcoding of expected types and also allows retrieving the stored data in other parts of the program when passing around data. This right now only works for in-memory non-read H5 data but I'm planning to also extend this to allowing typed access to H5Files that have been read. It could also solve my problem from the other issue #69 as a fixed structure would allow round-tripping of all the values that are expected during compile time. It would probably silently drop those that weren't expected but that might also be worked around.

Apollo3zehn commented 2 months ago

Thanks for the nice sample code and description! I think this could be useful for PureHDF and for the beginning I would like to see it living in a separate package in the src folder to learn how well it integrates with the main library. But as far as I know from my own source generator experiments (src/PureHDF.SourceGenerator), it is actually a requirement to have a separate library.

So, if you find the time and get it approved internally, please go ahead :-)