search

astropy / astropy

Astronomy and astrophysics core library
https://www.astropy.org
BSD 3-Clause "New" or "Revised" License
4.39k stars 1.75k forks source link

Saving Model Parameters to File #12958

Open robelgeda opened 2 years ago

robelgeda commented 2 years ago

Description

I don't believe there is a way to write out models to file. It would be useful to have the machinery to write out model parameters as FITS tables, ECSV, or YAML files. I think this feature should mainly focus on saving the parameter values as opposed to the model itself (for example it should not pickle the model object). As of opening this issue, the feature should be able to:

  1. Save model parameters to file.
  2. Have unit support.
  3. Metadata of the model class that the parameters come from. If the model is a compound model, it should also have data on the individual models that created that compound model. This metadata is only for documenting the parent model and not to be used for recreating the model. It should be on the user to choose the appropriate model based on the metadata they see in the file. In the future, we could introduce machinery to recreate models (in the future because custom models can be tricky to pull off).
  4. Either have load function for all model classes (probably in the base classes), where a copy of the model itself is returned with the loaded parameters --or--- a single function to load and convert parameters into quantities that can be used to initiate the model, for example: 
    >>> model.write_params(file_path)
>>> new_model = SomeModel2D.read_params((file_path)
>>> kwargs = load_model_params(file_path) # returns dict of quantities 
>>> new_model = SomeModel2D(**kwargs)
  1. Since this has us thinking about how to save parameters, we should also attempt or think about attempting to save bounds and fixed parameters.

My thinking is that it would be easy to do all of this using the astropy table machinery. We would add bounds and whether or not a parameter is fixed as separate columns in the table. This table can save the data to whatever format the user wants (though we should restrict it to formats that can store metadata). What I am not sure about is how the API should work, should this be a function in the model like Table.read(file_name) (i.e SomeModel2D.read(file_name)) or an external function like the code block above?

I would be willing to look into this if this idea seems interesting. All help, suggestions, and constructive criticism welcome!

Additional context

Related to PetroFit ticket

pllim commented 2 years ago

I think there is some sort of serialization support over at asdf-astropy: https://github.com/astropy/asdf-astropy/tree/main/asdf_astropy/converters/transform

Maybe @WilliamJamieson can clarify.

WilliamJamieson commented 2 years ago

I think there is some sort of serialization support over at asdf-astropy: https://github.com/astropy/asdf-astropy/tree/main/asdf_astropy/converters/transform

Correct, serialization support for astropy models is provided by saving to asdf files using the asdf-astropy library.

pllim commented 2 years ago

@WilliamJamieson , would be nice if there is some user doc we can point to.

nstarman commented 2 years ago

A larger project, but maybe worthwhile, is to add read, write methods to Model, using the unified registry. It's been a goal of mine to add entry points to the registry system. Then, asdf-astropy could register into Model.read/write ASDF serialization.

WilliamJamieson commented 2 years ago

A larger project, but maybe worthwhile, is to add read, write methods to Model, using the unified registry. It's been a goal of mine to add entry points to the registry system. Then, asdf-astropy could register into Model.read/write ASDF serialization.

This sounds reasonable to me, feel free to open PRs on modeling and asdf-astropy to accomplish this.

nstarman commented 2 years ago

This sounds reasonable to me, feel free to open PRs on modeling and asdf-astropy to accomplish this.

Haha. I have literally 35 still-open issues related to this for Cosmology. https://github.com/orgs/astropy/projects/8/views/13 It might be a LOOONG while.

WilliamJamieson commented 2 years ago

Haha. I have literally 35 still-open issues related to this for Cosmology. https://github.com/orgs/astropy/projects/8/views/13 It might be a LOOONG while.

I understand, I am fairly swamped with activities in other places. It might be best to wait until the transition from the internal astropy.io.misc.asdf to asdf-astropy has settled out anyways.

robelgeda commented 2 years ago

I might be able to help in the summer depending on other projects. I will post here to update. Very excited to hear about all of this!