search

Defra-Data-Science-Centre-of-Excellence / sds-data-model

A common data model for the Spatial Data Science unit
https://defra-data-science-centre-of-excellence.github.io/sds-data-model/
MIT License
0 stars 0 forks source link

Refactor `logging` and `graphing` functions to work with new wrappers #164

Open EFT-Defra opened 1 year ago

s-flint-defra commented 1 year ago

@EFT-Defra this issue has been quiet for a while: 1] Has this been completed? 2] If not, please can you confirm this is part of the Transformation History Epic?

EFT-Defra commented 1 year ago

Hi @s-flint-defra,

1) No, it hasn't been completed but @jpd-defra and I have started working on it again as part of our pair programming. @jpd-defra, do we have a pull request for this? 2) Yes, it should be part of the "Transformation History Epic"

JamesKeny0n commented 1 year ago

@EFT-Defra is this a limiter on #107?

EFT-Defra commented 1 year ago

@JamesKeny0n, no, it shouldn't be.

What @jpd-defra and I have been working on is a way of building a DAG using the new DataFrameWrapper methods, combining DAGs when DataFrameWappers are joined or unioned, and writing the DAG to the attrs attribute of a .zarr file.

Once we've got that working, the DatasetWrapper class and DatasetWrapper.from_files constructor will need to be modified to extract the DAG from the attrs but apart from that, it shouldn't affect work on the DatasetWrapper.