AllenCellModeling / datasetdatabase

Modeling DB Schema, Creation, and IO
Other
1 stars 0 forks source link

Add generalizable introspectors #2

Closed evamaxfield closed 6 years ago

evamaxfield commented 6 years ago

Generalizable Introspectors

Allow custom validators and parsers for any python object to Dataset object. This would allow for any data type to be consumed and processed by DSDB. Meaning, if a person wants to track a single object as it is process instead of a dataframe like dataset you can.

Example: Dataset(pytorch_model), allows for tracking of epochs/ cycles. Introspector could then read the state of the model to create Iota based off it's attributes.

Default: assumed DataFrame like.

introspectors.init

from .object import ObjectIntrospector
import dataframe
import ometif

introspector_map = {
   pd.DataFrame = dataframe.DataFrameIntrospector,
   OmeTifReader = ometif.OmeTifIntrospector
}

core

from .introspectors import introspector_map, ObjectIntrospector

class Dataset(object):
   def __init__(self, obj):
      try:
         self._introspector = introspector_map[type(obj)]
      except KeyError:
         self._introspector = ObjectIntrospector

      self._validated = self.introspector.validated

   def validate(self):
      self.introspector.validate()

   def generate(self):
      ds_id_map = self.introspector.generate()
evamaxfield commented 6 years ago

General model is complete so closing the issue.

Custom structures must have the following:

__init__(self, obj):
# initialize the basic parameters of your introspector
# at a minimum requires `_obj` and `_validated`.
self._obj = obj
self._validated = False

@property
def obj(self):
# so users cannot edit the object

@property
def validated(self):
# so users cannot edit the validated props

def validate(self):
# single call to validate that can call other validation methods

def deconstruct(self):
# take object and create Iota, Group, and IotaGroup lists

def reconstruct(self):
# take lists of Iota, Group, and IotaGroup and create object

def package(self):
# take object and form into Quilt package