Minyus / pipelinex

PipelineX: Python package to build ML pipelines for experimentation with Kedro, MLflow, and more
https://pipelinex.readthedocs.io/
Other
221 stars 11 forks source link

HatchDict: Is there a way to reference variables as parameters? #15

Closed npow closed 2 years ago

npow commented 2 years ago

Hi, came across HatchDict which has been super useful so far! I'm trying to reference variables in parameters. For example, below I have the variable C defined in some file already. Is there a way to achieve this with HatchDict? This is a contrived example, but it would be pretty useful for parameters like long lists.

model:
  =: sklearn.linear_classifier.SGDClassifier
  C: =: mylib.constants.C
Minyus commented 2 years ago

Yes, you can write like this:

from pipelinex import HatchDict

params_yaml="""
model:
  =: sklearn.linear_model.SGDClassifier
  alpha: 
    =: numpy.pi 
"""
model = HatchDict(parameters).get("model")

print(model)

Output:

SGDClassifier(alpha=3.141592653589793)

numpy.pi is just an example. You can use your constant in your package mylib. If not, make sure import mylib works fine.

npow commented 2 years ago

Awesome, works like a charm!