search

DAGWorks-Inc / hamilton

Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
https://hamilton.dagworks.io/en/latest/
BSD 3-Clause Clear License
1.84k stars 124 forks source link

allow_module_overrides doesn't work with AsyncBuilder #1216

Open rwhitten577 opened 4 hours ago

rwhitten577 commented 4 hours ago

Tried using the allow_module_overrides with an AsyncBuilder. It sets the param on the builder, but it does not get passed in when the AsyncDriver (and thus the Driver) is initialized:

  1. https://github.com/DAGWorks-Inc/hamilton/blob/main/hamilton/async_driver.py#L490-L495
  2. https://github.com/DAGWorks-Inc/hamilton/blob/main/hamilton/async_driver.py#L232-L246

I think the arg just needs to be passed down from AsyncBuilder -> AsyncDriver -> Driver.

Example: 

builder = (
     AsyncBuilder()
    .with_config(config)
    .with_modules(*self.included_modules)
    .allow_module_overrides()
    .with_adapters(self.result_builder)
)
driver = await builder.build()
rwhitten577 commented 4 hours ago

Working on a PR now