tamsanh / kedro-great

The easiest way to integrate Kedro and Great Expectations
MIT License
52 stars 14 forks source link

Changing BatchMarkers import to make it compatible with GE> 0.13.0 #15

Open lucasjamar opened 3 years ago

lucasjamar commented 3 years ago

Hi,

I made these changes to solve https://github.com/tamsanh/kedro-great/issues/14.

lucasjamar commented 3 years ago

@tamsanh Sorry I did not test thoroughly enough yesterday and I am now coming across error messages. I will try to find them asap.

lucasjamar commented 3 years ago

Hi @tamsanh ,

Sorry but it seems like I bit on more than I can chew and that this is not the quick fix I thought it would be. I am not awfully familiar with GE (I learnt about it through your kedro youtube videos) and I was wondering if you could help me solve this one. Could you perhaps run it on your machine and let me know if you any idea what the issue is? Feel free to write me an email. Let me know if you don't have time to dedicate to this issue.

Sorry again,

Here is the full error message that I am currently getting:

Traceback (most recent call last):
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\JAMAR_L\Anaconda3\envs\kedro-great\Scripts\kedro.exe\__main__.py", line 7, in <module>
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\site-packages\kedro\framework\cli\cli.py", line 725, in main
    cli_collection()
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\JAMAR_L\Documents\kg-test\kedro_cli.py", line 234, in run
    pipeline_name=pipeline,
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\site-packages\kedro\framework\context\context.py", line 767, in run
    raise exc
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\site-packages\kedro\framework\context\context.py", line 759, in run
    run_result = runner.run(filtered_pipeline, catalog, run_id)
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\site-packages\kedro\runner\runner.py", line 101, in run
    self._run(pipeline, catalog, run_id)
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\site-packages\kedro\runner\sequential_runner.py", line 90, in _run
    run_node(node, catalog, self._is_async, run_id)
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\site-packages\kedro\runner\runner.py", line 213, in run_node
    node = _run_node_sequential(node, catalog, run_id)
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\site-packages\kedro\runner\runner.py", line 225, in _run_node_sequential
    node=node, catalog=catalog, inputs=inputs, is_async=is_async, run_id=run_id
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\pluggy\hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\pluggy\manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\pluggy\manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\pluggy\callers.py", line 208, in _multicall
    return outcome.get_result()
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "C:\Users\JAMAR_L\AppData\Roaming\Python\Python37\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "c:\users\jamar_l\documents\kedro-great\kedro_great\kedro_great.py", line 81, in before_node_run
    self._run_validation(catalog, inputs, run_id, read_from_catalog=True)
  File "c:\users\jamar_l\documents\kedro-great\kedro_great\kedro_great.py", line 117, in _run_validation
    dataset_name, dataset_path, df, target_suite_name, run_id
  File "c:\users\jamar_l\documents\kedro-great\kedro_great\kedro_great.py", line 184, in _run_suite
    run_id=run_id
  File "c:\users\jamar_l\anaconda3\envs\kedro-great\lib\site-packages\great_expectations\core\usage_statistics\usage_statistics.py", line 260, in usage_sta
tistics_wrapped_method
    nested_update(event_payload, args_payload_fn(*args, **kwargs))
TypeError: run_validation_operator_usage_statistics() missing 1 required positional argument: 'assets_to_validate'