dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
11.7k stars 1.48k forks source link

Improve error message #181

Closed Aylr closed 5 years ago

Aylr commented 6 years ago

Output

dagster pipeline execute cohort_report_pipeline --env test.yml && open build/topline_cohort_report.html 
Traceback (most recent call last):
  File "/Users/taylor/repos/cricket-health-201806/analysis_backbone/.venv/bin/dagster", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/Users/taylor/repos/dagster/python_modules/dagster/bin/dagster", line 10, in <module>
    cli(obj={})
  File "/Users/taylor/repos/cricket-health-201806/analysis_backbone/.venv/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/taylor/repos/cricket-health-201806/analysis_backbone/.venv/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/taylor/repos/cricket-health-201806/analysis_backbone/.venv/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/taylor/repos/cricket-health-201806/analysis_backbone/.venv/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/taylor/repos/cricket-health-201806/analysis_backbone/.venv/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/taylor/repos/cricket-health-201806/analysis_backbone/.venv/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/taylor/repos/dagster/python_modules/dagster/dagster/cli/pipeline.py", line 207, in execute_command
    pipeline = pipeline_from_conf(conf, name)
  File "/Users/taylor/repos/dagster/python_modules/dagster/dagster/cli/pipeline.py", line 74, in pipeline_from_conf
    return repository.get_pipeline(name)
  File "/Users/taylor/repos/dagster/python_modules/dagster/dagster/core/definitions.py", line 1174, in get_pipeline
    pipeline = self.pipeline_dict[name]()
  File "/Users/taylor/repos/cricket-health-201806/analysis_backbone/backbone/backbone/cohort_reports.py", line 289, in cohort_report_pipeline
    context_definitions=backbone_context_definitions(),
  File "/Users/taylor/repos/dagster/python_modules/dagster/dagster/core/definitions.py", line 358, in __init__
    self.__validate_dependences(dependencies)
  File "/Users/taylor/repos/dagster/python_modules/dagster/dagster/core/definitions.py", line 492, in __validate_dependences
    'Solid {dep.solid} does not have output {dep.output}'.format(dep=dep),
dagster.core.errors.DagsterInvalidDefinitionError: Solid generate_cohort_population_stats_and_charts does not have output result
mgasner commented 5 years ago

@Aylr what actually caused this error from a user perspective -- what did you have to change or do differently to make it go away?

schrockn commented 5 years ago

This looks like it happens when you specify a dependency to an output that does not exist

Aylr commented 5 years ago

Wow, this is an old one! I think I was wishing for an error that guides the user a little more - and the word "result" is so generic - what actual abstraction is missing?

schrockn commented 5 years ago

I believe this happens when you do something like

dependencies={
   'solid_a' : DependencyDefinition('solid_b')
}

But solid_b actually have outputs named not the default ("result") so the dependency finds "solid_b" but not the output "result".

natekupp commented 5 years ago

@schrockn is this still an issue?

schrockn commented 5 years ago

Yes.

Offending error message is in container.py:285

alangenfeld commented 5 years ago

https://dagster.phacility.com/D742