WPRDC / wprdc-etl

MIT License
8 stars 3 forks source link

timing of pipeline status creation #38

Closed saylorsd closed 8 years ago

saylorsd commented 8 years ago

In pipeline.run() if an exception is raised before it's status is instantiated, this finally: block of code will still try to access the nonexistent status object. This makes it a bit harder to track down the real issue, and maybe we'd want to store other errors that arise in the status database.

If the instantiation is moved earlier in the code so that a status objects exists by the time the pipeline reaches that block, the test_checksum_duplicate_prevention() test fails since a new status is written instead of just failing and raising the 'Pipeline' object has no attribute 'status' error.

It seem's the the duplicate test is working by virtue of the exception occurring before status is created. However, if this happends or if an exception is raised in the instantiation of the connector click will just return 'Pipeline' object has no attribute 'status', since the others are further down the stack trace.

bsmithgall commented 8 years ago

Good catch. Which test is bad?

bsmithgall commented 8 years ago

I think this can be changed by changing the if statement to: if self.log_status and hasattr(self, 'status'): from if self.log_status:

bsmithgall commented 8 years ago

Should be fixed by f78f730

saylorsd commented 8 years ago

I guess I was way over thinking that. :laughing: I merged it and I'll close the issue.