Minor small fixes/enhancements accumulated using use

hibernating-rhinos / rhino-etl

Developer friendly ETL Library for .NET

http://hibernatingrhinos.com/open-source/rhino-etl

BSD 3-Clause "New" or "Revised" License

173 stars 54 forks source link

Minor small fixes/enhancements accumulated using use #2

Closed mr-miles closed 13 years ago

mr-miles commented 13 years ago

Hi,

I've accumulated some fixes and a couple of small changes through use, so I finally got the time to put them together into a pull request:

Add missing event notifications to multi-source/output operations (JoinOperation, AbstractBranchingOperation)
Provide hook into progress reporting from the SqlBulkInsert task
Change visibility of ThreadSafeGatedEnumerator, only one that was internal
Allow PartialProcessOperation to pass the PrepareForExecution event down to child operations
Add before/after execution events to the AbstractPipelineExecuter

Let me know if you need any info,

Cheers,

Miles

nathanpalmer commented 13 years ago

Thanks for the request. Do you have any unit tests for this? More specifically I'm curious with how you are utilizing the NotifyExecutionStarting and NotifyExecutionCompleting.

Thanks.

mr-miles commented 13 years ago

Hi,

Sure, I'll put some tests together,

On the events: I have a generic "directory" class which contains all the filenames, sql queries and so on for an entire batch of ETL processes. When running a particular process, the event allows all the necessary inputs to be queried/cached in one hit (and to fail straight away if unsuccessful, rather than in 20 minutes time), before moving on to the transform stage. It meant I got to remove a lot of repetitive operations from the pipelines, and also that I could specify in one place if I wanted to only use local copies of data.

Miles

On Wed, Apr 6, 2011 at 3:43 PM, nathanpalmer < reply@reply.github.com>wrote:

Thanks for the request. Do you have any unit tests for this? More specifically I'm curious with how you are utilizing the NotifyExecutionStarting and NotifyExecutionCompleting.

Thanks.

Reply to this email directly or view it on GitHub: https://github.com/hibernating-rhinos/rhino-etl/pull/2#issuecomment-964256

mr-miles commented 13 years ago

Okay, I added lots of tests but I've royally messed up the line-endings and tabbing. What's the policy? I've tried everything I can think of to get it back but I'm failing! any ideas?

nathanpalmer commented 13 years ago

I'm not intentionally ignoring this. However, I don't want to merge your changes in with all of the line-ending and tabbing changes. Policy is generally leave tabbing and line-endings the way you found them. Makes the history much easier to read and the project consistent.

mr-miles commented 13 years ago

Yeah, makes sense. When I have a bit more time I'll start again afresh, I think I've made such a mess of the line endings there's not much else that can be done with this particular branch.

On Tue, Apr 19, 2011 at 4:30 AM, nathanpalmer < reply@reply.github.com>wrote:

I'm not intentionally ignoring this. However, I don't want to merge your changes in with all of the line-ending and tabbing changes. Policy is generally leave tabbing and line-endings the way you found them. Makes the history much easier to read and the project consistent.

Reply to this email directly or view it on GitHub: https://github.com/hibernating-rhinos/rhino-etl/pull/2#issuecomment-1024900

nathanpalmer commented 13 years ago

Sounds like a plan. I've had to re-create branches in the past due to line ending problems. Make sure your git line ending settings are correct and try to avoid reformatting from applications such as ReSharper.