search

ourresearch / total-impact-core

An api and backend code to gather the impacts of diverse scholarly products online.
http://impactstory.org
Other
55 stars 7 forks source link

remove sleep from providers #80

Closed jasonpriem closed 12 years ago

kevcampb commented 12 years ago

Not 100% sure on what this means, but I've removed all timeout / retry error handling code from the providers. This should still be specificed per-provider in the json config, but the handling of this is now done in the outer controller class 'ProviderMetricsThread'.

Code is on the providerqueue branch.

jasonpriem commented 12 years ago

I like the pushing error-handling out of the providers, for sure. But it seems that the ProvidersAliasThread no longer handles errors at all. I think that perhaps the process_item method needs to be moved out of the ProviderMetricsThread and up into the QueueConsumer thread, so that both of the ProviderThreads can use it?

On Thu, Apr 12, 2012 at 9:18 AM, Kevin Campbell reply@reply.github.com wrote:

Not 100% sure on what this means, but I've removed all timeout / retry error handling code from the providers. This should still be specificed per-provider in the json config, but the handling of this is now done in the outer controller class 'ProviderMetricsThread'.

Code is on the providerqueue branch.

Reply to this email directly or view it on GitHub: https://github.com/total-impact/total-impact-core/issues/80#issuecomment-5090663

Jason Priem UNC Royster Scholar School of Information and Library Science University of North Carolina at Chapel Hill

kevcampb commented 12 years ago

I've not really done much with the Alias thread, I was waiting for results from today's discussion. From that discussion, it should effectively be the same as the Metrics thread, but processing a sequence of operations rather than a single call to metrics.

Should have that done early tomorrow.

K

On Fri, Apr 13, 2012 at 12:01 AM, Jason Priem < reply@reply.github.com

wrote:

I like the pushing error-handling out of the providers, for sure. But it seems that the ProvidersAliasThread no longer handles errors at all. I think that perhaps the process_item method needs to be moved out of the ProviderMetricsThread and up into the QueueConsumer thread, so that both of the ProviderThreads can use it?

On Thu, Apr 12, 2012 at 9:18 AM, Kevin Campbell reply@reply.github.com wrote:

Not 100% sure on what this means, but I've removed all timeout / retry error handling code from the providers. This should still be specificed per-provider in the json config, but the handling of this is now done in the outer controller class 'ProviderMetricsThread'.

Code is on the providerqueue branch.

Reply to this email directly or view it on GitHub:

https://github.com/total-impact/total-impact-core/issues/80#issuecomment-5090663

Jason Priem UNC Royster Scholar School of Information and Library Science University of North Carolina at Chapel Hill

Reply to this email directly or view it on GitHub:

https://github.com/total-impact/total-impact-core/issues/80#issuecomment-5102624

jasonpriem commented 12 years ago

Kevin, do you think this one is ready to close now?

kevcampb commented 12 years ago

Sleep will go from the providers as part of the cleanup / rewrite. Will make that a larger task.

jasonpriem commented 12 years ago

perfect.

On Fri, Apr 27, 2012 at 5:44 AM, Kevin Campbell < reply@reply.github.com

wrote:

Sleep will go from the providers as part of the cleanup / rewrite. Will make that a larger task.

Reply to this email directly or view it on GitHub:

https://github.com/total-impact/total-impact-core/issues/80#issuecomment-5378149

Jason Priem UNC Royster Scholar School of Information and Library Science University of North Carolina at Chapel Hill