search

WebCuratorTool / webcurator

The root of the webcurator tool project, containing all modules needed to run a fully functional webcurator tool.
Apache License 2.0
2 stars 1 forks source link

Simplified harvest-agent-h3 configuration #9

Closed obrienben closed 4 years ago

obrienben commented 4 years ago

Tried to simplify the configuration for harvest-agent-h3, and remove some of the duplicate settings

leefrank9527 commented 4 years ago
  1. The value of agent.port in application-local.properties could refer ${server.port} in application.properties.
  2. The logging level configuration items do exist in both application.properties and application-local.properties. Recommend to get rid them from application-local.properties. This is common for all components. A question here is how to change the logging level when application is running without rebuilding or rebooting?
leefrank9527 commented 4 years ago
  1. Comparing with WebApp component, "application-local.properties" are unnecessary for other components. Furthermore, for WebApp it would be more concise if the configuration items NOT related to DB could be abstracted and moved to wct-webapp.properties.
obrienben commented 4 years ago

Thanks @leefrank9527 I'll address your first two comments and update this PR, and do some testing around your third point and see if it makes sense to move all non-db settings back into the properties files. @hannakoppelaar thoughts on shifting all non-db settings back into the properties files?

obrienben commented 4 years ago

@leefrank9527 I've taken the simplification further,

Have tested crawls on H3 and H1.

@hannakoppelaar what are your thoughts on these changes? Are there any implications for your Prod deployment?

hannakoppelaar commented 4 years ago

I have an external application-local.properties to configure store, which contains the key file.archive.repository. This variable is no longer being picked up. I had to change it to fileArchive.archiveRepository. Other than that, it seems fine. I can run harvests and archive them.

obrienben commented 4 years ago

Closing as the commits in this branch need to be redone. Now super-seeded by PR #13