search

logstash-plugins / logstash-input-salesforce

Logstash input for pulling objects from salesforce
Apache License 2.0
16 stars 29 forks source link

Support Scheduling for the Salesforce Input Plugin #30

Closed marc-sensenich closed 2 years ago

marc-sensenich commented 2 years ago

Adds support for scheduling the Salesforce Input Plugin within the pipeline. This currently doesn't hold any sort of state within the plugin or filesystem, this just allows for the re-running of the plugin from the plugin itself

Closes #19

Thanks for contributing to Logstash! If you haven't already signed our CLA, here's a handy link: https://www.elastic.co/contributor-agreement/

mkreth commented 2 years ago

Have you thought about how to get information about the schedule into the query, e.g. to get only records which were created or changed during the last time the scheduler fired vs the current firing time? The Job.last_time and Job.scheduled_at properties look like good inputs for such a query. The filter of the query could be "WHERE SystemModstamp > AND SystemModstamp <= ".

marc-sensenich commented 2 years ago

@mkreth I don't work with Salesforce any more and haven't thought about injecting more information into the query to solve those issues. With my current personal and professional schedule, I'm not going to be carrying this PR forward. But if you have a use case where this would be valuable, please carry forth with the work I started.

qury commented 6 months ago

would it be possible to resurrect this commit? I would gladly use this in it's current form just to trigger the connector every so often.