splunk / eventgen

Splunk Event Generator: Eventgen
Apache License 2.0
380 stars 179 forks source link

[BUG] EventgenX event count is not matched when end > 1 #250

Closed Yangxulight closed 5 years ago

Yangxulight commented 5 years ago

Describe the bug Uncorrect count result when generatorWorkers > 1

To Reproduce Use any samples with settings "end = 20, count = 100" to build events. Also, set "generatorWorkers=10" Expected behavior Get 20*100 = 2000 result events.

Actual behavior I got 200 events at all. It may cause by codes in eventgentimmer.py: "self.executions += 1" increase at any generator jobs. It should has 20 executions for each generators(totally 200), now it only has 20 executions for all generators(totally 20).

                        for worker_id in range(self.config.generatorWorkers):
                            # self.generatorPlugin is only an instance, now we need a real plugin. Make a copy of
                            # of the sample in case another generator corrupts it.
                            # copy_sample = copy.copy(self.sample)
                            # tokens = copy.deepcopy(self.sample.tokens)
                            # copy_sample.tokens = tokens
                            # genPlugin = self.generatorPlugin(sample=copy_sample)
                            genPlugin = self.generatorPlugin(sample=self.sample)
                            # Adjust queue for threading mode
                            genPlugin.updateConfig(config=self.config, outqueue=self.outputQueue)
                            genPlugin.updateCounts(count=count, start_time=et, end_time=lt)

                            try:
                                self.generatorQueue.put(genPlugin)
                                self.executions += 1
                                self.logger.debug(("Worker# {0}: Put {1} MB of events in queue for sample '{2}'" +
                                                   "with et '{3}' and lt '{4}'").format(
                                                      worker_id, round((count / 1024.0 / 1024), 4),
                                                      self.sample.name, et, lt))`

Screenshots If applicable, add screenshots to help explain your problem.

Sample files and eventgen.conf file Please attach your sample files and eventgen conf file

Do you run eventgen with SA-eventgen? Yes/No(No means you run eventgen with pip module mode)

If you are using SA-Eventgen with Splunk (please complete the following information):

If you are using eventgen with pip module mode (please complete the following information):

Additional context Add any other context about the problem here.

Yangxulight commented 5 years ago

Already fixed in this PR. https://github.com/splunk/eventgen/pull/251