aws-samples / shopfloor-connectivity

Shop Floor Connectivity (SFC) is an industrial data ingestion enabler, that can quickly deliver customizable greenfield & brownfield connectivity solutions.
https://aws.amazon.com/blogs/industries/collecting-data-from-industrial-devices-to-aws-services/
MIT No Attribution
30 stars 4 forks source link

[Feature Request] Improve interval debug messages #19

Closed qoomon closed 3 months ago

qoomon commented 3 months ago

Currently interval debug messages are not containing any information about the interval e.g. scheduler.

example message Read cycle took 5.107901997s, which is more than read interval of 1s.

I'd be glad if these warning messages get extended by context information to be able to debug the root cause.

And thx again for this awesome project.

arieleeuw commented 3 months ago

Thanks, good feedback!

The warning which is currently based on the total duration of the parallel reads of all sources use by the schedule.

In a next release we'll add:

Will this work for you?

qoomon commented 3 months ago

Actually I'd like to see the scheduler name in the log message, because I have multiple schedules on the same source.

arieleeuw commented 3 months ago

Just added the code. will look like this

2024-07-29 16:09:48.586 WARNING- Reading from  source "OPCUASOURCE" took 1.097s, which is more than the schedule read interval of 1s
2024-07-29 16:09:48.587 WARNING- Read cycle took 1.098090s, which is more than read interval of 1s

Expect to have this release out within the next few days

qoomon commented 3 months ago

Thanks for the quick reply.

If I read a warning like '...which is more than the schedule read interval of 1s' I need to know the actual schedule name.

If an SFC process is reading from a lot sources by a lot of schedules a warning like "Read cycle took 1.098090s, which is more than read interval of 1s" really does not help. I'd need to know the name of the read interval and the name of the source at least.

arieleeuw commented 3 months ago

Hi,

We've added the schedule name as well to the message.

2024-07-30 09:33:21.120 WARNING- Reading from  source "OPCUASOURCE" took 19.221s, which is more than the read interval of 1s for schedule "TestSchedule"
2024-07-30 09:33:21.122 WARNING- Read cycle took 19.23s, which is more than read interval of 1s  for schedule "TestSchedule"

The first warning if for reading for (each) individual source for which the reading takes more than the interval of the schedule. The second one is for the total time it took for reading from all sources. Reading of these are done in parallel and do overlap. The max number of sources that are read in parallel is 5. This number can be tuned with the "MaxConcurrentSourceReaders" setting in the optional "Tuning" section at the root of the configuration.

arieleeuw commented 3 months ago

Implemented in V1.3.1

qoomon commented 3 months ago

@arieleeuw Thank you very much

qoomon commented 3 months ago

just a minor finding there is a double space between from and source "Reading from source"

qoomon commented 3 months ago

The log output is helping a lot thanks a gain