Automating graph generation and creating reports for number of cars over a speed threshold

[johncohn]

FIrst .. I want to thank you for a wonderful(!!!) piece of software. I've been using it successfully for several weeks… It's very well-crafted, and there are so many options I'm enjoying exploring them all. I do have two questions which I'm sure are easily answered and I just happen to be missing it

1. whats the best way to automate the generation of the count and speed average graphs ? .. I am able to run python sql-make-graph-speed-ave.py and python sql-make-graph-count-totals.py

to manually generate the graphs… But they seem to be written as a loop that once you start them they automatically do it on a timed basis… I tried to put that as a Cron job… But I have a feeling that just keeps starting more jobs. Is there a way to automatically start the graph creation so that it's done at a regular interval?… I know I could rewrite those python script so that they didn't loop forever but I figure there must be already something like that out there for me to use

2. What's the best way to view t Data on cars that are going over a fixed threshold… E.g. I am monitoring a 25 mile an hour zone… I'd like to be able to get a report that showed me how many cars were going over 40 miles an hour on a given day or week … in addition to a total count of cars going over a certain speed I'd like to know when they went by and ideally a pointer to the picture of the car. Ideally I'd like this kind of report automated on a Cron basis as well. thanks again for the work that went in to making this !

[pageauc]

If NO parameters are supplied for the make speed and totals graphs scripts they will read settings in config.py per matplotlib graph image settings. Graphs are generated per the GRAPH_RUN_LIST and the loop timer is per GRAPH_RUN_TIMER_HOURS You can customize and add to the GRAPH_RUN_LIST to generate the required graphs per the timer schedule. These can be run in the background eg

/home/pi/speed-camera/sql-make-graph-count-totals.py &

This can be added to cron @boot or add command to /etc/rc.local or several other ways
see https://www.dexterindustries.com/howto/run-a-program-on-your-raspberry-pi-at-startup/

You can also generate graphs by providing parameters per the script command help.

 ./sql-make-graph-count-totals.py --help

Loading ver 1.1 DEBUG= False ... usage: sql-make-graph-count-totals.py [-h] -s SPEED -d DAYS -t TOTALS

This program will query the speed camera sqlite3 database per config.py settings or command line parameters. It will then generate matplotlib graph images per selection and grouping values.

Command Line parameters

Selects Records per -s speed over, -d days previous and groups counts by -t totals eg hour, day, month matplotlib will then create jpg graph images to auto created filenames. You can add optional datetime prefix by setting config.py GRAPH_ADD_DATE_TO_FILENAME boolean variable. Default is False (overwrites previous existing filenames)

Graph will show record count totals for specified speeds over and days previous totaled by log_timestamp Grouping eg hour, day, month.

No Command Line Parameters

If NO command line parameters are supplied then multiple graphs can be created per the speed camera config.py GRAPH_RUN_DATA list variable. This can generate multiple graph image files per specified list data criteria. See config.py comments for required GRAPH_RUN_DATA list values.

optional arguments: -h, --help show this help message and exit -s SPEED speed over- Integer - Selects Database Records with Speeds >= specified Value -d DAYS days prev - Integer - Selects Database Records where log_timestamp is between now and specified number of days previous -t TOTALS total by - String - Groups Count Totals by specified string. Valid Strings are: hour day month

Note each command has similar but slightly different parameters. Check help for each for details.

The commands can be put into a bash script and executed as a single crontab entry or added individually to a crontab entry in case you want a different schedule.

Let me know if this helps Claude ...

[pageauc]

Also sorry for the delay but we were visiting my wife's sister at her camp in northern Ontario. No Internet or Cell.

[johncohn]

Claude, Glad that you were able to get some time offline !.. Once again thanks SO MUCH for this excellent package.. I'm really enjoying using it. I'm experimenting with the best way to get some sort of histogram about 'how many cars are going 5,1015,20,+ mph over the speed limit ' Do you know of anyone who's' created anything like that ?

Also.. I'm using the built in reporting and its working really well. i do have some crazy anomalies .. of speeds that are obviously over estimated.. Any idea what might be going one here.. or how i can filter these ?

thanks again for all the work you've put in to this.. Can I buy you a case of beer or something as a thanks ? -jc

Like this

for example... the fastest car picture is here .. clearly not going 152mph :-) obviously not

or this one.. note the pedestrian near the car.. perhaps thats a problem ?

or heres my favorite :-)

John Cohn Ph.D, IBM Fellow , MIT-IBM Watson AI Lab phone 802-578-0985

From: "Claude Pageau" @.> To: "pageauc/speed-camera" @.> Cc: "john cohn" @.>, "Author" @.> Date: 08/19/2021 04:53 PM Subject: [EXTERNAL] Re: [pageauc/speed-camera] Automating graph generation and creating reports for number of cars over a speed threshold (#99)

Also sorry for the delay but we were visiting my wife's sister at her camp in northern Ontario. No Internet or Cell. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or ‍‍‍‍‍‍ Also sorry for the delay but we were visiting my wife's sister at her camp in northern Ontario. No Internet or Cell. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[pageauc]

Summary of Motion Tracking

1 Take Image from Video Stream, Convert to grayscale to reduce computation 2 Take a second image and convert to grayscale. 3 Subtract the two image arrays. If no Motion result will be zero for each pixel and image will be black If motion found there will be a halo around movement since some pixels will be revealed and other covered between images 4 To enhance contours a threshold is used. pixels below threshold are forced to zero and above threshold to 255 (white) 5 To improve contours the image is blurred to merge pixels that are close to each other. 6 Find contours in the enhanced grayscale image and return x, y, w, h of each contour. 7 Find largest contour and record x,y of center of contour. This is the start of a motion tracking sequence.

8. Take another video stream image and repeat using the most recent grayscale image from above. then repeat steps 3 to 7. 9 if the track start and next point are close to each other then record track point otherwise it will be out of range 10 When Track counter reaches end save last colour stream image and record motion track info..

This is a simplified logic for speed camera motion tracking but there are anomalies that can happen that can mess up speed calculation The above works well if the same point on the object is tracked but usually there are multiple contours and sometimes the location of the largest contour shifts. This can change the speed calculation results. Filtering the data can reduce anomalies but sometimes the moving objects are too large in the camera view and therefore there might be quite a few contours. Eg Large vehicles like trucks, trailers or things too close to camera view that take up too much width. Things work best if the moving objects are smaller in the camera view. Should be more than 16 meters. Object can also be too far away so there is a range

I am retired and just wrote it as a fun project. Due to the computing power of the early RPI's it does not use object recognition or AI. It just tracks the largest moving contour.. Lighting, reflections, object shape and colouring or markings, multiple moving objects, Etc can cause anomalies. I often get birds flying close to the camera in the camera view with very fast speeds reported since they are not at the calibration distances. It would be possible to add a filter for an upper speed range. This is not in the current code.

Data anomalies are common with sensors and other computer data collection systems. Filters can help but not sure if you can eliminate everything without a lot of computing power and programming logic.

Hope this explanation helps. BTW I live a pretty healthy lifestyle but thanks for the offer. Claude ....

On Fri, Aug 20, 2021 at 10:31 AM john cohn @.***> wrote:

Claude, Glad that you were able to get some time offline !.. Once again thanks SO MUCH for this excellent package.. I'm really enjoying using it. I'm experimenting with the best way to get some sort of histogram about 'how many cars are going 5,1015,20,+ mph over the speed limit ' Do you know of anyone who's' created anything like that ?

Also.. I'm using the built in reporting and its working really well. i do have some crazy anomalies .. of speeds that are obviously over estimated.. Any idea what might be going one here.. or how i can filter these ?

thanks again for all the work you've put in to this.. Can I buy you a case of beer or something as a thanks ? -jc

Like this

for example... the fastest car picture is here .. clearly not going 152mph :-) obviously not

or this one.. note the pedestrian near the car.. perhaps thats a problem ?

or heres my favorite :-)

John Cohn Ph.D, IBM Fellow , MIT-IBM Watson AI Lab phone 802-578-0985

From: "Claude Pageau" @.> To: "pageauc/speed-camera" @.> Cc: "john cohn" @.>, "Author" @.> Date: 08/19/2021 04:53 PM Subject: [EXTERNAL] Re: [pageauc/speed-camera] Automating graph generation and creating reports for number of cars over a speed threshold (#99)

Also sorry for the delay but we were visiting my wife's sister at her camp in northern Ontario. No Internet or Cell. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or ‍‍‍‍‍‍ Also sorry for the delay but we were visiting my wife's sister at her camp in northern Ontario. No Internet or Cell. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pageauc/speed-camera/issues/99#issuecomment-902734260, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNPKZCE5EWCG42QPMWSXRDT5ZRM3ANCNFSM5CFMLJJA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

-- YouTube Channel at https://www.youtube.com/user/pageaucp http://www.youtube.com/user/pageaucp GitHub Repository at https://github.com/pageauc