Use of AI in order to analyse incidents

[Jamstew2]

In order to help with long term education and reform, we are relying on AI / analytics in order to identify patterns and provide insights which can be turned into actions which can help with education and reform for the public, police and NGO's.

Analysis of incidents: Type of analysis being considered

• Video analysis (entities & description of what is happening in the video)
• Transcript for further Natural Language Processing  (tone analysis, keywords, phrases)
• Enhance a description of the incident based on analysis
• Auto grouping of multiple incident entries (future cross reference with police data for a holistic view)
• Identify patterns in behaviors / utterances which help escalate / de-escalate
• Use the above to highlight helpful / unhelpful behaviors which could be used for further education and reform
• The more data captured via our API, the more accurate the analysis, the stronger the change!

Analysis of incidents in order to….

• Summarise the number of incidents (e.g. Stop & Search)
• Patterns showing specific areas to focus on and why
• The ability for the police to re-strategise, re-educate & re-form

Additional users who may benefit if the app were extended…

• Support services (e.g. mental health, community support)
• Support for people with autism (routine is critical - stop & search could be disruptive and lead to escalations)
• Use example scenarios as the basis for education (e.g. VR recreation)

Our API is key to helping connect services (for example)…..

• People with autism can register for an ID tag
• Imagine if we used our API to integrate with this Database and were able to notify police that someone had certain requirements / alert specially trained support workers.
• We are striving for Equality & Equal justice

[otissmart]

For some the above incident analytics (particularly bullets 1, 2, 5 and 6 of the "Analysis of incidents: Type of analysis being considered" section), we suggest a canonical ML (machine learning) pipeline (see screenshot) to identify insights (predictive patterns with possibly practical significance).

1. Video recordings are the input to the primary analytics (i.e., subsequent processes [2], [3], and [4]). The videos may be separate instances of incidents (which is the typical analysis application) or/and clips of a single incident (which is an alternative analysis approach).
2. Each video undergoes decomposition (isolation) into three time-synchronous data modalities: imagery (image frames over time), speech (audio over time), and text (via conversion of the same audio signal to text sentences).
3. Each data modality undergoes respective analysis to compute corresponding variables, whereby each variable has time-series form (i.e., keep one value per time epoch) or aggregate form (i.e., reduce a time series to one value for all time epochs with a germane mathematical function such as but not limited to measures of central tendency, measures of dispersion, or/and other statistics). The variables from the three analysis domains get concatenation into a single unified dataframe for input of the ensuing statistical analytics and machine-learning analytics in [4].
4. The descriptive and inferential statistics encompass hypothesis-testing analytics to quantitatively investigate inquiries such as possible similarities or/and disparities across demographics, geographies, or/and other considerable factors. The machine-learning analytics encompass techniques to cluster data, group variables, make predictions, discern important variables of those predictions, or/and more. All results can comprise spatiotemporal visualizations in a dashboard.
5. The dashboard findings may have display in various IoT (internet of things) devices such as mobile phones, tablets, desktops, laptops, vehicles, televisions, city projectors, wearables, aware homes, or/and other technology.

Additionally, we provide the following helpful hyperlinks also as a suggestion rather than requirement for implementation, particularly for the "variables extraction" (aka "feature extraction" aka "feature engineering") pipeline component whereby one may use techniques such as audio spectral analysis or/and audio-to-text sentiment analysis as predictive variables. For instance, python need not be the chosen development language; moreover, IBM tools are not a necessity and any such technology has only a "for demonstration purpose" intention and not a "for prescribed solution purpose" intent.

• https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html
• https://pjreddie.com/darknet/yolo/
• https://kkroening.github.io/ffmpeg-python/
• http://www.spectralpython.net/
• https://pypi.org/project/spectrum/
• https://www.audiocontentanalysis.org/python/
• https://www.kdnuggets.com/2020/02/audio-data-analysis-deep-learning-python-part-1.html
• https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0144610
• https://medium.com/swlh/sentiment-analysis-using-python-and-nltk-library-d68caba27e1d
• https://towardsdatascience.com/a-beginners-guide-to-sentiment-analysis-in-python-95e354ea84f6
• https://towardsdatascience.com/natural-language-processing-nlp-for-machine-learning-d44498845d5b
• file:///Users/otissmart/Downloads/ICCV17_EmotionalArcs_Final.pdf
• https://developer.ibm.com/?s=streaming+video&orderby=date&order=desc&category_name=&type=
• https://developer.ibm.com/articles/text-mining-and-analysis-from-webex-recordings/
• https://github.com/IBM/use-advanced-nlp-and-tone-analyser-to-analyse-speaker-insights/blob/master/README.md
• https://en.wikipedia.org/wiki/Natural_language_processing
• https://en.wikipedia.org/wiki/Voice_analysis
• https://en.wikipedia.org/wiki/Time_series
• https://en.wikipedia.org/wiki/Multimodal_sentiment_analysis

[github-actions[bot]]

:wave: Hi! This issue has been marked stale due to inactivity. If no further activity occurs, it will automatically be closed.