Quantify difference in responses

jerbaroo commented 5 years ago

Given the responses collected at different points and under different scenarios in #101

Determine some measure to quantify the difference in these distributions

jerbaroo commented 4 years ago

Distribution of displacement at different points under normal traffic.

distributions-normal-lam-5-y-translation-normal-lane-0

Distribution of displacement at different points under normal traffic. Left-most pier is displaced by 1mm. Notice how the displacement is different for low values of x, compared to previous image. The label shows the value of a chi squared test comparing each plot to the respective one from the image above.

distributions-normal-lam-5-y-translation-displacement-0,001m-pier-0-lane-0

rozsasarpi commented 4 years ago

Is it the chi-squared statistics what is shown on each figure?

jerbaroo commented 4 years ago

Is it the chi-squared statistics what is shown on each figure?

Yes exactly, though I think I may have applied the input to the test incorrectly. Will check.

jerbaroo commented 4 years ago

@rozsasarpi I wanted to get your opinion on a little experiment.

Healthy scenario

Consider sensors distributed evenly over the deck.
Normal traffic is sent over the deck.
Distribution of responses then exists for each sensor (as a vector).
Normalize each distribution (maybe by subtracting the smallest vector Vs).
Now have normalized distribution of responses over the deck. (1)

Damage scenario

Displace a pier by 1mm (or apply concrete cracked damage when complete).
Run normal traffic and record responses and normalize. (2)
Compare (1) and (2) over the deck.
Detect location of damage.

I think this kind of information about how the responses change over the deck, in particular compared to the baseline state, could be useful in a DSS. Such a function would provide useful information in regards "if there is damage it is here" e.g. damage location. Which can be combined with other more complex detection functions later.

rozsasarpi commented 4 years ago

I like the general idea. Be careful with using the pier settlement as damage case in combination of displacement measuring sensors: it is quite trivial to detect in this setup (with a sensor above each pier). You could also indicate those sensor locations which would trigger a warning/notification according to some anomaly measure.
Normalization: why not just using standardization? xs = (x - mean(x))/std(x)

rozsasarpi commented 4 years ago

Is it the chi-squared statistics what is shown on each figure?

Yes exactly, though I think I may have applied the input to the test incorrectly. Will check.

I think it would be more informative if you displayed the corresponding p-value. In any case be cautious with any null hypothesis testing. Did you have a course (or learned by yourself) on statistics and hypothesis testing? If not, see this for example.

You might want to consider the Kolmogorov-Smirnov statistics and Kullback-Leibler divergence as well.

jerbaroo commented 4 years ago

Okay I will look at these measures, I was considering KS. Valeria also suggested one based on the angle between vectors. I am trying out the method I presented above under different pier displacement scenarios. Here is a plot showing displacement of the bridge deck when piers are displaced by 0.01m for the left piers, to 0.06m for the right piers, a linear gradient.

Note, no quantification here yet. Just a scenario to look at for quantification as it affects the whole bridge.

pier-1

jerbaroo commented 4 years ago

This shows a start on the method I suggested. Comparing sensors under healthy and damaged scenarios, and plotting the difference between each pair of sensors. The damage scenario in this case was displacement of the 1 pier closest to the large blue are. Note that all labels and things like that are wrong in this plot, it is just rough work, please don't analyze it.

Still working on trying different quantification measures and how to best normalize the data, at each point.

jerbaroo commented 4 years ago

I refined this "snapshot comparison" method. It's now based on the idea that similar sensors should remain similar (similar idea to the Sydney Harbour Bridge paper), rather than before where I was just comparing a sensor to itself.

Method:

In healthy scenario: for each pair of sensors: compare responses with kolmogorov smirnov
In damage scenario: for each pair of sensors: compare responses with kolmogorov smirnov
For each pair of sensors: find how much this comparison statistic has changed (D_p0_p1)
For each sensor p0: sum over D_p0_pi

I think this method makes sense in a DSS to show where responses of the bridge have changed relative to the global change.

If the entire bridge is shifted by some value, this will not highlight any change (I think, to be shown).

Probably the responses (that each KS 2 sample test is based on) should have outliers removed, otherwise the KS D statistic may be really high, even if there is only one point with large difference.