prop at collision height

[MarkTrinder]

I am trying to replicate Excel outputs using the stochastic model, but with all stochasticity turned off (i.e. entering 0 for all the SDs etc and providing fixed flight height data values). I have checked to make sure the input data are identical for both model formulations: the flight height data are the same in both the sCRM input data and the Band sheet ('flightheight') and I have set the same tidal offset value. I can't see any other sources of possible difference between the two models and I am getting single deterministic outputs for each month (i.e. no variations).

I can only obtain results which correspond to Excel Op2 outputs from sCRM if I set the sCRM op1 PCH value to the one calculated by Excel for op2 (from cell m45 in the 'extended'). The sCRM output for op2, which should be the same if the calculation is conducted the same way, is 10% lower. This suggests to me that the way the op2 pch value is being calculated in sCRM is different from that used in the original Excel sheet. As stated above, as far as I can tell the input data are identical so I think this has to be in the calculations.

It would be good to understand why this discrepancy is occurring and which model is 'right'!

[thomasevans]

I have just had a quick look over the code. I think the issue may be that the Op2 script (https://github.com/dmpstats/stochCRM/blob/master/scripts/Option2.r) is incorrectly calculating risk in each part of the rotor (i.e. what option 3 should do), see lines 17-23. That would explain an underestimation of risk, as only the rotor-swept area (circle) is included, not a square for the whole PCH height zone. I think this could be rectified by editing the Option2.r script, probably by adopting some of the code from the Option1.r script.

[thomasevans]

Having considered this further, after discussing with Mark, I don't think the issue is what I have suggested above. I think it is something else - though unsure what!

[statsgeeknz]

Hi Mark/all. Apologies for the delay - could you tell me the species you're looking at, so I can replicate exactly?

We did find discrepancies between the Masden original code and Band, but thought that had been rectified down to minor differences. I'll run your example through the three and see what matches. I can debug whatever is required from then.

Regards C

[MarkTrinder]

Hi

It was kittiwake I ran the comparison on. I was getting a c. 10% difference, so not a rounding-type error.

Thanks Mark

From: Statsgeek notifications@github.com Sent: 30 September 2019 13:01 To: dmpstats/stochCRM stochCRM@noreply.github.com Cc: Mark Trinder Mark.Trinder@macarthurgreen.com; Author author@noreply.github.com Subject: Re: [dmpstats/stochCRM] prop at collision height (#13)

Hi Mark/all. Apologies for the delay - could you tell me the species you're looking at, so I can replicate exactly?

We did find discrepancies between the Masden original code and Band, but thought that had been rectified down to minor differences. I'll run your example through the three and see what matches. I can debug whatever is required from then.

Regards C

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/dmpstats/stochCRM/issues/13?email_source=notifications&email_token=AGMMRDLE3QCFF74PCV4SNMDQMHTA3A5CNFSM4IS6YRQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD75MYYQ#issuecomment-536530018, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGMMRDIGGT3SJ5I7WGOZHV3QMHTA3ANCNFSM4IS6YRQQ.

[statsgeeknz]

Hi Mark/all. One last request I think.

Can you indicate the source of your Band spreadsheet and which flight-height data you have used e.g. the Masden FHD for kittiwake then ported to the Band spreadsheet or vice-versa?

I appreciate that this may not matter, but the source of the problem may be unexpected e.g. something to do with the mismatch of FHD altitude coverage.

Regards C

[MarkTrinder]

Hi

These are straight off the SOSS website:

Flight height data from here: https://www.bto.org/sites/default/files/u28/downloads/Projects/Final_Report_SOSS02_FlightHeights2014.xls

Band CRM from here: https://www.bto.org/sites/default/files/u28/downloads/Projects/Final_Report_SOSS02_Band2Tool.xlsm

For Band I copied the kittiwake flight height column into the relevant tab of the spreadsheet and for sCRM I copied and duplicated it (however many times, 200?) into the template provided.

Thanks m

From: Statsgeek notifications@github.com Sent: 01 October 2019 13:30 To: dmpstats/stochCRM stochCRM@noreply.github.com Cc: Mark Trinder Mark.Trinder@macarthurgreen.com; Author author@noreply.github.com Subject: Re: [dmpstats/stochCRM] prop at collision height (#13)

Hi Mark/all. One last request I think.

Can you indicate the source of your Band spreadsheet and which flight-height data you have used e.g. the Masden FHD for kittiwake then ported to the Band spreadsheet or vice-versa?

I appreciate that this may not matter, but the source of the problem may be unexpected e.g. something to do with the mismatch of FHD altitude coverage.

Regards C

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/dmpstats/stochCRM/issues/13?email_source=notifications&email_token=AGMMRDKDUWAHWXKHSXID5ZTQMM7FRA5CNFSM4IS6YRQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEABC3MY#issuecomment-537013683, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGMMRDNQNPTOBWHWCLQSAGLQMM7FRANCNFSM4IS6YRQQ.

[statsgeeknz]

Hi Mark. Thanks - exactly what I have done previously. Albeit not for this exact case. Will report back - bit slow sorry, other projects nagging.

Regards C

[statsgeeknz]

Hi Mark/others. OK - I replicated this as far as possible comparing the original Masden stoch-CRM code and the Band spreadsheet you indicated. First pass through, I found what you indicated. However, it is really sensitive to the nocturnal activity parameter and I'm not sure how those are to be made comparable. In the Band sheet it indicates "Nocturnal activity factor (1-5)". For the stoch-CRM we have a mean of 0.033 (plus some variance - turned off).

So, can't really figure out if there is a problem without clarity on how these are to be set. I've not dug deeper yet - I suspect someone has the answer immediately in hand.

Regards C

[MarkTrinder]

Hi

The Band 1-5 is just converted in the spreadsheet to a value between 0-1 (1=0, 5=1) so it is straightforward to use the same value in both – for kittiwake set the Band value to 3 and in the sCRM use 0.5.

M

From: Statsgeek notifications@github.com Sent: 04 October 2019 10:02 To: dmpstats/stochCRM stochCRM@noreply.github.com Cc: Mark Trinder Mark.Trinder@macarthurgreen.com; Author author@noreply.github.com Subject: Re: [dmpstats/stochCRM] prop at collision height (#13)

Hi Mark/others. OK - I replicated this as far as possible comparing the original Masden stoch-CRM code and the Band spreadsheet you indicated. First pass through, I found what you indicated. However, it is really sensitive to the nocturnal activity parameter and I'm not sure how those are to be made comparable. In the Band sheet it indicates "Nocturnal activity factor (1-5)". For the stoch-CRM we have a mean of 0.033 (plus some variance - turned off).

So, can't really figure out if there is a problem without clarity on how these are to be set. I've not dug deeper yet - I suspect someone has the answer immediately in hand.

Regards C

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/dmpstats/stochCRM/issues/13?email_source=notifications&email_token=AGMMRDPDLUNU57QLQG26TKDQM4A7LA5CNFSM4IS6YRQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEALASQA#issuecomment-538315072, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGMMRDPWJ2GXYS2GP4YVJULQM4A7LANCNFSM4IS6YRQQ.

[statsgeeknz]

Thanks Mark. OK - I also find non-negligible differences between the Band and orginal stoch-CRM code. Can you send me your Band spreadsheet so we're all on the same page?

The Band sheet we worked to originally didn't have migration options - but I'm assuming this isn't an issue if it's just set to zero.

Regards C

[MarkTrinder]

Hi

The spreadsheet version I used is the one on the SOSS website, but I don’t think I should send you the data I was using as that is from a wind farm still in the application process, sorry. It shouldn’t matter what density values you use though – the key is to be consistent across both model versions, right? The migration output runs through a separate process so won’t affect the comparison you are looking at.

M

From: Statsgeek notifications@github.com Sent: 04 October 2019 10:32 To: dmpstats/stochCRM stochCRM@noreply.github.com Cc: Mark Trinder Mark.Trinder@macarthurgreen.com; Author author@noreply.github.com Subject: Re: [dmpstats/stochCRM] prop at collision height (#13)

Thanks Mark. OK - I also find non-negligible differences between the Band and orginal stoch-CRM code. Can you send me your Band spreadsheet so we're all on the same page?

The Band sheet we worked to originally didn't have migration options - but I'm assuming this isn't an issue if it's just set to zero.

Regards C

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/dmpstats/stochCRM/issues/13?email_source=notifications&email_token=AGMMRDLOJ7KBKN4RHM7PRNTQM4EQTA5CNFSM4IS6YRQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEALDBPA#issuecomment-538325180, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGMMRDPYT4SMMVWURUIRFJDQM4EQTANCNFSM4IS6YRQQ.

[statsgeeknz]

Hi Mark. Sure - I'm just mindful that there are a lot of values that need to be matched/altered. One thing out of place and the results are all over the shop. So it is mainly QA.

We found and fixed (I think) one discrepancy between the Band and Masden calculations. I'm getting around 17% differences from the Masden code - I'll see what the app gives, could be closer. Obviously the app code is broadly similar to the calculations in Masden, so if that isn't giving results the same as Band, neither will the app.

Regards C

[MarkTrinder]

Totally agree that the challenge is to input the same values – I was #fairly# confident I had achieved that before I posted this, but there is always a risk of missing something.

I am also slightly hampered in this now as due to a new locked-down PC I can only run sCRM online and can’t actually use the Band spreadsheet either as the poptools addin for excel doesn’t work on 64 bit machines!

From: Statsgeek notifications@github.com Sent: 04 October 2019 10:58 To: dmpstats/stochCRM stochCRM@noreply.github.com Cc: Mark Trinder Mark.Trinder@macarthurgreen.com; Author author@noreply.github.com Subject: Re: [dmpstats/stochCRM] prop at collision height (#13)

Hi Mark. Sure - I'm just mindful that there are a lot of values that need to be matched/altered. One thing out of place and the results are all over the shop. So it is mainly QA.

We found and fixed (I think) one discrepancy between the Band and Masden calculations. I'm getting around 17% differences from the Masden code - I'll see what the app gives, could be closer. Obviously the app code is broadly similar to the calculations in Masden, so if that isn't giving results the same as Band, neither will the app.

Regards C

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/dmpstats/stochCRM/issues/13?email_source=notifications&email_token=AGMMRDMX6GBIZOFCRKCOIT3QM4HTRA5CNFSM4IS6YRQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEALFEWQ#issuecomment-538333786, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGMMRDPRWMMWOA76YE56QP3QM4HTRANCNFSM4IS6YRQQ.

[statsgeeknz]

@MarkTrinder - that doesn't sound like fun.

We're pretty snowed under for the next two weeks, but I'll see if I can't get to the bottom of it before then. Be able to sort it thereafter for sure.

[statsgeeknz]

Hi Mark/All. Finally got around to this. The bug stems from two things in the original Masden code:

• the first row of the flight height distributions gets cut off, so FHD is shifted by 1 m (minor issue).
• double accounting of the hub-height, shifting the part of the FHD used in calculations by say 50m (major issue).

With these alterations, we get within 0.1/0.01 of the Band option 2 count results. Mutliple rounding and minor stochasticity means I don't expect 100% agreement. They're rounded to counts, so maybe occasionally disagree by 1. I'm going to make a repo with the original Masden code and make this correction. The rest will be as originally written (caveat emptor) - it can be a community project for those interested. We'll also propagate the correction into the CRM app. That should appear soon.

Regards C

[statsgeeknz]

Hi all. A "final" update on this.

• There is now a repo for the original Masden code, with minor bug fixes, and code to perform a comparison with the Band spreadsheet outputs. They are effectively the same.
• Minor fixes propagated here to the local Shiny code. There is also the facility to compare with Band outputs - again, effectively the same.

In summation, there were 3 issues:

1. An error in the original Masden code with regards up and downwind calculations making small differences in option 3 outputs
2. A shift of the FHD by 1m due to clipping off the first value, giving small differences.
3. Uploading a user-supplied bootstrap FHD was failing silently, so would revert to the BTO files, which might be quite different from what is expected.

Generally, any large differences between Band and R versions were due to not getting the parameterisations identical, or failing to suppress the stochasticity - hence I've supplied the files for Kittiwake for comparisons. Differences in monthly collision estimates are <0.5% for options 1-3 with all things being equal.

The user-supplied FHD bootstraps ought to be functioning soon, and all changes propagated to the online version.