GP Representation turns flat even for constant noise

[FrankD]

I used the following settings (on the test data in run_script.R):

agm(data=dataTest,time=timeTest,ode.system=LV_func,numberOfParameters=4,temperMismatchParameter=TRUE,
   maxIterations=100000,originalSignalOnlyPositive=TRUE, showProgress=TRUE, noiseInfer=FALSE, noiseFixed=0.1, 
   defaultPrior="Gamma",defaultTemperingScheme="LB10")

With fixed noise, this shouldn't be happening, should it?

I'll try to investigate, but let me know if you have any ideas?

[BennMacdonald]

What is the true noise in the system?

[FrankD]

0.31 I think, do you suspect the fixed noise is somehow too restrictive? But I don't understand why that would cause it to jump to a state with lower log likelihood.

[BennMacdonald]

That is very strange. Flattening usually occurs when the noise can be pushed to levels higher than the true value (as long as the true value is not extremely high already). Also, the log likelihood usually jumps to higher values, not lower when this happens.

Just out of interest, is 0.31 the true SD of the observational noise, not the variance? Is LV_func the Lotka-Volterra system of equations I had originally uploaded?

[BennMacdonald]

I am looking at the files just now. I can see that LV_func is newly defined and that 0.31 is SD not variance. Will comment back when I test a few things.

[FrankD]

Thanks Benn, it's always possible that I introduced some bug. For what it's worth, the explicit solution MCMC seems to work for this example.

[BennMacdonald]

No problem Frank. Can you help me out with something? I am unable to use the load_all function. My working directory is AGM_RPackage-master and within there is deGradInfer and inside there is R. I thought load_all('deGradInfer/') should work correctly from there, but R cannot find it

"Error: Can't find 'C:\Users\0702424k\Documents\R Package\Ongoing GitHub\AGM_RPackage-master\AGM_RPackage-master\deGradInfer\'."

Thoughts on what I am doing incorrectly?

P.s. I am in the "AGM_RPackage-master" directly above "deGradInfer"

[BennMacdonald]

P.p.s I think I know what the problem is, but I want to run it and confirm.

[FrankD]

Try it without the slash at the end: load_all('deGradInfer') The forward slash is a unix directory divider, so it might not work on Windows. Also I notice your path has two nested directories with the same name, is that correct?

[BennMacdonald]

Yeah, it was the slash, thank you. The double nested directories is because of how winrar opens zip files - you end up with two names the same name. Will run this and get back to you.

[BennMacdonald]

It is taking a long time to install the necessary packages and I must leave to catch a bus soon(ish). I will outline a couple of things I have noticed that might be the/an issue.

1. in ode.R, the function "getODEGradient" creates XODE, but it appears that X is passed on to auxVars$ode.system instead. Can you check this is not the cause of the error?

2. Now that the argument "explicit" has been added to the function agm, we must be careful of

   dataConstants <- matrix(,nrow=nrow(data),ncol=ncol(data)) for (i in 1:ncol(dataConstants)) { dataConstants[,i] <- mean(data[,i]) }

   dataset <- data - dataConstants

inside the agm function. We probably need to put an IF statement around the dataset <- data-dataConstants, so that we don't end up with zero mean data when we use the explicit solution method. If we compare an ODE solution to data that has the mean subtracted, we would assign a small likelihood to this (unless the ODE solution happened to have zero mean).

Can you change those on your side and see if you get the same problem? I would also like to check that auxVars$originalPositive is also doing its job properly and not just setting the signal to 0 (which would look like flattening). If 1. above does not solve the problem, can you make auxVars$originalPositive to FALSE and see if you get the same output. When these packages finally install I will try it myself.

[BennMacdonald]

Managed to get the packages installed. Will play around with what I mentioned above and let you know what happens. I am first going to see if I can reproduce what you got.

[FrankD]

Argh, you're right about XODE. Completely my fault, I forgot about the offset when I wrote that bit of code (even though I should have realized that it would be an issue).

I'll fix it now, unless you're currently working on it?

On 3 May 2017 at 17:42, BennMacdonald notifications@github.com wrote:

Managed to get the packages installed. Will play around with what I mentioned above and let you know what happens. I am first going to see if I can reproduce what you got.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-298967260, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAstpn5j0Cw4IpDGUVROlR48-gROLakks5r2K5bgaJpZM4NPb6Q .

[BennMacdonald]

No problem :) I am testing to see if the problem goes away, but won't know until tomorrow as off for my Birthday and at my parent's. I would say though, potentially the fixed sd might not be enough. We might need to implement the band around an initial GP to do the trick. Going to test with the old code under the same conditions to see if it is still present. Will let you know.

Regards, Benn

University Of Glasgow School Of Mathematics And Statistics University Gardens Room 303 G12 8QW

----- Reply message ----- From: "FrankD" notifications@github.com To: "BennMacdonald/AGM_RPackage" AGM_RPackage@noreply.github.com Cc: "BennMacdonald" Benn.Macdonald@glasgow.ac.uk, "Assign" assign@noreply.github.com Subject: [BennMacdonald/AGM_RPackage] GP Representation turns flat even for constant noise (#8) Date: Thu, May 4, 2017 14:02

Argh, you're right about XODE. Completely my fault, I forgot about the

offset when I wrote that bit of code (even though I should have realized

that it would be an issue).

I'll fix it now, unless you're currently working on it?

On 3 May 2017 at 17:42, BennMacdonald notifications@github.com wrote:

Managed to get the packages installed. Will play around with what I

mentioned above and let you know what happens. I am first going to see if I

can reproduce what you got.

—

You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub

https://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-298967260,

or mute the thread

https://github.com/notifications/unsubscribe-auth/AAAstpn5j0Cw4IpDGUVROlR48-gROLakks5r2K5bgaJpZM4NPb6Q

.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/BennMacdonald/AGM_RPackage","title":"BennMacdonald/AGM_RPackage","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/BennMacdonald/AGM_RPackage"}},"updates":{"snippets":[{"icon":"PERSON","message":"@FrankD in #8: Argh, you're right about XODE. Completely my fault, I forgot about the\noffset when I wrote that bit of code (even though I should have realized\nthat it would be an issue).\n\nI'll fix it now, unless you're currently working on it?\n\nOn 3 May 2017 at 17:42, BennMacdonald \u003cnotifications@github.com\u003e wrote:\n\n\u003e Managed to get the packages installed. Will play around with what I\n\u003e mentioned above and let you know what happens. I am first going to see if I\n\u003e can reproduce what you got.\n\u003e\n\u003e —\n\u003e You are receiving this because you were assigned.\n\u003e Reply to this email directly, view it on GitHub\n\u003e \u003chttps://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-298967260\u003e,\n\u003e or mute the thread\n\u003e \u003chttps://github.com/notifications/unsubscribe-auth/AAAstpn5j0Cw4IpDGUVROlR48-gROLakks5r2K5bgaJpZM4NPb6Q\u003e\n\u003e .\n\u003e\n"}],"action":{"name":"View Issue","url":"https://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-299178641"}}}

[FrankD]

Ok, thanks Benn. And happy birthday!

Frank

On 4 May 2017 at 15:43, BennMacdonald notifications@github.com wrote:

No problem :) I am testing to see if the problem goes away, but won't know until tomorrow as off for my Birthday and at my parent's. I would say though, potentially the fixed sd might not be enough. We might need to implement the band around an initial GP to do the trick. Going to test with the old code under the same conditions to see if it is still present. Will let you know.

Regards, Benn

University Of Glasgow School Of Mathematics And Statistics University Gardens Room 303 G12 8QW

----- Reply message ----- From: "FrankD" notifications@github.com To: "BennMacdonald/AGM_RPackage" AGM_RPackage@noreply.github.com Cc: "BennMacdonald" Benn.Macdonald@glasgow.ac.uk, "Assign" < assign@noreply.github.com> Subject: [BennMacdonald/AGM_RPackage] GP Representation turns flat even for constant noise (#8) Date: Thu, May 4, 2017 14:02

Argh, you're right about XODE. Completely my fault, I forgot about the

offset when I wrote that bit of code (even though I should have realized

that it would be an issue).

I'll fix it now, unless you're currently working on it?

On 3 May 2017 at 17:42, BennMacdonald notifications@github.com wrote:

Managed to get the packages installed. Will play around with what I

mentioned above and let you know what happens. I am first going to see if I

can reproduce what you got.

—

You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub

https://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment- 298967260,

or mute the thread

https://github.com/notifications/unsubscribe-auth/ AAAstpn5j0Cw4IpDGUVROlR48-gROLakks5r2K5bgaJpZM4NPb6Q

.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":" 05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity": {"external_key":"github/BennMacdonald/AGM_RPackage"," title":"BennMacdonald/AGM_RPackage","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/ 143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png"," avatar_image_url":"https://cloud.githubusercontent.com/ assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png ","action":{"name":"Open in GitHub","url":"https://github. com/BennMacdonald/AGM_RPackage"}},"updates":{"snippets":[{" icon":"PERSON","message":"@FrankD in #8: Argh, you're right about XODE. Completely my fault, I forgot about the\noffset when I wrote that bit of code (even though I should have realized\nthat it would be an issue).\n\nI'll fix it now, unless you're currently working on it?\n\nOn 3 May 2017 at 17:42, BennMacdonald \u003cnotifications@github.com\u003e wrote:\n\n\u003e Managed to get the packages installed. Will play around with what I\n\u003e mentioned above and let you know what happens. I am first going to see if I\n\u003e can reproduce what you got.\n\u003e\n\u003e —\n\u003e You are receiving this because you were assigned.\n\u003e Reply to this email directly, view it on GitHub\n\u003e \u003chttps:// github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment- 298967260\u003e,\n\u003e http://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-298967260%5Cu003e,%5Cn%5Cu003e or mute the thread\n\u003e \u003chttps://github.com/ notifications/unsubscribe-auth/AAAstpn5j0Cw4IpDGUVROlR48- gROLakks5r2K5bgaJpZM4NPb6Q\u003e\n\u003e http://github.com/notifications/unsubscribe-auth/AAAstpn5j0Cw4IpDGUVROlR48-gROLakks5r2K5bgaJpZM4NPb6Q%5Cu003e%5Cn%5Cu003e .\n\u003e\n"}],"action":{"name":"View Issue","url":"https://github. com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-299178641"}}}

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-299206167, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAsttvgH3AZXQOJBvrMezleVvottZhoks5r2eQDgaJpZM4NPb6Q .

[BennMacdonald]

Hi Frank. Still doing a couple of checks on this, was at a conference last week. Will send an update through soon.

[BennMacdonald]

Hi Frank. I can confirm that the problem appears to be XODE not being passed inside the function ode.R and passing X instead.

The first image passes X instead of XODE. As you can see, flattening occurs at roughly 11% into the MCMC.

The second image passes XODE instead of X, using the same seed as for the first image. Flattening does not occur. I repeated this again when setting originalSignalOnlyPositive=FALSE, in case this was having some combined effect. The results are identical to originalSignalOnlyPositive=TRUE, so we can rule this out.

I will run things again, but for 1,000,000 iterations in case for some reason flattening is just occurring much later. If we see the same results as here, we can conclude with reasonable confidence that the problem was XODE not being passed through. I will update you if there is a change and close the issue if there is no change.

[FrankD]

Thanks for looking into it Benn, it does sound like your initial thought was correct!

On 16/05/2017 14:40, BennMacdonald wrote:

Hi Frank. I can confirm that the problem appears to be XODE not being passed inside the function ode.R and passing X instead.

image https://cloud.githubusercontent.com/assets/26483161/26108598/ecc426d4-3a44-11e7-90ed-8c2a5bb6326e.png

The first image passes X instead of XODE. As you can see, flattening occurs at roughly 11% into the MCMC.

image https://cloud.githubusercontent.com/assets/26483161/26108642/0d45cb4c-3a45-11e7-902f-fa4c546b2169.png

The second image passes XODE instead of X, using the same seed as for the first image. Flattening does not occur. I repeated this again when setting originalSignalOnlyPositive=FALSE, in case this was having some combined effect. The results are identical to originalSignalOnlyPositive=TRUE, so we can rule this out.

I will run things again, but for 1,000,000 iterations in case for some reason flattening is just occurring much later. If we see the same results as here, we can conclude with reasonable confidence that the problem was XODE not being passed through. I will update you if there is a change and close the issue if there is no change.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-301785359, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAstluT5Y8NE8CeI15A0GhfxHnH-GdGks5r6adKgaJpZM4NPb6Q.

[BennMacdonald]

Thanks very much, Frank. I can confirm that for 1 million iterations, flattening does not occur once we use XODE and fixed noise. Do you want me to push the updated ode.R file to Github?

[FrankD]

Yes please!

On 18 May 2017 at 15:50, BennMacdonald notifications@github.com wrote:

Thanks very much, Frank. I can confirm that for 1 million iterations, flattening does not occur once we use XODE and fixed noise. Do you want me to push the updated ode.R file to Github?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BennMacdonald/AGM_RPackage/issues/8#issuecomment-302427595, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAstqV2c5VYXeqFQB07B32GmsHDb70sks5r7FrAgaJpZM4NPb6Q .

[BennMacdonald]

Done :) Will close issue now