lfiondella / SRT

Software Reliabilty Testing
5 stars 9 forks source link

Model Result Plot Curves Could Be More Accurate #74

Open a1729nikora opened 8 years ago

a1729nikora commented 8 years ago

When plotting model results on Tab 2, the lines that are drawn aren't always good representations of the model results. This is particularly true when the points representing model predictions into the future are widely spaced, and the straight lines that are drawn between the points don't give a good idea of what the model actually predicted.

a1729nikora commented 8 years ago

Suggest redoing the functionality in "PlotModelResults.R" to redraw the model result lines at a much finer resolution than is done at present. I've been experimenting in my local development branch. I can draw more accurate curves and extend them arbitrarily far into the future. It requires recomputing the model results (but not the model parameters) when the plot is redrawn, but it doesn't seem to cause a big delay.

arhik commented 8 years ago

You mean interpolation of data points by a polynomial instead of straight line.

On Wed, Oct 28, 2015 at 8:47 PM, a1729nikora notifications@github.com wrote:

Suggest redoing the functionality in "PlotModelResults.R" to redraw the model result lines at a much finer resolution than is done at present. I've been experimenting in my local development branch. I can draw more accurate curves and extend them arbitrarily far into the future. It requires recomputing the model results (but not the model parameters) when the plot is redrawn, but it doesn't seem to cause a big delay.

— Reply to this email directly or view it on GitHub.

a1729nikora commented 8 years ago

Actually, no – I guess I didn’t explain it well. For drawing lines, what I’m thinking is dividing the interval to be plotted into very small intervals, estimating the endpoints of those intervals using the models (whose parameters have already been determined), and then connecting the endpoints of those very small intervals with straight lines. That gives a pretty good visual approximation of the correct curve in all circumstances I’ve tried so far.

-Allen N.

Allen P. Nikora, Ph.D. Program Element Manager, Assurance Technology Program Office Software Element

Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 Mail Stop 125-233

vox: 818.393.1104 fax: 818.393.1362 email: Allen.P.Nikora@jpl.nasa.gov

From: Karthik Katipally notifications@github.com<mailto:notifications@github.com> Reply-To: lfiondella/SRT reply@reply.github.com<mailto:reply@reply.github.com> Date: Wednesday, October 28, 2015 at 6:42 PM To: lfiondella/SRT SRT@noreply.github.com<mailto:SRT@noreply.github.com> Cc: Allen Nikora Allen.P.Nikora@jpl.nasa.gov<mailto:Allen.P.Nikora@jpl.nasa.gov> Subject: Re: [SRT] Model Result Plot Curves Could Be More Accurate (#74)

You mean interpolation of data points by a polynomial instead of straight line.

On Wed, Oct 28, 2015 at 8:47 PM, a1729nikora notifications@github.com<mailto:notifications@github.com> wrote:

Suggest redoing the functionality in "PlotModelResults.R" to redraw the model result lines at a much finer resolution than is done at present. I've been experimenting in my local development branch. I can draw more accurate curves and extend them arbitrarily far into the future. It requires recomputing the model results (but not the model parameters) when the plot is redrawn, but it doesn't seem to cause a big delay.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/lfiondella/SRT/issues/74#issuecomment-152048788.

arhik commented 8 years ago

Okay. But i am thinking of points only option - too many dots on the plot will not be appealing. Users may be interested in actual data points and its corresponding model estimate. I could be wrong here but i point this here just in case you were not thinking about it. Lets wait for professors view on it too.

If there is a way to mask the mini intervals from ggplot plot function then it is good to implement it.

@lfiondella What do you think professor ?

On Wed, Oct 28, 2015 at 9:46 PM, a1729nikora notifications@github.com wrote:

Actually, no – I guess I didn’t explain it well. For drawing lines, what I’m thinking is dividing the interval to be plotted into very small intervals, estimating the endpoints of those intervals using the models (whose parameters have already been determined), and then connecting the endpoints of those very small intervals with straight lines. That gives a pretty good visual approximation of the correct curve in all circumstances I’ve tried so far.

-Allen N.

Allen P. Nikora, Ph.D. Program Element Manager, Assurance Technology Program Office Software Element

Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 Mail Stop 125-233

vox: 818.393.1104 fax: 818.393.1362 email: Allen.P.Nikora@jpl.nasa.gov

From: Karthik Katipally notifications@github.com<mailto:notifications@github.com> Reply-To: lfiondella/SRT reply@reply.github.com<mailto:reply@reply.github.com> Date: Wednesday, October 28, 2015 at 6:42 PM To: lfiondella/SRT SRT@noreply.github.com<mailto:SRT@noreply.github.com> Cc: Allen Nikora Allen.P.Nikora@jpl.nasa.gov<mailto:Allen.P.Nikora@jpl.nasa.gov> Subject: Re: [SRT] Model Result Plot Curves Could Be More Accurate (#74)

You mean interpolation of data points by a polynomial instead of straight line.

On Wed, Oct 28, 2015 at 8:47 PM, a1729nikora notifications@github.com<mailto:notifications@github.com> wrote:

Suggest redoing the functionality in "PlotModelResults.R" to redraw the model result lines at a much finer resolution than is done at present. I've been experimenting in my local development branch. I can draw more accurate curves and extend them arbitrarily far into the future. It requires recomputing the model results (but not the model parameters) when the plot is redrawn, but it doesn't seem to cause a big delay.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/lfiondella/SRT/issues/74#issuecomment-152048788. — Reply to this email directly or view it on GitHub.

a1729nikora commented 8 years ago

It’s probably easier just to show you all on Friday. I’ve attached a plot that shows some of it. This is a lines-only plot where I’ve allowed the lines to extend 30,000 time units past the end of the last prediction point I’ve specified for the model (I asked to predict 8 failures into the future). You can extend the lines out as far as you want.

-Allen N.

Allen P. Nikora, Ph.D. Program Element Manager, Assurance Technology Program Office Software Element

Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 Mail Stop 125-233

vox: 818.393.1104 fax: 818.393.1362 email: Allen.P.Nikora@jpl.nasa.gov

From: Karthik Katipally notifications@github.com<mailto:notifications@github.com> Reply-To: lfiondella/SRT reply@reply.github.com<mailto:reply@reply.github.com> Date: Wednesday, October 28, 2015 at 7:07 PM To: lfiondella/SRT SRT@noreply.github.com<mailto:SRT@noreply.github.com> Cc: Allen Nikora Allen.P.Nikora@jpl.nasa.gov<mailto:Allen.P.Nikora@jpl.nasa.gov> Subject: Re: [SRT] Model Result Plot Curves Could Be More Accurate (#74)

Okay. But i am thinking of points only option - too many dots on the plot will not be appealing. Users may be interested in actual data points and its corresponding model estimate. I could be wrong here but i point this here just in case you were not thinking about it. Lets wait for professors view on it too.

If there is a way to mask the mini intervals from ggplot plot function then it is good to implement it.

@lfiondella What do you think professor ?

On Wed, Oct 28, 2015 at 9:46 PM, a1729nikora notifications@github.com<mailto:notifications@github.com> wrote:

Actually, no – I guess I didn’t explain it well. For drawing lines, what I’m thinking is dividing the interval to be plotted into very small intervals, estimating the endpoints of those intervals using the models (whose parameters have already been determined), and then connecting the endpoints of those very small intervals with straight lines. That gives a pretty good visual approximation of the correct curve in all circumstances I’ve tried so far.

-Allen N.

Allen P. Nikora, Ph.D. Program Element Manager, Assurance Technology Program Office Software Element

Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 Mail Stop 125-233

vox: 818.393.1104 fax: 818.393.1362 email: Allen.P.Nikora@jpl.nasa.govmailto:Allen.P.Nikora@jpl.nasa.gov

From: Karthik Katipally notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> Reply-To: lfiondella/SRT reply@reply.github.com<mailto:reply@reply.github.commailto:reply@reply.github.com> Date: Wednesday, October 28, 2015 at 6:42 PM To: lfiondella/SRT SRT@noreply.github.com<mailto:SRT@noreply.github.commailto:SRT@noreply.github.com> Cc: Allen Nikora Allen.P.Nikora@jpl.nasa.gov<mailto:Allen.P.Nikora@jpl.nasa.govmailto:Allen.P.Nikora@jpl.nasa.gov> Subject: Re: [SRT] Model Result Plot Curves Could Be More Accurate (#74)

You mean interpolation of data points by a polynomial instead of straight line.

On Wed, Oct 28, 2015 at 8:47 PM, a1729nikora notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Suggest redoing the functionality in "PlotModelResults.R" to redraw the model result lines at a much finer resolution than is done at present. I've been experimenting in my local development branch. I can draw more accurate curves and extend them arbitrarily far into the future. It requires recomputing the model results (but not the model parameters) when the plot is redrawn, but it doesn't seem to cause a big delay.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/lfiondella/SRT/issues/74#issuecomment-152048788. — Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/lfiondella/SRT/issues/74#issuecomment-152051748.

arhik commented 8 years ago

okay i got it. Sure we can talk about this tomorrow and see its working.

karthik.

On Wed, Oct 28, 2015 at 10:53 PM, a1729nikora notifications@github.com wrote:

It’s probably easier just to show you all on Friday. I’ve attached a plot that shows some of it. This is a lines-only plot where I’ve allowed the lines to extend 30,000 time units past the end of the last prediction point I’ve specified for the model (I asked to predict 8 failures into the future). You can extend the lines out as far as you want.

-Allen N.

Allen P. Nikora, Ph.D. Program Element Manager, Assurance Technology Program Office Software Element

Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 Mail Stop 125-233

vox: 818.393.1104 fax: 818.393.1362 email: Allen.P.Nikora@jpl.nasa.gov

From: Karthik Katipally notifications@github.com<mailto:notifications@github.com> Reply-To: lfiondella/SRT reply@reply.github.com<mailto:reply@reply.github.com> Date: Wednesday, October 28, 2015 at 7:07 PM To: lfiondella/SRT SRT@noreply.github.com<mailto:SRT@noreply.github.com> Cc: Allen Nikora Allen.P.Nikora@jpl.nasa.gov<mailto:Allen.P.Nikora@jpl.nasa.gov> Subject: Re: [SRT] Model Result Plot Curves Could Be More Accurate (#74)

Okay. But i am thinking of points only option - too many dots on the plot will not be appealing. Users may be interested in actual data points and its corresponding model estimate. I could be wrong here but i point this here just in case you were not thinking about it. Lets wait for professors view on it too.

If there is a way to mask the mini intervals from ggplot plot function then it is good to implement it.

@lfiondella What do you think professor ?

On Wed, Oct 28, 2015 at 9:46 PM, a1729nikora notifications@github.com<mailto:notifications@github.com> wrote:

Actually, no – I guess I didn’t explain it well. For drawing lines, what I’m thinking is dividing the interval to be plotted into very small intervals, estimating the endpoints of those intervals using the models (whose parameters have already been determined), and then connecting the endpoints of those very small intervals with straight lines. That gives a pretty good visual approximation of the correct curve in all circumstances I’ve tried so far.

-Allen N.

Allen P. Nikora, Ph.D. Program Element Manager, Assurance Technology Program Office Software Element

Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 Mail Stop 125-233

vox: 818.393.1104 fax: 818.393.1362 email: Allen.P.Nikora@jpl.nasa.govmailto:Allen.P.Nikora@jpl.nasa.gov

From: Karthik Katipally

notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> Reply-To: lfiondella/SRT

reply@reply.github.com<mailto:reply@reply.github.commailto:reply@reply.github.com> Date: Wednesday, October 28, 2015 at 6:42 PM To: lfiondella/SRT

SRT@noreply.github.com<mailto:SRT@noreply.github.commailto:SRT@noreply.github.com> Cc: Allen Nikora

Allen.P.Nikora@jpl.nasa.gov<mailto:Allen.P.Nikora@jpl.nasa.govmailto:Allen.P.Nikora@jpl.nasa.gov> Subject: Re: [SRT] Model Result Plot Curves Could Be More Accurate (#74)

You mean interpolation of data points by a polynomial instead of straight line.

On Wed, Oct 28, 2015 at 8:47 PM, a1729nikora

notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Suggest redoing the functionality in "PlotModelResults.R" to redraw the model result lines at a much finer resolution than is done at present. I've been experimenting in my local development branch. I can draw more accurate curves and extend them arbitrarily far into the future. It requires recomputing the model results (but not the model parameters) when the plot is redrawn, but it doesn't seem to cause a big delay.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on

GitHubhttps://github.com/lfiondella/SRT/issues/74#issuecomment-152048788. — Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/lfiondella/SRT/issues/74#issuecomment-152051748. — Reply to this email directly or view it on GitHub.

lfiondella commented 8 years ago

I think this sounds reasonable, so long as it does not create any computational delay on data sets with long time intervals.

----- Original Message -----

From: "Karthik Katipally" notifications@github.com To: "lfiondella/SRT" SRT@noreply.github.com Cc: "lfiondella" lfiondella@umassd.edu Sent: Wednesday, October 28, 2015 10:07:04 PM Subject: Re: [SRT] Model Result Plot Curves Could Be More Accurate (#74)

Okay. But i am thinking of points only option - too many dots on the plot will not be appealing. Users may be interested in actual data points and its corresponding model estimate. I could be wrong here but i point this here just in case you were not thinking about it. Lets wait for professors view on it too.

If there is a way to mask the mini intervals from ggplot plot function then it is good to implement it.

@lfiondella What do you think professor ?

On Wed, Oct 28, 2015 at 9:46 PM, a1729nikora notifications@github.com wrote:

Actually, no – I guess I didn’t explain it well. For drawing lines, what I’m thinking is dividing the interval to be plotted into very small intervals, estimating the endpoints of those intervals using the models (whose parameters have already been determined), and then connecting the endpoints of those very small intervals with straight lines. That gives a pretty good visual approximation of the correct curve in all circumstances I’ve tried so far.

-Allen N.

Allen P. Nikora, Ph.D. Program Element Manager, Assurance Technology Program Office Software Element

Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 Mail Stop 125-233

vox: 818.393.1104 fax: 818.393.1362 email: Allen.P.Nikora@jpl.nasa.gov

From: Karthik Katipally notifications@github.com<mailto:notifications@github.com> Reply-To: lfiondella/SRT reply@reply.github.com<mailto:reply@reply.github.com> Date: Wednesday, October 28, 2015 at 6:42 PM To: lfiondella/SRT SRT@noreply.github.com<mailto:SRT@noreply.github.com> Cc: Allen Nikora Allen.P.Nikora@jpl.nasa.gov<mailto:Allen.P.Nikora@jpl.nasa.gov> Subject: Re: [SRT] Model Result Plot Curves Could Be More Accurate (#74)

You mean interpolation of data points by a polynomial instead of straight line.

On Wed, Oct 28, 2015 at 8:47 PM, a1729nikora notifications@github.com<mailto:notifications@github.com> wrote:

Suggest redoing the functionality in "PlotModelResults.R" to redraw the model result lines at a much finer resolution than is done at present. I've been experimenting in my local development branch. I can draw more accurate curves and extend them arbitrarily far into the future. It requires recomputing the model results (but not the model parameters) when the plot is redrawn, but it doesn't seem to cause a big delay.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/lfiondella/SRT/issues/74#issuecomment-152048788. — Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub .

arhik commented 8 years ago

This is done.

Thank you @a1729nikora