PlotProperties getting QPromptCellWidget instead of cell widget

[remram44]

From UV-CDAT/VisTrails#30

It seems that PlotProperties is updated from ProjectController#update_workflow() just after execute_plot() and this is not safe. The PlotProperties gets the QPromptCellWidget instead of the new cell widget.

A workaround can be added for 2.2 but this should definitely be fixed later.

[mcenerney1]

The data cam35_data.tar and taylor_obs_data.tar should work. Found on ftp ftp.llnl.gov

[hgohil2805]

Ok I'll give it a run.

[hgohil2805]

Guys, New issue. So this is what happens when I load obs_data (Observation 1). I don't think this is because of the local changes that I might have made.

[painter1]

This does look like a bug, but it is totally unrelated.

1. You are using the same path for "Dataset 1" and "Observations 1". They are supposed to be different. The whole point of all this is to compare two different sets of data.
2. You are testing plot set 2. This issue is about plot set 14, Taylor diagrams.

[painter1]

To expand on what I just wrote: The GUI was used incorrectly. It is a bug because there shouldn't be an exception in this case. But fixing that bug will not contribute to fixing this issue, #1181. Hardik, you need to test with a real observation data set under "Observations 1", e.g. the NCEP ANN data. This must be in a different directory from the one which has model data, cam35_data in this case. You also need to test plot set 14, Taylor diagrams. The other plot sets are not relevant to this issue.

[hgohil2805]

Jeff, I took the wrong print screen (I did do the same required things before but then tried to continue doing things anyways to test ).

Here's the one after selecting the correct things.

[painter1]

Under your obs path is a pulldown menu "Obs set 1". Can you choose anything there? My guess is that there's no data in your obs path. But whatever the reason, there is supposed to be a default obs set in the pulldown menu; and if there are more possibilities you should see them if you click on it.

[hgohil2805]

No the drop down menu is unusable before as well as after after the error pops up. Don't know if that's related.

In the obs_data directory, I have 16 .nc files. Is there something specific that you'd like me to check?

[painter1]

Please list the files in obs_data, and tell me what versions you have for options.py and diagnosticsDockWidget.py. Make sure that you have actually installed the ones you think you have. Put print statements in Options.init() (options.py) near the end of the medthod, and prepareObs1 (diagnosticsDockWidget.py) before the line which raises the exception. Print out the values of self._opts['obs'] and self.opts['obs'] respectively. Install these changes and run again. Then please tell me what got printed out.

[hgohil2805]

ok, So I think we should just try and move on to fixing the actual problem of the cell widgets here. Jeff or Jim, can you post a stack trace of what you get after you load everything correctly and the error pops up?.

I'll talk to Remi and we'll try to fix the actual problem and hopefully that should help plot things correctly too.

[hgohil2805]

Ok I'll do those things and let you know. Here are the files that I have in the obs_data directory

[painter1]

Thanks. Actually the only one you need is NCEP_ANN_climo.nc, because you're testing with the ANN season. The others are for other seasons. The xml.orig file is harmless garbage. Something's seriously wrong, because you have the data and the system isn't seeing it. I suspect that you are still seeing an issue with incompatible versions of the diagnostics and VisTrails - or something else, it's hard to guess from here. Adding print statements will at least ensure that you are running what you think you're running.

[hgohil2805]

Ok I am not sure if it is diagnostics. Because I used to get the error pop up as soon as I tried to load the cam data before but now cam loads fine but it's obs which is causing the issue. Let's check with the prints.

[hgohil2805]

Prints out an empty list for options.init()

file - options.py, value of self._opts['obs'] is = []

and prints out

File - diagnosticsWidget, value of self.opts['obs'] is = [{'filter': None, 'path': '/home/iceman/testdata/obs_data'}]

for prepareObs1

[painter1]

Hardik, Copy NCEP_ANN_climo.nc to SPAM_ANN_climo.nc. I think we may have a bug where the system expects there to be more than one obs set available.

[mcenerney1]

Here's a the output

Traceback (most recent call last): File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/diagnosticsDockWidget.py", line 661, in buttonClicked self.applyClicked() File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/diagnosticsDockWidget.py", line 783, in applyClicked self.displayCell(RES30, row, col, dropInfo=(plot_number == 0)) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/diagnosticsDockWidget.py", line 853, in displayCell projectController.variable_was_dropped(varDropInfo) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/project_controller.py", line 779, in plot_was_dropped self.check_update_cell(sheetName,row,col, update) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/project_controller.py", line 913, in check_update_cell self.update_cell(sheetName, row, col, reuse_workflow) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/project_controller.py", line 995, in update_cell reuse_workflow) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/project_controller.py", line 1042, in update_workflow self.update_plot_configure(sheetName, row, column) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/project_controller.py", line 849, in update_plot_configure widget = self.get_plot_configuration(sheetName,row,col) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/project_controller.py", line 907, in get_plot_configuration cell.plots) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 76, in show_configuration_widget return CDMSPlotWidget(controller,version,plots,vars) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 1112, in init self.create_plot_widget() File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 1237, in create_plot_widget stacked_widget = self.create_conf_widgets() File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 1278, in create_conf_widgets self.add_conf_widget(plot_module) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 1285, in add_conf_widget show_buttons=False) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/packages/uvcdat_cdms/widgets.py", line 37, in init self.gmEditor = self.createEditor(self, self.gmName) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/packages/uvcdat_cdms/widgets.py", line 82, in createEditor return QTaylorDiagramEditor(self.tabWidget, gmName) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/graphicsMethodsWidgets.py", line 830, in init self.initValues(self.gm) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/graphicsMethodsWidgets.py", line 867, in initValues self.markersTab.initValues(gm) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/graphicsMethodsWidgets.py", line 988, in initValues s = self.getGMMarkerAttributeValue(M,"status",i) File "/usr/local/uvcdat/03_25_15/vistrails/vistrails/gui/uvcdat/graphicsMethodsWidgets.py", line 1081, in getGMMarkerAttributeValue return v[i] IndexError: list index out of range

[mcenerney1]

Hardik I could call you and walk through the diagnostics if that would help. What's a good phone number? Jim

[hgohil2805]

Hi Jim, I wasn't in at work yesterday so couldn't reply. I am trying what jeff had suggested, will let you know if it works. If not, give me a call at +1-917-717-6588

[mcenerney1]

Good! Let me know. I have a meeting at 9 am(PT).

[hgohil2805]

I was able to load both the files now but now it shows two new errors. Here's the first one,

File "/home/iceman/uvcdat-latest/15-03-24/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 364, in build_plot_pipeline_action
    ops2, new_conns, pm = CDMSPipelineHelper.create_actions_from_plot_obj( controller, var_modules, cell_module, plot, added_vars, i+1 )
  File "/home/iceman/uvcdat-latest/15-03-24/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 311, in create_actions_from_plot_obj
    plot_functions.append((attr,[pickle.dumps(getattr(initial_values,attr))]))
  File "/home/iceman/uvcdat-latest/15-03-24/lib/python2.7/copy_reg.py", line 77, in _reduce_ex
    raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled

So I changed the pickling to the binary protocol in pipeline_helper.py, .

-plot_functions.append((attr,[pickle.dumps(getattr(initial_values,attr))))
+plot_functions.append((attr,[pickle.dumps(getattr(initial_values,attr)),-1]))

But now it shows the following error. It could be because of the change from Ascii to Binary pickling . I am looking into this.

Traceback (most recent call last):
  File "/home/iceman/uvcdat-latest/15-03-24/vistrails/vistrails/gui/uvcdat/diagnosticsDockWidget.py", line 661, in buttonClicked
    self.applyClicked()
  File "/home/iceman/uvcdat-latest/15-03-24/vistrails/vistrails/gui/uvcdat/diagnosticsDockWidget.py", line 743, in applyClicked
    self.plot_spec = self.ds_menu[diagnostic](model, obs, variable, season, self.region_box, aux)
  File "/home/iceman/uvcdat-latest/15-03-24/lib/python2.7/site-packages/metrics/packages/amwg/amwg.py", line 2705, in __init__
    self.plan_computation( model, obs, varid, seasonid, aux )
  File "/home/iceman/uvcdat-latest/15-03-24/lib/python2.7/site-packages/metrics/packages/amwg/amwg.py", line 2791, in plan_computation
    L = FN.split('/')
AttributeError: 'NoneType' object has no attribute 'split'

[mcenerney1]

I think this would result from either a bad specification of either the model data or obs data.

[hgohil2805]

Hey Jim, Can you call me? I have a couple of questions.

[mcenerney1]

I got your answering machine. I'm here 925-422-1963 or I'll call again.

[hgohil2805]

UV-CDAT Diagnostics, command-line version
file - options.py, value of self._opts['obs'] is =  []
Added set:  {'climos': 'yes', 'end': None, 'name': None, 'filter': None, 'start': None, 'path': '/home/iceman/uvcmetrics_test_data/cam35_data/', 'type': 'model'}
Added set:  {'climos': 'yes', 'end': None, 'name': None, 'filter': 'f_startswith("NCEP")', 'start': None, 'path': '/home/iceman/uvcmetrics_test_data/obs_data/', 'type': 'obs'}
Enabling NetCDF compression on output files
ERROR, output directory /home/iceman/Documents/Climatology/ClimateData/diagout/ does not exist!

[painter1]

Try creating the directory yourself.

[hgohil2805]

Works.. !

[doutriaux1]

Hardik,

I fixed the pickle issue your vcs must not be up to date.

C.

From: Hardik Gohil notifications@github.com<mailto:notifications@github.com> Reply-To: UV-CDAT/uvcdat reply@reply.github.com<mailto:reply@reply.github.com> Date: Wednesday, April 15, 2015 at 9:40 AM To: UV-CDAT/uvcdat uvcdat@noreply.github.com<mailto:uvcdat@noreply.github.com> Cc: "Doutriaux, Charles" doutriaux1@llnl.gov<mailto:doutriaux1@llnl.gov> Subject: Re: [uvcdat] PlotProperties getting QPromptCellWidget instead of cell widget (#1181)

I was able to load both the files now but now it shows two new errors. Here's the first one,

File "/home/iceman/uvcdat-latest/15-03-24/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 364, in build_plot_pipeline_action ops2, new_conns, pm = CDMSPipelineHelper.create_actions_from_plot_obj( controller, var_modules, cell_module, plot, added_vars, i+1 ) File "/home/iceman/uvcdat-latest/15-03-24/vistrails/vistrails/packages/uvcdat_cdms/pipeline_helper.py", line 311, in create_actions_from_plot_obj plot_functions.append((attr,[pickle.dumps(getattr(initial_values,attr))])) File "/home/iceman/uvcdat-latest/15-03-24/lib/python2.7/copy_reg.py", line 77, in _reduce_ex raise TypeError("a class that defines slots without " TypeError: a class that defines slots without defining getstate cannot be pickled

So I changed the pickling to the binary protocol in pipeline_helper.py, .

-plot_functions.append((attr,[pickle.dumps(getattr(initial_values,attr)))) +plot_functions.append((attr,[pickle.dumps(getattr(initial_values,attr)),-1]))

But now it shows the following error. It could be because of the change from Ascii to Binary pickling . I am looking into this.

Traceback (most recent call last): File "/home/iceman/uvcdat-latest/15-03-24/vistrails/vistrails/gui/uvcdat/diagnosticsDockWidget.py", line 661, in buttonClicked self.applyClicked() File "/home/iceman/uvcdat-latest/15-03-24/vistrails/vistrails/gui/uvcdat/diagnosticsDockWidget.py", line 743, in applyClicked self.plot_spec = self.ds_menu[diagnostic](model, obs, variable, season, self.region_box, aux) File "/home/iceman/uvcdat-latest/15-03-24/lib/python2.7/site-packages/metrics/packages/amwg/amwg.py", line 2705, in init self.plan_computation( model, obs, varid, seasonid, aux ) File "/home/iceman/uvcdat-latest/15-03-24/lib/python2.7/site-packages/metrics/packages/amwg/amwg.py", line 2791, in plan_computation L = FN.split('/') AttributeError: 'NoneType' object has no attribute 'split'

— Reply to this email directly or view it on GitHubhttps://github.com/UV-CDAT/uvcdat/issues/1181#issuecomment-93480894.

[mcenerney1]

Good you can get the graphic from the command line. you even have the missing x axis as I do.

[mcenerney1]

Hardik It's been quite a while since we talked about this one. Now that uvcdat is released we should resurrect this issue. I just tried it again from the gui and get the same error as on April 13, above. Any thoughts on what's going on? I'm available to help on this!! Jim

[hgohil2805]

Jim, Yes we should try to get this working. I am currently working on GUI testing but I'll look into this too.

[mcenerney1]

Hardik Some new info on this issue. I spent yesterday afternoon in the debugger trying to isolate the issue and could not find it. However, for some reason uvcdat does not crash as before. Now it simply runs but produces no graphics. I don't know what changed but this may help you. Jim

[remram44]

What changed is probably UV-CDAT/VisTrails#30

[doutriaux1]

as @remram44 mentioned in https://github.com/UV-CDAT/VisTrails/pull/30 it needs a "real" fix. But nobody seemed to have found time for it...

[hgohil2805]

Jim, I thought that you were using the latest build with that fix in it. I am going to try and solve the cell and the plot properties problem, hopefully that helps to generate the plot correctly too.

[mcenerney1]

I was using a build with all of the updates. I just rebuilt from the new release (just to be sure) and it behaves the same: it doesn't crash but it proceeds normally and there is no graphic.

[aashish24]

@mcenerney1 are we going to fix this in 2.4?

[doutriaux1]

@chaosphere2112 can you take a look, I'd say if it takes more than 2 minutes, let's wait for the new GUI.

[chaosphere2112]

@doutriaux1 Yup, I'll poke at it.

[chaosphere2112]

@doutriaux1 OK, definitely going to take more than a couple of minutes. I'm gonna vote we wait for the new GUI on this one.

[doutriaux1]

moved to 3.0