search

hpc-io / dxt-explorer

DXT Explorer is an interactive web-based log analysis tool for Darshan DXT logs.
https://dxt-explorer.readthedocs.io
Other
16 stars 6 forks source link

[Bug]: unable to generate web page #11

Closed roblatham00 closed 1 year ago

roblatham00 commented 2 years ago

What happened?

I ran

 python3 explore/dxt.py sample/jlbez_IO_test5.Linux_id1797024_12-1-78642-14680324743816948210_159.darshan

But it failed with lots of errors of the form 

2022-08-08 14:54:49,890 dxt - ERROR - Call `lifecycle::last_lifecycle_warnings()` to see where this warning was generated.
2022-08-08 14:54:49,890 dxt - ERROR - Error in readPNG(paste(base, "dxt-explorer.png", sep = "/")) :
2022-08-08 14:54:49,891 dxt - ERROR -   unable to open /home/robl/work/dxt-explorer/explore/plots/dxt-explorer.png
2022-08-08 14:54:49,891 dxt - ERROR - Calls: %>% ... layout.plotly -> raster2uri -> <Anonymous> -> as.raster -> readPNG
2022-08-08 14:54:49,891 dxt - ERROR - In addition: Warning messages:
2022-08-08 14:54:49,891 dxt - ERROR - 1: 'scattergl' objects don't have these attributes: 'hoveron'
2022-08-08 14:54:49,891 dxt - ERROR - Valid attributes include:
2022-08-08 14:54:49,891 dxt - ERROR - 'connectgaps', 'customdata', 'customdatasrc', 'dx', 'dy', 'error_x', 'error_y', 'fill', 'fillcolor', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup',
'legendgrouptitle', 'legendrank', 'line', 'marker', 'meta', 'metasrc', 'mode', 'name', 'opacity', 'selected', 'selectedpoints', 'showlegend', 'stream', 'text', 'textfont', 'textposition', 'textpositionsrc', 'textsrc', 'texttemplate', 'texttemplatesrc', 'transforms', 'type', 'uid'
, 'uirevision', 'unselected', 'visible', 'x', 'x0', 'xaxis', 'xcalendar', 'xhoverformat', 'xperiod', 'xperiod0', 'xperiodalignment', 'xsrc', 'y', 'y0', 'yaxis', 'ycalendar', 'yhoverformat', 'yperiod', 'yperiod0', 'yperiodalignment', 'ysrc', 'key', 'set', 'frame', 'transforms', '_
isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'

then

2022-08-08 14:54:56,910 dxt - ERROR - Execution halted

This is from version 'main'.

What did you expect to happen?

I expected to find a generated `explore.html' file

Version

0.2 (Default)

Relevant log output

% python3 -d explore/dxt.py sample/jlbez_IO_test5.Linux_id1797024_12-1-78642-14680324743816948210_159.darshan > dxt-explore.debug
/home/robl/work/dxt-explorer/explore/dxt.py:33: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  from distutils.spawn import find_executable
2022-08-08 15:01:12,754 dxt - INFO - FILE: /gpfs/alpine/csc300/scratch/jlbez/vpic-brtnfld/results/vpic-hdf5-1662725/field_hdf5/T.1/fields_1.h5 (ID 12661278810446642700)
2022-08-08 15:01:12,754 dxt - INFO - FILE: /gpfs/alpine/csc300/scratch/jlbez/vpic-brtnfld/results/vpic-hdf5-1662725/hydro_hdf5/T.1/hydro_proton_1.h5 (ID 5261984978546744446)
2022-08-08 15:01:12,754 dxt - INFO - FILE: /gpfs/alpine/csc300/scratch/jlbez/vpic-brtnfld/results/vpic-hdf5-1662725/hydro_hdf5/T.1/hydro_electron_1.h5 (ID 10742304905959859)
2022-08-08 15:01:12,754 dxt - INFO - FILE: /gpfs/alpine/csc300/scratch/jlbez/vpic-brtnfld/results/vpic-hdf5-1662725/particle_hdf5/T.1/proton_1.h5 (ID 14724760704877721098)
2022-08-08 15:01:12,754 dxt - INFO - FILE: /gpfs/alpine/csc300/scratch/jlbez/vpic-brtnfld/results/vpic-hdf5-1662725/particle_hdf5/T.1/electron_1.h5 (ID 9202435221339760075)
2022-08-08 15:01:12,754 dxt - INFO - 102447 I/O trace observation records from 5 files
2022-08-08 15:01:12,756 dxt - INFO - generating datasets
2022-08-08 15:01:12,780 dxt - INFO - generating interactive operation for: /gpfs/alpine/csc300/scratch/jlbez/vpic-brtnfld/results/vpic-hdf5-1662725/field_hdf5/T.1/fields_1.h5
2022-08-08 15:01:14,306 dxt - ERROR - failed to generate the interactive plots (error 1)
2022-08-08 15:01:14,306 dxt - ERROR - Attaching package: ‘R.oo’
2022-08-08 15:01:14,306 dxt - ERROR - The following object is masked from ‘package:R.methodsS3’:
2022-08-08 15:01:14,306 dxt - ERROR -     throw
2022-08-08 15:01:14,306 dxt - ERROR - The following objects are masked from ‘package:methods’:
2022-08-08 15:01:14,306 dxt - ERROR -     getClasses, getMethods
2022-08-08 15:01:14,306 dxt - ERROR - The following objects are masked from ‘package:base’:
2022-08-08 15:01:14,306 dxt - ERROR -     attach, detach, load, save
2022-08-08 15:01:14,307 dxt - ERROR - Warning message:
2022-08-08 15:01:14,307 dxt - ERROR - `gather_()` was deprecated in tidyr 1.2.0.
2022-08-08 15:01:14,307 dxt - ERROR - Please use `gather()` instead.
2022-08-08 15:01:14,307 dxt - ERROR - This warning is displayed once every 8 hours.
2022-08-08 15:01:14,307 dxt - ERROR - Call `lifecycle::last_lifecycle_warnings()` to see where this warning was generated.
2022-08-08 15:01:14,307 dxt - ERROR - Error in readPNG(paste(base, "dxt-explorer.png", sep = "/")) :
2022-08-08 15:01:14,307 dxt - ERROR -   unable to open /home/robl/work/dxt-explorer/explore/plots/dxt-explorer.png
2022-08-08 15:01:14,307 dxt - ERROR - Calls: %>% ... layout.plotly -> raster2uri -> <Anonymous> -> as.raster -> readPNG
2022-08-08 15:01:14,307 dxt - ERROR - In addition: Warning messages:
2022-08-08 15:01:14,307 dxt - ERROR - 1: 'scattergl' objects don't have these attributes: 'hoveron'
2022-08-08 15:01:14,307 dxt - ERROR - Valid attributes include:
2022-08-08 15:01:14,307 dxt - ERROR - 'connectgaps', 'customdata', 'customdatasrc', 'dx', 'dy', 'error_x', 'error_y', 'fill', 'fillcolor', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup', 'legendgrouptitle', 'legendrank', 'line', 'marker', 'meta', 'metasrc', 'mode', 'name', 'opacity', 'selected', 'selectedpoints', 'showlegend', 'stream', 'text', 'textfont', 'textposition', 'textpositionsrc', 'textsrc', 'texttemplate', 'texttemplatesrc', 'transforms', 'type', 'uid', 'uirevision', 'unselected', 'visible', 'x', 'x0', 'xaxis', 'xcalendar', 'xhoverformat', 'xperiod', 'xperiod0', 'xperiodalignment', 'xsrc', 'y', 'y0', 'yaxis', 'ycalendar', 'yhoverformat', 'yperiod', 'yperiod0', 'yperiodalignment', 'ysrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'
2022-08-08 15:01:14,307 dxt - ERROR - 2: 'scattergl' objects don't have these attributes: 'hoveron'
2022-08-08 15:01:14,307 dxt - ERROR - Valid attributes include:
2022-08-08 15:01:14,307 dxt - ERROR - 'connectgaps', 'customdata', 'customdatasrc', 'dx', 'dy', 'error_x', 'error_y', 'fill', 'fillcolor', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup', 'legendgrouptitle', 'legendrank', 'line', 'marker', 'meta', 'metasrc', 'mode', 'name', 'opacity', 'selected', 'selectedpoints', 'showlegend', 'stream', 'text', 'textfont', 'textposition', 'textpositionsrc', 'textsrc', 'texttemplate', 'texttemplatesrc', 'transforms', 'type', 'uid', 'uirevision', 'unselected', 'visible', 'x', 'x0', 'xaxis', 'xcalendar', 'xhoverformat', 'xperiod', 'xperiod0', 'xperiodalignment', 'xsrc', 'y', 'y0', 'yaxis', 'ycalendar', 'yhoverformat', 'yperiod', 'yperiod0', 'yperiodalignment', 'ysrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'
2022-08-08 15:01:14,307 dxt - ERROR - 3: 'scattergl' objects don't have these attributes: 'hoveron'
2022-08-08 15:01:14,307 dxt - ERROR - Valid attributes include:
2022-08-08 15:01:14,307 dxt - ERROR - 'connectgaps', 'customdata', 'customdatasrc', 'dx', 'dy', 'error_x', 'error_y', 'fill', 'fillcolor', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup', 'legendgrouptitle', 'legendrank', 'line', 'marker', 'meta', 'metasrc', 'mode', 'name', 'opacity', 'selected', 'selectedpoints', 'showlegend', 'stream', 'text', 'textfont', 'textposition', 'textpositionsrc', 'textsrc', 'texttemplate', 'texttemplatesrc', 'transforms', 'type', 'uid', 'uirevision', 'unselected', 'visible', 'x', 'x0', 'xaxis', 'xcalendar', 'xhoverformat', 'xperiod', 'xperiod0', 'xperiodalignment', 'xsrc', 'y', 'y0', 'yaxis', 'ycalendar', 'yhoverformat', 'yperiod', 'yperiod0', 'yperiodalignment', 'ysrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'
2022-08-08 15:01:14,307 dxt - ERROR - 4: 'scattergl' objects don't have these attributes: 'hoveron'
2022-08-08 15:01:14,307 dxt - ERROR - Valid attributes include:
2022-08-08 15:01:14,307 dxt - ERROR - 'connectgaps', 'customdata', 'customdatasrc', 'dx', 'dy', 'error_x', 'error_y', 'fill', 'fillcolor', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup', 'legendgrouptitle', 'legendrank', 'line', 'marker', 'meta', 'metasrc', 'mode', 'name', 'opacity', 'selected', 'selectedpoints', 'showlegend', 'stream', 'text', 'textfont', 'textposition', 'textpositionsrc', 'textsrc', 'texttemplate', 'texttemplatesrc', 'transforms', 'type', 'uid', 'uirevision', 'unselected', 'visible', 'x', 'x0', 'xaxis', 'xcalendar', 'xhoverformat', 'xperiod', 'xperiod0', 'xperiodalignment', 'xsrc', 'y', 'y0', 'yaxis', 'ycalendar', 'yhoverformat', 'yperiod', 'yperiod0', 'yperiodalignment', 'ysrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'
2022-08-08 15:01:14,308 dxt - ERROR - 5: 'scattergl' objects don't have these attributes: 'hoveron'
2022-08-08 15:01:14,308 dxt - ERROR - Valid attributes include:
2022-08-08 15:01:14,308 dxt - ERROR - 'connectgaps', 'customdata', 'customdatasrc', 'dx', 'dy', 'error_x', 'error_y', 'fill', 'fillcolor', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup', 'legendgrouptitle', 'legendrank', 'line', 'marker', 'meta', 'metasrc', 'mode', 'name', 'opacity', 'selected', 'selectedpoints', 'showlegend', 'stream', 'text', 'textfont', 'textposition', 'textpositionsrc', 'textsrc', 'texttemplate', 'texttemplatesrc', 'transforms', 'type', 'uid', 'uirevision', 'unselected', 'visible', 'x', 'x0', 'xaxis', 'xcalendar', 'xhoverformat', 'xperiod', 'xperiod0', 'xperiodalignment', 'xsrc', 'y', 'y0', 'yaxis', 'ycalendar', 'yhoverformat', 'yperiod', 'yperiod0', 'yperiodalignment', 'ysrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'
2022-08-08 15:01:14,308 dxt - ERROR - 6: 'scattergl' objects don't have these attributes: 'hoveron'
2022-08-08 15:01:14,308 dxt - ERROR - Valid attributes include:
2022-08-08 15:01:14,308 dxt - ERROR - 'connectgaps', 'customdata', 'customdatasrc', 'dx', 'dy', 'error_x', 'error_y', 'fill', 'fillcolor', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup', 'legendgrouptitle', 'legendrank', 'line', 'marker', 'meta', 'metasrc', 'mode', 'name', 'opacity', 'selected', 'selectedpoints', 'showlegend', 'stream', 'text', 'textfont', 'textposition', 'textpositionsrc', 'textsrc', 'texttemplate', 'texttemplatesrc', 'transforms', 'type', 'uid', 'uirevision', 'unselected', 'visible', 'x', 'x0', 'xaxis', 'xcalendar', 'xhoverformat', 'xperiod', 'xperiod0', 'xperiodalignment', 'xsrc', 'y', 'y0', 'yaxis', 'ycalendar', 'yhoverformat', 'yperiod', 'yperiod0', 'yperiodalignment', 'ysrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'
2022-08-08 15:01:14,308 dxt - ERROR - 7: 'scattergl' objects don't have these attributes: 'hoveron'
2022-08-08 15:01:14,308 dxt - ERROR - Valid attributes include:
2022-08-08 15:01:14,308 dxt - ERROR - 'connectgaps', 'customdata', 'customdatasrc', 'dx', 'dy', 'error_x', 'error_y', 'fill', 'fillcolor', 'hoverinfo', 'hoverinfosrc', 'hoverlabel', 'hovertemplate', 'hovertemplatesrc', 'hovertext', 'hovertextsrc', 'ids', 'idssrc', 'legendgroup', 'legendgrouptitle', 'legendrank', 'line', 'marker', 'meta', 'metasrc', 'mode', 'name', 'opacity', 'selected', 'selectedpoints', 'showlegend', 'stream', 'text', 'textfont', 'textposition', 'textpositionsrc', 'textsrc', 'texttemplate', 'texttemplatesrc', 'transforms', 'type', 'uid', 'uirevision', 'unselected', 'visible', 'x', 'x0', 'xaxis', 'xcalendar', 'xhoverformat', 'xperiod', 'xperiod0', 'xperiodalignment', 'xsrc', 'y', 'y0', 'yaxis', 'ycalendar', 'yhoverformat', 'yperiod', 'yperiod0', 'yperiodalignment', 'ysrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule', '_bbox'
...
jeanbez commented 2 years ago

@roblatham00 Thanks for the report! Can you confirm the loaded modules and if you installed it using Spack or directly from source?

roblatham00 commented 2 years ago

I installed from git@main and am using a mix of Ubuntu packages, pip, and CRAN

I will try with spack, but where is the spack recipie?

jeanbez commented 2 years ago

@roblatham00, thanks for reporting this! Could you give it a try with the current main branch?

roblatham00 commented 2 years ago

Ubuntu-22.04:

% ./explorer/dxt.py sample/short.darshan
./explorer/dxt.py: 20:
DXT Explorer.

DXT Explorer Copyright (c) 2021, The Regents of the University of
California, through Lawrence Berkeley National Laboratory (subject
to receipt of any required approvals from the U.S. Dept. of Energy).
All rights reserved.

If you have questions about your rights to use or distribute this software,
please contact Berkeley Lab's Intellectual Property Office at
IPO@lbl.gov.

NOTICE.  This Software was developed under funding from the U.S. Department
of Energy and the U.S. Government consequently retains certain rights.  As
such, the U.S. Government has been granted for itself and others acting on
its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the
Software to reproduce, distribute copies to the public, prepare derivative
works, and perform publicly and display publicly, and to permit others to do
so.
: File name too long

Ubuntu-20.04:

$ python3 explorer/dxt.py sample/jlbez_IO_test5.Linux_id1797024_12-1-78642-14680324743816948210_159.darshan
Traceback (most recent call last):
  File "explorer/dxt.py", line 35, in <module>
    from explorer import version
ModuleNotFoundError: No module named 'explorer'
jeanbez commented 2 years ago

Hmm, you should not be running it directly. Could you give it a try with pip install .?

roblatham00 commented 2 years ago

thanks that did the trick! Ubuntu-20.04 produces a graphic. Ubuntu-22.04 however renders a blank page with only title and file name, no graph.

jeanbez commented 2 years ago

Great! I tried on Ubuntu 22, and I managed to view the plot. Could you confirm the R version you had there? Also, could you share the generated .html file?

roblatham00 commented 2 years ago
jeanbez commented 1 year ago

@roblatham00 did you run into this issue with the latest release? Were you able to run and render it in Ubuntu 22.04?

roblatham00 commented 1 year ago

Sorry for not responding in a timely fashion.

command line:
% /home/robl/.local/bin/dxt-explorer sample/jlbez_IO_test5.Linux_id1797024_12-1-78642-14680324743816948210_159.darshan

generated HTML report did not render in either Microsoft Edge or Filrefox browsers: just a two line header and an otherwise blank web page.