Open Jeff62129 opened 1 month ago
I think it is somehow confusing double data in the file as float in memory. I see the vestiges of the circular shaped dark region in the VisIt plot but they are all separated by some distance.
@markcmiller86 sorry for some missing information. The plot should show a simple simulation of an antenna emitting a Gaussian beam pulse to either direction as shown in the plot on the right side via yt. Visit weirdly separates it. I can not fix it easily with some built-in transformation functions. Is VisIt only applicable to 3D rendering? Thank you so much for your reply. I appreciate the great work the VisIt team has done!
Is VisIt only applicable to 3D rendering?
No, VisIt handles 2D and 3D.
I haven't yet looked at your files but I suspect that this can be added to the list of problems that will be solved by upgrading our openPMD reader to use the openPMD API.
We have a lot of tickets these days so we will get to this when we can!
So, when working on the OpenPMD plugin last time, I noticed some differences in behavior depending on whether it was being run in serial or parallel. When I run two-processor parallel for this data, I get an even different answer than the one VisIt produced above...
And for 5 processors...
looks very much like a striding blunder
looks very much like a striding blunder
Thanks! I am trying to get to the bottom of it...still puzzled.
@Jeff62129 what can you tells us about what yt
(and/or its dependencies such as h5py
) are doing to the data once read from the file? You are comparing what the VisIt plugin is doing to (potentially) a lot of data interpretation in the python tools being invoked here.
My reason for asking is that there is no interpretation of the data being made in the VisIt plugin. It is obtaing the buffer of data directly from the file and mapping that buffer 1:1 with the rectilinear mesh it is defined on.
Now, in the above comments, I have identified a second issue...that the striding used to decompose the input data across parallel tasks appears to be wrong. Otherwise, we'd get the same result no matter how many processors we used. We don't. But, that is orthogonal to the problem you originally reported (and which I can reproduce with the data you've provided).
I see the below metadata in the file (I've removed the raw data). yt
must be interpreting some of this because it is showing a square result but the number of points in each dimension is different (768 vs 128). What do the values open
and reflecting
in fieldBoundary
mean? On the other hand, yt
is getting some things wrong. The file says axisLabels
are x
and z
which is what VisIt says but the result from yt
is showing x
and y
.
So, I believe there is something yt
is doing to interpret this data correctly and that logic does not exist in the VisIt plugin and we need help to get it resolved.
/ Group
Attribute: basePath scalar
Type: 10-byte null-terminated ASCII string
Data: "/data/%T/"
Attribute: date scalar
Type: 26-byte null-terminated ASCII string
Data: "2024-07-11 17:11:58 +0800"
Attribute: iterationEncoding scalar
Type: 10-byte null-terminated ASCII string
Data: "fileBased"
Attribute: iterationFormat scalar
Type: 13-byte null-terminated ASCII string
Data: "openpmd_%06T"
Attribute: meshesPath scalar
Type: 8-byte null-terminated ASCII string
Data: "fields/"
Attribute: openPMD scalar
Type: 6-byte null-terminated ASCII string
Data: "1.1.0"
Attribute: openPMDextension scalar
Type: native unsigned int
Data: 1
Attribute: software scalar
Type: 6-byte null-terminated ASCII string
Data: "WarpX"
Attribute: softwareVersion scalar
Type: 8-byte null-terminated ASCII string
Data: "Unknown"
Location: 1:96
Links: 1
/data Group
Location: 1:800
Links: 1
/data/400 Group
Attribute: dt scalar
Type: native double
Data: 1
Attribute: time scalar
Type: native double
Data: 1.01792e-13
Attribute: timeUnitSI scalar
Type: native double
Data: 1
Location: 1:1832
Links: 1
/data/400/fields Group
Attribute: chargeCorrection scalar
Type: 5-byte null-terminated ASCII string
Data: "none"
Attribute: currentSmoothing scalar
Type: 9-byte null-terminated ASCII string
Data: "Binomial"
Attribute: currentSmoothingParameters scalar
Type: 55-byte null-terminated ASCII string
Data: "period=1;compensator=false;numPasses_x=1;numPasses_z=1"
Attribute: fieldBoundary {4}
Type: 11-byte null-terminated ASCII string
Data: "open", "open", "reflecting", "reflecting"
Attribute: fieldSolver scalar
Type: 4-byte null-terminated ASCII string
Data: "Yee"
Attribute: particleBoundary {4}
Type: 10-byte null-terminated ASCII string
Data: "absorbing", "absorbing", "absorbing", "absorbing"
Location: 1:2864
Links: 1
/data/400/fields/B Group
Attribute: axisLabels {2}
Type: 2-byte null-terminated ASCII string
Data: "z", "x"
Attribute: dataOrder scalar
Type: 2-byte null-terminated ASCII string
Data: "C"
Attribute: fieldSmoothing scalar
Type: 5-byte null-terminated ASCII string
Data: "none"
Attribute: geometry scalar
Type: 10-byte null-terminated ASCII string
Data: "cartesian"
Attribute: gridGlobalOffset {2}
Type: native double
Data: -3e-05, -3e-05
Attribute: gridSpacing {2}
Type: native double
Data: 7.8125e-08, 4.6875e-07
Attribute: gridUnitSI scalar
Type: native double
Data: 1
Attribute: timeOffset scalar
Type: native float
Data: 0
Attribute: unitDimension {7}
Type: native double
Data:
(0) 0, 1, -2, -1, 0, 0, 0
Location: 1:4456
Links: 1
/data/400/fields/B/x Dataset {768/768, 128/128}
Attribute: position {2}
Type: native double
Data: 0.5, 0.5
Attribute: unitSI scalar
Type: native double
Data: 1
Location: 1:5488
Links: 1
Chunks: {128, 128} 131072 bytes
Storage: 786432 logical bytes, 786432 allocated bytes, 100.00% utilization
Type: native double
Data:
(0,0) 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
.
.
.
(767,119) 0, 0, 0, 0, 0, 0, 0, 0, 0
/data/400/fields/B/y Dataset {768/768, 128/128}
Attribute: position {2}
Type: native double
Data: 0.5, 0.5
Attribute: unitSI scalar
Type: native double
Data: 1
Location: 1:795232
Links: 1
Chunks: {128, 128} 131072 bytes
Storage: 786432 logical bytes, 786432 allocated bytes, 100.00% utilization
Type: native double
Data:
/data/400/fields/B/z Dataset {768/768, 128/128}
Attribute: position {2}
Type: native double
Data: 0.5, 0.5
Attribute: unitSI scalar
Type: native double
Data: 1
Location: 1:1584648
Links: 1
Chunks: {128, 128} 131072 bytes
Storage: 786432 logical bytes, 786432 allocated bytes, 100.00% utilization
Type: native double
Data:
/data/400/fields/E Group
Attribute: axisLabels {2}
Type: 2-byte null-terminated ASCII string
Data: "z", "x"
Attribute: dataOrder scalar
Type: 2-byte null-terminated ASCII string
Data: "C"
Attribute: fieldSmoothing scalar
Type: 5-byte null-terminated ASCII string
Data: "none"
Attribute: geometry scalar
Type: 10-byte null-terminated ASCII string
Data: "cartesian"
Attribute: gridGlobalOffset {2}
Type: native double
Data: -3e-05, -3e-05
Attribute: gridSpacing {2}
Type: native double
Data: 7.8125e-08, 4.6875e-07
Attribute: gridUnitSI scalar
Type: native double
Data: 1
Attribute: timeOffset scalar
Type: native float
Data: 0
Attribute: unitDimension {7}
Type: native double
Data:
(0) 1, 1, -3, -1, 0, 0, 0
Location: 1:2374816
Links: 1
/data/400/fields/E/x Dataset {768/768, 128/128}
Attribute: position {2}
Type: native double
Data: 0.5, 0.5
Attribute: unitSI scalar
Type: native double
Data: 1
Location: 1:2375520
Links: 1
Chunks: {128, 128} 131072 bytes
Storage: 786432 logical bytes, 786432 allocated bytes, 100.00% utilization
Type: native double
Data:
/data/400/fields/E/y Dataset {768/768, 128/128}
Attribute: position {2}
Type: native double
Data: 0.5, 0.5
Attribute: unitSI scalar
Type: native double
Data: 1
Location: 1:3165264
Links: 1
Chunks: {128, 128} 131072 bytes
Storage: 786432 logical bytes, 786432 allocated bytes, 100.00% utilization
Type: native double
Data:
/data/400/fields/E/z Dataset {768/768, 128/128}
Attribute: position {2}
Type: native double
Data: 0.5, 0.5
Attribute: unitSI scalar
Type: native double
Data: 1
Location: 1:3954680
Links: 1
Chunks: {128, 128} 131072 bytes
Storage: 786432 logical bytes, 786432 allocated bytes, 100.00% utilization
Type: native double
Data:
Actually, I think what may be happening is that it is mixing up the x and y dimensions of the dataset in the file. I see datasets there of dimension 768, 128
but it is the x
axis in the associated mesh that is getting 768 values and the y axis is getting 128 values.
Ok, yes indeed, I have proven to myself that the plugin is interposing the coordinate axes. When I change the plugin to treat the y-values as the x coordinates and the x-values as the y coordinates of the underlying mesh (which also changes the # points it is expecting in x and y dimensions), I get the plot you get but rotated 90 degrees...
I think the issue maybe how the plugin is handling the axisLabels
metadata. We have other test data with...
scratlantis:databases/OpenPMD/OpenPMDClasses] miller86% h5ls -vlr ~/visit/visit/data/openpmd_test_data/data00000500.opmd | grep -A 3 -i label
Attribute: axisLabels {3}
Type: 1-byte null-padded ASCII string
Data: "x", "y", "z"
Attribute: dataOrder scalar
--
Attribute: axisLabels {3}
Type: 1-byte null-padded ASCII string
Data: "x", "y", "z"
Attribute: dataOrder scalar
which plots fine but note that your data has...
/data/400/fields/E Group
Attribute: axisLabels {2}
Type: 2-byte null-terminated ASCII string
Data: "z", "x"
which places "x"
as the second of the two dimensions in the file and not the first. VisIt's plugin is not coded to deal with this.
Hi @markcmiller86,
Thank you for your response. Unfortunately, I'm not familiar enough with the underlying mechanics of yt to provide a detailed explanation. However, I can share that I encountered some unusual results when attempting to visualize particle-related quantities with yt. It is worth noting that WarpX does not implement OpenPMD-related I/O operations directly but instead utilizes the OpenPMD-API provided by the OpenPMD team.
Interestingly, all the visualization issues I experienced with VisIt were accurately resolved using the library available in the official OpenPMD GitHub repository.
Do you think it's viable that VisIt also uses openPMD-api to read opmd files?
Thank you for your assistance, and I hope this information is helpful.
Do you think it's viable that VisIt also uses openPMD-api to read opmd files?
We are already planning to add this feature. I can't say yet when we will have it in but this will certainly factor into our upcoming prioritization discussions for our next release.
@JustinPrivitera, I have a somewhat vague idea how I might proceed on the existing plugin to correct the originally reported issue. We can use the data provided here to create a test as well.
But, in working through this I have identified at least 3 other issues...
My question is, should I pause activity here until we've decided the path forward?
My question is, should I pause activity here until we've decided the path forward?
I think so. Let's discuss, either today or on Thursday.
Just capturing here another issue I discovered in the plugin code. In this line,
It is declaring the memory type into which the data is being read to be the data type of the dataset in the file (e.g. the file's datatype), from here...
It should really be setting the memory type, datasetType
like so...
datasetType = ((dataSize == 4) ? H5T_NATIVE_FLOAT : H5T_NATIVE_DOUBLE);
Because the memory type into which the data is being read is whatever the executable's float
or double
is.
Describe the bug
When drawing a pseudocolor plot for a simple 2D WarpX simulation, the result differs from that plotted with the Python yt module.
To Reproduce
Load the .opmd file (included in openpmd_000400.zip), define an expression "Ex" for "Fields/E/x", and then draw pseudocolor plot for "Ex" You can also use the session file provided here: 2d_bug_report.session.zip The necessity of redefining a variable for the x component of the E field can be found here. The python file used to generate the plot using yt can be found here: laser_propagation_analysis.zip![2dplot_bug_report](https://github.com/visit-dav/visit/assets/175114380/25188f67-36eb-455f-8ac6-8c02fa229d63)
Server