search

visit-dav / visit

VisIt - Visualization and Data Analysis for Mesh-based Scientific Data
https://visit.llnl.gov
BSD 3-Clause "New" or "Revised" License
422 stars 109 forks source link

VisIt produces wrong 2D pseudo color plot for 2D WarpX (OpenPMD) data #19648

Open Jeff62129 opened 1 month ago

Jeff62129 commented 1 month ago

Describe the bug

When drawing a pseudocolor plot for a simple 2D WarpX simulation, the result differs from that plotted with the Python yt module.

To Reproduce

Load the .opmd file (included in openpmd_000400.zip), define an expression "Ex" for "Fields/E/x", and then draw pseudocolor plot for "Ex" You can also use the session file provided here: 2d_bug_report.session.zip The necessity of redefining a variable for the x component of the E field can be found here. The python file used to generate the plot using yt can be found here: laser_propagation_analysis.zip 2dplot_bug_report

Server

markcmiller86 commented 1 month ago

I think it is somehow confusing double data in the file as float in memory. I see the vestiges of the circular shaped dark region in the VisIt plot but they are all separated by some distance.

Jeff62129 commented 1 month ago

@markcmiller86 sorry for some missing information. The plot should show a simple simulation of an antenna emitting a Gaussian beam pulse to either direction as shown in the plot on the right side via yt. Visit weirdly separates it. I can not fix it easily with some built-in transformation functions. Is VisIt only applicable to 3D rendering? Thank you so much for your reply. I appreciate the great work the VisIt team has done!

JustinPrivitera commented 1 month ago

Is VisIt only applicable to 3D rendering?

No, VisIt handles 2D and 3D.

I haven't yet looked at your files but I suspect that this can be added to the list of problems that will be solved by upgrading our openPMD reader to use the openPMD API.

We have a lot of tickets these days so we will get to this when we can!

markcmiller86 commented 1 month ago

So, when working on the OpenPMD plugin last time, I noticed some differences in behavior depending on whether it was being run in serial or parallel. When I run two-processor parallel for this data, I get an even different answer than the one VisIt produced above... Screenshot 2024-07-12 at 2 48 10 PM

markcmiller86 commented 1 month ago

And for 5 processors...

Screenshot 2024-07-12 at 2 50 24 PM

cyrush commented 1 month ago

looks very much like a striding blunder

markcmiller86 commented 1 month ago

looks very much like a striding blunder

Thanks! I am trying to get to the bottom of it...still puzzled.

markcmiller86 commented 1 month ago

@Jeff62129 what can you tells us about what yt (and/or its dependencies such as h5py) are doing to the data once read from the file? You are comparing what the VisIt plugin is doing to (potentially) a lot of data interpretation in the python tools being invoked here.

My reason for asking is that there is no interpretation of the data being made in the VisIt plugin. It is obtaing the buffer of data directly from the file and mapping that buffer 1:1 with the rectilinear mesh it is defined on.

Now, in the above comments, I have identified a second issue...that the striding used to decompose the input data across parallel tasks appears to be wrong. Otherwise, we'd get the same result no matter how many processors we used. We don't. But, that is orthogonal to the problem you originally reported (and which I can reproduce with the data you've provided).

I see the below metadata in the file (I've removed the raw data). yt must be interpreting some of this because it is showing a square result but the number of points in each dimension is different (768 vs 128). What do the values open and reflecting in fieldBoundary mean? On the other hand, yt is getting some things wrong. The file says axisLabels are x and z which is what VisIt says but the result from yt is showing x and y.

So, I believe there is something yt is doing to interpret this data correctly and that logic does not exist in the VisIt plugin and we need help to get it resolved.

/                        Group
    Attribute: basePath scalar
        Type:      10-byte null-terminated ASCII string
        Data:  "/data/%T/"
    Attribute: date scalar
        Type:      26-byte null-terminated ASCII string
        Data:  "2024-07-11 17:11:58 +0800"
    Attribute: iterationEncoding scalar
        Type:      10-byte null-terminated ASCII string
        Data:  "fileBased"
    Attribute: iterationFormat scalar
        Type:      13-byte null-terminated ASCII string
        Data:  "openpmd_%06T"
    Attribute: meshesPath scalar
        Type:      8-byte null-terminated ASCII string
        Data:  "fields/"
    Attribute: openPMD scalar
        Type:      6-byte null-terminated ASCII string
        Data:  "1.1.0"
    Attribute: openPMDextension scalar
        Type:      native unsigned int
        Data:  1
    Attribute: software scalar
        Type:      6-byte null-terminated ASCII string
        Data:  "WarpX"
    Attribute: softwareVersion scalar
        Type:      8-byte null-terminated ASCII string
        Data:  "Unknown"
    Location:  1:96
    Links:     1
/data                    Group
    Location:  1:800
    Links:     1
/data/400                Group
    Attribute: dt scalar
        Type:      native double
        Data:  1
    Attribute: time scalar
        Type:      native double
        Data:  1.01792e-13
    Attribute: timeUnitSI scalar
        Type:      native double
        Data:  1
    Location:  1:1832
    Links:     1
/data/400/fields         Group
    Attribute: chargeCorrection scalar
        Type:      5-byte null-terminated ASCII string
        Data:  "none"
    Attribute: currentSmoothing scalar
        Type:      9-byte null-terminated ASCII string
        Data:  "Binomial"
    Attribute: currentSmoothingParameters scalar
        Type:      55-byte null-terminated ASCII string
        Data:  "period=1;compensator=false;numPasses_x=1;numPasses_z=1"
    Attribute: fieldBoundary {4}
        Type:      11-byte null-terminated ASCII string
        Data:  "open", "open", "reflecting", "reflecting"
    Attribute: fieldSolver scalar
        Type:      4-byte null-terminated ASCII string
        Data:  "Yee"
    Attribute: particleBoundary {4}
        Type:      10-byte null-terminated ASCII string
        Data:  "absorbing", "absorbing", "absorbing", "absorbing"
    Location:  1:2864
    Links:     1
/data/400/fields/B       Group
    Attribute: axisLabels {2}
        Type:      2-byte null-terminated ASCII string
        Data:  "z", "x"
    Attribute: dataOrder scalar
        Type:      2-byte null-terminated ASCII string
        Data:  "C"
    Attribute: fieldSmoothing scalar
        Type:      5-byte null-terminated ASCII string
        Data:  "none"
    Attribute: geometry scalar
        Type:      10-byte null-terminated ASCII string
        Data:  "cartesian"
    Attribute: gridGlobalOffset {2}
        Type:      native double
        Data:  -3e-05, -3e-05
    Attribute: gridSpacing {2}
        Type:      native double
        Data:  7.8125e-08, 4.6875e-07
    Attribute: gridUnitSI scalar
        Type:      native double
        Data:  1
    Attribute: timeOffset scalar
        Type:      native float
        Data:  0
    Attribute: unitDimension {7}
        Type:      native double
        Data:
            (0) 0, 1, -2, -1, 0, 0, 0
    Location:  1:4456
    Links:     1
/data/400/fields/B/x     Dataset {768/768, 128/128}
    Attribute: position {2}
        Type:      native double
        Data:  0.5, 0.5
    Attribute: unitSI scalar
        Type:      native double
        Data:  1
    Location:  1:5488
    Links:     1
    Chunks:    {128, 128} 131072 bytes
    Storage:   786432 logical bytes, 786432 allocated bytes, 100.00% utilization
    Type:      native double
    Data:
        (0,0) 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
         .
         .
         .
        (767,119) 0, 0, 0, 0, 0, 0, 0, 0, 0
/data/400/fields/B/y     Dataset {768/768, 128/128}
    Attribute: position {2}
        Type:      native double
        Data:  0.5, 0.5
    Attribute: unitSI scalar
        Type:      native double
        Data:  1
    Location:  1:795232
    Links:     1
    Chunks:    {128, 128} 131072 bytes
    Storage:   786432 logical bytes, 786432 allocated bytes, 100.00% utilization
    Type:      native double
    Data:

/data/400/fields/B/z     Dataset {768/768, 128/128}
    Attribute: position {2}
        Type:      native double
        Data:  0.5, 0.5
    Attribute: unitSI scalar
        Type:      native double
        Data:  1
    Location:  1:1584648
    Links:     1
    Chunks:    {128, 128} 131072 bytes
    Storage:   786432 logical bytes, 786432 allocated bytes, 100.00% utilization
    Type:      native double
    Data:

/data/400/fields/E       Group
    Attribute: axisLabels {2}
        Type:      2-byte null-terminated ASCII string
        Data:  "z", "x"
    Attribute: dataOrder scalar
        Type:      2-byte null-terminated ASCII string
        Data:  "C"
    Attribute: fieldSmoothing scalar
        Type:      5-byte null-terminated ASCII string
        Data:  "none"
    Attribute: geometry scalar
        Type:      10-byte null-terminated ASCII string
        Data:  "cartesian"
    Attribute: gridGlobalOffset {2}
        Type:      native double
        Data:  -3e-05, -3e-05
    Attribute: gridSpacing {2}
        Type:      native double
        Data:  7.8125e-08, 4.6875e-07
    Attribute: gridUnitSI scalar
        Type:      native double
        Data:  1
    Attribute: timeOffset scalar
        Type:      native float
        Data:  0
    Attribute: unitDimension {7}
        Type:      native double
        Data:
            (0) 1, 1, -3, -1, 0, 0, 0
    Location:  1:2374816
    Links:     1
/data/400/fields/E/x     Dataset {768/768, 128/128}
    Attribute: position {2}
        Type:      native double
        Data:  0.5, 0.5
    Attribute: unitSI scalar
        Type:      native double
        Data:  1
    Location:  1:2375520
    Links:     1
    Chunks:    {128, 128} 131072 bytes
    Storage:   786432 logical bytes, 786432 allocated bytes, 100.00% utilization
    Type:      native double
    Data:

/data/400/fields/E/y     Dataset {768/768, 128/128}
    Attribute: position {2}
        Type:      native double
        Data:  0.5, 0.5
    Attribute: unitSI scalar
        Type:      native double
        Data:  1
    Location:  1:3165264
    Links:     1
    Chunks:    {128, 128} 131072 bytes
    Storage:   786432 logical bytes, 786432 allocated bytes, 100.00% utilization
    Type:      native double
    Data:

/data/400/fields/E/z     Dataset {768/768, 128/128}
    Attribute: position {2}
        Type:      native double
        Data:  0.5, 0.5
    Attribute: unitSI scalar
        Type:      native double
        Data:  1
    Location:  1:3954680
    Links:     1
    Chunks:    {128, 128} 131072 bytes
    Storage:   786432 logical bytes, 786432 allocated bytes, 100.00% utilization
    Type:      native double
    Data:
markcmiller86 commented 1 month ago

Actually, I think what may be happening is that it is mixing up the x and y dimensions of the dataset in the file. I see datasets there of dimension 768, 128 but it is the x axis in the associated mesh that is getting 768 values and the y axis is getting 128 values.

markcmiller86 commented 1 month ago

Ok, yes indeed, I have proven to myself that the plugin is interposing the coordinate axes. When I change the plugin to treat the y-values as the x coordinates and the x-values as the y coordinates of the underlying mesh (which also changes the # points it is expecting in x and y dimensions), I get the plot you get but rotated 90 degrees...

Screenshot 2024-07-15 at 8 02 05 PM

markcmiller86 commented 1 month ago

I think the issue maybe how the plugin is handling the axisLabels metadata. We have other test data with...

scratlantis:databases/OpenPMD/OpenPMDClasses] miller86% h5ls -vlr ~/visit/visit/data/openpmd_test_data/data00000500.opmd | grep -A 3 -i label
    Attribute: axisLabels {3}
        Type:      1-byte null-padded ASCII string
        Data:  "x", "y", "z"
    Attribute: dataOrder scalar
--
    Attribute: axisLabels {3}
        Type:      1-byte null-padded ASCII string
        Data:  "x", "y", "z"
    Attribute: dataOrder scalar

which plots fine but note that your data has...

/data/400/fields/E       Group
    Attribute: axisLabels {2}
        Type:      2-byte null-terminated ASCII string
        Data:  "z", "x"

which places "x" as the second of the two dimensions in the file and not the first. VisIt's plugin is not coded to deal with this.

Jeff62129 commented 1 month ago

Hi @markcmiller86,

Thank you for your response. Unfortunately, I'm not familiar enough with the underlying mechanics of yt to provide a detailed explanation. However, I can share that I encountered some unusual results when attempting to visualize particle-related quantities with yt. It is worth noting that WarpX does not implement OpenPMD-related I/O operations directly but instead utilizes the OpenPMD-API provided by the OpenPMD team.

Interestingly, all the visualization issues I experienced with VisIt were accurately resolved using the library available in the official OpenPMD GitHub repository.

Do you think it's viable that VisIt also uses openPMD-api to read opmd files?

Thank you for your assistance, and I hope this information is helpful.

JustinPrivitera commented 1 month ago

Do you think it's viable that VisIt also uses openPMD-api to read opmd files?

We are already planning to add this feature. I can't say yet when we will have it in but this will certainly factor into our upcoming prioritization discussions for our next release.

markcmiller86 commented 1 month ago

@JustinPrivitera, I have a somewhat vague idea how I might proceed on the existing plugin to correct the originally reported issue. We can use the data provided here to create a test as well.

But, in working through this I have identified at least 3 other issues...

My question is, should I pause activity here until we've decided the path forward?

JustinPrivitera commented 1 month ago

My question is, should I pause activity here until we've decided the path forward?

I think so. Let's discuss, either today or on Thursday.

markcmiller86 commented 1 month ago

Just capturing here another issue I discovered in the plugin code. In this line,

https://github.com/visit-dav/visit/blob/282449eb6b5573ad96dc90bcbb62c9d836044e9e/src/databases/OpenPMD/OpenPMDClasses/PMDFile.C#L561

It is declaring the memory type into which the data is being read to be the data type of the dataset in the file (e.g. the file's datatype), from here...

https://github.com/visit-dav/visit/blob/282449eb6b5573ad96dc90bcbb62c9d836044e9e/src/databases/OpenPMD/OpenPMDClasses/PMDFile.C#L545

It should really be setting the memory type, datasetType like so...

datasetType = ((dataSize == 4) ? H5T_NATIVE_FLOAT : H5T_NATIVE_DOUBLE);

Because the memory type into which the data is being read is whatever the executable's float or double is.