jjhelmus / nmrglue

A module for working with NMR data in Python
BSD 3-Clause "New" or "Revised" License
208 stars 84 forks source link

Process_pipe_3d:There seems to be some issues with the processing script #187

Open 18270020070 opened 1 year ago

18270020070 commented 1 year ago
 1. We used the nmrglue 3D data processing script to process a 3D full-sampled bruker data, but the processing results are quite different from those processed by NMRPipe. The results are as follows, where a, b, and c are the results of NMRPipe processing, and d, e, and f are nmrglue processes the results.

a: G1ANJ@OCKNJUA9$N_0{O_%9 b: KA7F19}~X3N4 GS6C8 5D`2 c: S0%N9SVJI11Z MESM0WA6`4 d: Figure_xyplane e: Figure_xzplane f: Figure_yzplane At the same time, we use NMRPipe to process the direct dimension of the 3D data, and then use nmrglue to process the two indirect dimensions of the 3D data and find that the results are consistent with the NMRPipe results. So we think the problem should lie in the processing of the direct dimension. 2.Also, it seems that nmrglue's bruker and varian also have problems reading raw data. It is normal when we use bruker and varian to read 3D fully sampled raw data, but when we use bruker and varian to read 3D undersampled raw data, the read data becomes 2D data. We hope the author can provide some help, it is greatly appreciated!

kaustubhmote commented 1 year ago

Can you share more information about what these spectra are? Are they acquired in the Echo-Antiecho manner or the States/TPPI/States-TPPI?

kaustubhmote commented 1 year ago

Also, can you share any of the undersampled datasest to debug this issue?

18270020070 commented 1 year ago

Can you share more information about what these spectra are? Are they acquired in the Echo-Antiecho manner or the States/TPPI/States-TPPI?

This is a 3D HNCACB spectrum of a small protein GB1-HttNTQ7 data, its first indirect dimension is acquired in the Echo-Antiecho manner and second indirect dimension is acquired in States-TPPI.As shown in Figure 1. Figure 1: UKAV3UN_UX59`4X~%SBHZ$B

18270020070 commented 1 year ago

Also, can you share any of the undersampled datasest to debug this issue?

Of course, I will privately send it to your email.

kaustubhmote commented 1 year ago

In this case, you will have to manually do the echo-antiecho processing. You can take a look at #149 for suggestions on how to do this.

18270020070 commented 1 year ago

Also, can you share any of the undersampled datasest to debug this issue?

Of course, I will privately send it to your email.

Sorry, I have privately sent the undersampling data to your email kaustubh.mote@gmail.com three times, but they were all returned. Do you have another email address?

kaustubhmote commented 1 year ago

Strange. Maybe you can share the folder in dropbox/google-drive and send the link.

kaustubhmote commented 1 year ago

About reading the non-uniformly sampled data: the ser file in non uniformly sample data generated by Bruker spectrometers is always stored as a 2D. This is the probably appropriate as this is also acquired in a randomized manner, and we can use the nuslist file in the data folder to reshape the data appropriately when required. As such, nmrglue reads it as a 2D data. I believe this should be considered as a bug in nmrglue, and one should write a custom code to check whether the data is non-uniformly sampled, but a simple hack will fix this for now:

dic, data = ng.bruker.read(".") 
data = data.reshape(*data.shape, 1) # reshapes data to a 3d
udic = ng.bruker.guess_udic(dic, data) # this now reads 3 dimensions correctly
data = data.reshape(data.shape[:2]) # reshape back to original 2d if necessary

I will add the custom code for nus when I find some time to do it. For now I have added the bug label to this.

18270020070 commented 1 year ago

Thank you very much for your reply, this is a simple and effective method, however, in this case the third dimension is 1, maybe I should use the nuslist file to reshape the data to the three dimensions I want.

kaustubhmote commented 1 year ago

I just realized that bruker seems to also save a file called ser_full , which is the ser file that has zeros as placeholders for fids that are not acquired. So this should also work:

dic, data = ng.bruker.read(".", bin_file="ser_full") 
data = data.reshape(128, 128, 1024) # reshapes data to a 3d, these numbers seem to be lost in acqus files
udic = ng.bruker.guess_udic(dic, data) # this now reads 3 dimensions correctly

This still needs a proper fix though.

18270020070 commented 1 year ago

Yes, but I am not sure whether the file "ser_full" comes with the raw data or is generated by bruker, as shown in the red box in Figure 1. Figure1:
QF3M${K%{( `T{JHESGF0YP

kaustubhmote commented 1 year ago

Ah, it seems to be coming from nmrpipe! No problem. Then this is simpler to work with.

18270020070 commented 1 year ago

Yeah,but in this case, we first need to read the raw data with the help of nmrpipe. We hope that nmrglue can directly read the undersampled raw data and reshape it into a three-dimensional array (for example, the data is 128x128x1024).