Closed andrealorenzon closed 3 years ago
This again seems like an issue with the way quadrature in handled the indirect dimension while processing data using nmrglue. Can you share the scripts to help debug this?
sure.
This is my pipeline function:
def runPipeline (dic: dict, data: ndarray, dmxFlag: bool,
PS1p0:float, PS1p1:float,
PS2p0:float, PS2p1:float):
import nmrglue as ng
try:
os.remove('.data/tempSave/outputFile.ft2')
print("removed old outputFile")
except:
print("no file to remove")
dic, data = ng.pipe_proc.sp(dic, data,off=0.5, end=0.95, pow=2, c=0.5)
dic, data = ng.pipe_proc.zf(dic, data, zf=1, auto=True)
dic, data = ng.pipe_proc.ft(dic, data)
N=dic.get("FDDMXVAL",0)
if not dmxFlag:
dic, data = ng.pipe_proc.ps(dic, data, p0=0, p1=-360*N,)
dic, data = ng.pipe_proc.ps(dic, data, p0=PS1p0, p1=PS1p1)
dic, data = ng.pipe_proc.di(dic, data)
dic, data = ng.pipe_proc.tp(dic, data)
dic, data = ng.pipe_proc.sp(dic, data, off=0.5, end=0.95, pow=2, c=1)
dic, data = ng.pipe_proc.zf(dic, data, auto=True,)
dic, data = ng.pipe_proc.ft(dic, data, alt=True)
dic, data = ng.pipe_proc.ps(dic, data, p0=PS2p0, p1=PS2p1,)
dic, data = ng.pipe_proc.di(dic, data)
dic, data = ng.pipe_proc.tp(dic, data)
return dic, data
I think that I missed something about how to get from dic, data
, or from the original data source, all the information needed.
The data themselves are retrieved with this script, from the uploaded folder (it runs in a Streamlit app):
def openZipFromTmpPath(source: str):
"""retrieves dic, data from folder"""
import fs
import os
import nmrglue as ng
from zipfile import ZipFile
from pathlib import Path
from fs.tempfs import TempFS
from fs.zipfs import ZipFS
import streamlit as st
tmp_fs = fs.open_fs("temp://fidViewerBuffer")
with tmp_fs:
with fs.zipfs.ZipFS(source) as zip_fs:
fs.copy.copy_fs(zip_fs,tmp_fs)
all_dirs = os.listdir('/tmp/')
res = filter(lambda x: 'fidViewer' in x, all_dirs)
for r in res:
p = os.path.join('/','tmp',r)
folder = list(os.walk(p))[2][0]
tempDic, tempData = ng.bruker.read(folder)
return tempDic, tempData
def readZIP(uploaded_file):
""" unzip in temp directory"""
import fs
import os
import nmrglue as ng
from zipfile import ZipFile
from fs.tempfs import TempFS
from fs.zipfs import ZipFS
import streamlit as st
try:
dic, data = openZipFromTmpPath(uploaded_file)
data = ng.fileio.bruker.remove_digital_filter(dic, data)
#convert from bruker to pipe
udic = ng.bruker.guess_udic(dic, data)
C = ng.convert.converter()
C.from_bruker(dic, data, udic)
dic, data = C.to_pipe()
return dic, data
except Exception as e:
st.error(e)
The idea is to be able to open correctly ANY kind of 2D spectra, dynamically retrieving all required parameters from the files themselves. If any additional parameter has to be specified by the user, I'll make some control to enter its value. Problem is, depending from my lack of topic knowledge, for sure, to be able to understand which parameters can be inferred by nmrglue from the files, and which are to be inserted by the user, and where, to obtain a correctly oriented/rolled/processed data, and there's where I need some help.
The processing seems fine to me. Is the data from a sensitivity enhanced experiment? If so, you may need to do reshuffling of the data (Echo-Antiecho or Rance-Kay) before you process it. Normally, in nmrpipe, this is done at the conversion step, so you should look at your fid.com
file and see if it says either of those two things. If this is the case, nmrglue has a function in nmrglue/process/nmrtxt/rance_kay.py
which you can use. This parameter should also be in the dic["acqus"]["FnMODE"]
.
These are 2 examples of fid.com
files.
For data correctly visualized:
#!/bin/csh
bruk2pipe -verb -in ./ser \
-bad 0.0 -ext -aswap -AMX -decim 1680 -dspfvs 21 -grpdly 76 \
-xN 4096 -yN 1024 \
-xT 2048 -yT 512 \
-xMODE DQD -yMODE Complex \
-xSW 11904.762 -ySW 12003.656 \
-xOBS 600.183 -yOBS 600.183 \
-xCAR 4.771 -yCAR 4.771 \
-xLAB 1Hx -yLAB 1Hy \
-ndim 2 -aq2D States \
| nmrPipe -fn MULT -c 3.12500e+01 \
-out ./test.fid -ov
and for data that are not correctly visualized:
bruk2pipe -verb -in ./ser \
-bad 0.0 -ext -aswap -AMX -decim 1680 -dspfvs 20 -grpdly 68 \
-xN 2048 -yN 800 \
-xT 1024 -yT 400 \
-xMODE DQD -yMODE Echo-AntiEcho \
-xSW 11904.762 -ySW 15093.772 \
-xOBS 600.183 -yOBS 150.938 \
-xCAR 4.773 -yCAR 150.715 \
-xLAB 1H -yLAB 13C \
-ndim 2 -aq2D Complex \
| nmrPipe -fn MULT -c 8.33333e+00 \
-out ./test.fid -ov
I see the Echo-AntiEcho
in the latter.
This parameter should also be in the dic["acqus"]["FnMODE"].
At the moment that parameters is == 0 for both source files. It's still unclear how can I infer the correct procedure from dic
content.
Oh, I should have said dic["acqu2s"]["FnMODE"]
since this is the indirect dimension. Echo-AntiEcho should be == 6. The numbers correspond to the order in which the options appear in Topspin's drop-down menu for this parameter in the ACQUS tab.
dic["acqu2s"]["FnMODE"] == 5
for NOESY data, and == 6
for HSQC data.. great so know I know what can I infer the mode from. :) The only info I still miss is which transformation should I apply in either case to fix the reading of data.
You can look at the file nmrglue/process/nmrtxt/rance_kay.py
for the function that reshuffles the data. This seems to be unavailable directly from any module. Perhaps the best solution is to reshuffle the data manually right after it is read in.
Ok, I'll look into it. Thanks.
EDIT: great, it fixed the issue. Thank you.
Using
bruker.read()
to read from folders with Bruker experiments.For some experiments, everything seems fine.
In other cases, data appear to be modified: in this particular case, I expected the array to look like this: so, with top and bottom half inverted.
I can hardcode a fix with a
np.roll(data, -int(data.shape[0]/2), axis=0)
but I wanted to understand why it happens only sometimes. Could there be something I forgot?