Importing spectral data with common X (cm-1) values

[clsandt]

Hi I always have the problem when importing data from different experiments they end up being shifted in the data table (see image), either because they have sligthly different X values (cm-1) or because Orange detects that they are duplicate values and add an indice. This makes importing data from multiple measurements difficult. Unscrambler solved that by allowing some 'slack' for the X values, you could allow a % of deviation and Unscrambler would consider that they would be the same X values. Could that be implemented in the Multifile widget?

Cheers

C

[clsandt]

Also it makes it impossible to concatenate several data tables.

Marko did a script that could solve the issue in Orange 3 but it doesn't work in Quasar 7.

[markotoplak]

Thanks!

Yes, this "slack" could be part of Multifile. Which was the file format you used?

Does it help if you use Interpolate after loading the data? Could you paste the script here so that I fix it?

[borondics]

Yes, interpolation should be one way of fixing it...

On Tue, Feb 18, 2020, 19:25 Marko Toplak notifications@github.com wrote:

Thanks!

Yes, this "slack" could be part of Multifile. Which was the file format you used?

Does it help if you use Interpolate after loading the data? Could you paste the script here so that I fix it?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Quasars/orange-spectroscopy/issues/405?email_source=notifications&email_token=ABQDIGRXHMYEXRVY52RQJ6TRDQKZ5A5CNFSM4KXHX7H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMC3SUQ#issuecomment-587577682, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQDIGUXKGMUQ52HRJLYUL3RDQKZ5ANCNFSM4KXHX7HQ .

[clsandt]

Hello Marko, I had the same problem with different formats: .csv, opus and .spa. I also had the problem with different widgets (fle, multifile, csv reader). I did not try to interpolate the data.

Here is the script you wrote couple of centuries ago ;)

from Orange.data import Domain, ContinuousVariable, StringVariable, Table from orangecontrib.spectroscopy.data import build_spec_table import numpy as np

wavenumbers = in_data.X[:, 0]

labels = [] take_X = []

for a in in_data.domain.attributes[1:]: if isinstance(a, ContinuousVariable): labels.append(a.name) take_X.append(in_data[:, a].X[:, 0])

fnvar = StringVariable.make("Filename") lvar = StringVariable.make("Label")

fns = [in_data.name]len(labels) metas = list(zip(([labels, fns])))

mtable = Table.from_numpy(Domain([], None, [lvar, fnvar]), \ X=np.zeros((len(labels), 0)), \ metas=np.asarray(metas, dtype=object))

out_data = build_spec_table(wavenumbers, np.vstack(take_X), mtable)

Cheers

C

De : Marko Toplak [mailto:notifications@github.com] Envoyé : mardi 18 février 2020 18:26 À : Quasars/orange-spectroscopy Cc : SANDT Christophe; Author Objet : Re: [Quasars/orange-spectroscopy] Importing spectral data with common X (cm-1) values (#405)

Thanks!

Yes, this "slack" could be part of Multifile. Which was the file format you used?

Does it help if you use Interpolate after loading the data? Could you paste the script here so that I fix it?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/Quasars/orange-spectroscopy/issues/405?email_source=notifications&email_token=AE2OKNRXPLFVQO3YGHC4LQTRDQKZ5A5CNFSM4KXHX7H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMC3SUQ#issuecomment-587577682, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2OKNVI4QKOO5WNPDMC5FLRDQKZ5ANCNFSM4KXHX7HQ.