search

jywarren / spectral-workbench

Web-based tools for collecting, analyzing, and sharing data from a DIY spectrometer
http://spectralworkbench.org
GNU General Public License v3.0
95 stars 12 forks source link

Provide programmatic import/export of spectra #134

Closed tclarke closed 12 years ago

tclarke commented 12 years ago

It would be useful to have an import/export (using common CSV and other formats) either manually or via a web service so that spectra can be uploaded from external sources and downloaded for use in other tools (for example, http://opticks.org).

jywarren commented 12 years ago

Hi, tclarke - export in CSV, XML, and JSON is already implemented, check lower down on a spectrum page. However you're totally right about import, and I've been asking folks for sample data from another instrument, or an online (open source) resource which we can import. I have a copy of the ASTER database but it's for remote sensing, so dunno if it's applicable. Check out the issue here; i'm going to close this one as a duplicate but feel free to refine it and reopen if you disagree.

https://github.com/jywarren/spectral-workbench/issues/54

tclarke commented 12 years ago

I see it now. It would be useful to document (inline or on the download page) what the units are. For example, the CSV has wavelength, avg, r, g, and b. The wavelength seems to be in nm but it’s not obvious if the other values are in 0-100 reflectance/transmittance or 0-255 raw (or some other value). Generally, CSV spectra would be passed as reflectance/transmittance/emissivity floats in 0.0-1.0 (or > 1.0 for emissive) so this might be a better way to transmit for compatibility with other tools. Also, CSV are often 2, 3 or 4 column with wavelength then reflectance/transmittance (as you have) but often col 3 (in a 3 column) is FWHM of the reading. (cols 3-4 are start/end in 4 column) This is more often used for remote sensing spectra but could have use here especially if people begin uploading from other hardware with different spectral resolution. I’d suggest adding a FWHM (can be easily calculated with a default value based on the separation of the wavelength centers…use the point between two centers as the start/end point) before the r, g, b values for possible future use with different hardware.

I can put this info into a separate ticket if you would rather.

From: Jeffrey Warren [mailto:notifications@github.com] Sent: Monday, October 01, 2012 11:13 AM To: jywarren/spectral-workbench Cc: Clarke, Trevor Subject: Re: [spectral-workbench] Provide programmatic import/export of spectra (#134)

Hi, tclarke - export in CSV, XML, and JSON is already implemented, check lower down on a spectrum page. However you're totally right about import, and I've been asking folks for sample data from another instrument, or an online (open source) resource which we can import. I have a copy of the ASTER database but it's for remote sensing, so dunno if it's applicable. Check out the issue here; i'm going to close this one as a duplicate but feel free to refine it and reopen if you disagree.

jywarren/spectral-workbench#54 https://github.com/jywarren/spectral-workbench/issues/54

— Reply to this email directly or view it on GitHub https://github.com/jywarren/spectral-workbench/issues/134#issuecomment-9035274 .

This message and any enclosures are intended only for the addressee. Please

notify the sender by email if you are not the intended recipient. If you are

not the intended recipient, you may not use, copy, disclose, or distribute this

message or its contents or enclosures to any other person and any such actions

may be unlawful. Ball reserves the right to monitor and review all messages

and enclosures sent to or from this email address.

jywarren commented 12 years ago

That would be great. The documentation could be a wiki page on publiclaboratory.org we could link to from next to the exports. On Oct 1, 2012 1:54 PM, "Trevor R.H. Clarke" notifications@github.com wrote:

I see it now. It would be useful to document (inline or on the download page) what the units are. For example, the CSV has wavelength, avg, r, g, and b. The wavelength seems to be in nm but it’s not obvious if the other values are in 0-100 reflectance/transmittance or 0-255 raw (or some other value). Generally, CSV spectra would be passed as reflectance/transmittance/emissivity floats in 0.0-1.0 (or > 1.0 for emissive) so this might be a better way to transmit for compatibility with other tools. Also, CSV are often 2, 3 or 4 column with wavelength then reflectance/transmittance (as you have) but often col 3 (in a 3 column) is FWHM of the reading. (cols 3-4 are start/end in 4 column) This is more often used for remote sensing spectra but could have use here especially if people begin uploading from other hardware with different spectral resolution. I’d suggest adding a FWHM (can be easily calculated with a default value based on the separation of the wavelength centers…use the point between two centers as the start/end point) before the r, g, b values for possible future use with different hardware.

I can put this info into a separate ticket if you would rather.

From: Jeffrey Warren [mailto:notifications@github.com] Sent: Monday, October 01, 2012 11:13 AM To: jywarren/spectral-workbench Cc: Clarke, Trevor Subject: Re: [spectral-workbench] Provide programmatic import/export of spectra (#134)

Hi, tclarke - export in CSV, XML, and JSON is already implemented, check lower down on a spectrum page. However you're totally right about import, and I've been asking folks for sample data from another instrument, or an online (open source) resource which we can import. I have a copy of the ASTER database but it's for remote sensing, so dunno if it's applicable. Check out the issue here; i'm going to close this one as a duplicate but feel free to refine it and reopen if you disagree.

jywarren/spectral-workbench#54 < https://github.com/jywarren/spectral-workbench/issues/54>

— Reply to this email directly or view it on GitHub < https://github.com/jywarren/spectral-workbench/issues/134#issuecomment-9035274> .

This message and any enclosures are intended only for the addressee. Please

notify the sender by email if you are not the intended recipient. If you are

not the intended recipient, you may not use, copy, disclose, or distribute this

message or its contents or enclosures to any other person and any such actions

may be unlawful. Ball reserves the right to monitor and review all messages

and enclosures sent to or from this email address.

— Reply to this email directly or view it on GitHubhttps://github.com/jywarren/spectral-workbench/issues/134#issuecomment-9042849.