read_spc issue with w data

[BenoitCarreres]

Current version of R: 4.2.1. However, this is not an issue related to the version of R.

I haven't tested all the versions, but the last known version that was working to me was "HyperSpec 0.99-20171005" This issue is specific to w data point and has been existing for a while. Still present in github's version of "hySpc.read.spc", and was present before the latest version of R in cran of the "hyperSpec" package ("read.spc").

While the crash info does not reveal much, I have found the problematic lines: 784 and 816 https://github.com/r-hyperspec/hySpc.read.spc/blob/develop/R/read_spc.R The line contains the following: data[s, "w"] <- hdr$subhdr$w However, when looking into the hdr$subhdr object there is nothing called w, hence the error. I may be wrong, but I suspect that names must have been changed and should now be refering to subwlevel: data[s, "w"] <- hdr$subhdr$subwlevel

This is what my hdr$subhdr object contains:

> str(hdr$subhdr)
List of 9
 $ subflgs  : raw 00
 $ subexp   : int -128
 $ subindx  : int 4
 $ subtime  : num -46
 $ subnext  : num -41
 $ subnois  : num 0
 $ subnpts  : int 737
 $ subscan  : int 0
 $ subwlevel: num -4109

Note that this crashed at the first round of the for loop, and I have pretty much this problem on all files.

For testing, I can only refer to those available on zenodo for testing. Just to be sure, I have tested the file called "200528-7 A1.spc". https://doi.org/10.5281/zenodo.5607596 I am afraid you might have to download the 30gb archive to extract that 400 mb example. If inconvenient, I guess we could find some way to transfer only that file of interest.

---

Messages returned in the terminal:

Loading file 1/1: 200528-7 A1.spc
Warning: Error in <-: replacement has length zero
  1: runApp
SEND {"busy":"idle"}
Session stopped

Messages returned while manually testing. The message is the same if running the whole function:

> data[s, "w"] <- hdr$subhdr$w
Error in x[[jj]][iseq] <- vjj : replacement has length zero

Of course, running with the proposed fix does not crash. I am still testing if this is fine data wise. Also, I do not remember if I even need these data.

[BenoitCarreres]

Update:

I see that the lines 461:467 have been commented out in the new version of the code compared to the version "0.99-20171005". As suspected, it makes de bridge subwlevel to w.

As in Version 0.99-20171005:

## the w values
    if (hdr$fwplanes > 0) {
        if (subhdr$subwlevel != 0) {
            subhdr$w <- subhdr$subwlevel

        } else if (subhdr$subindx %% hdr$fwplanes == 1)
            subhdr$w <- hdr$subhdr$w +  hdr$fwinc
        else
            subhdr$w <- hdr$subhdr$w
    }

In latest version:

    # if (subhdr$subwlevel != 0) {
    #   subhdr$w <- subhdr$subwlevel
    #
    # } else if (subhdr$subindx %% hdr$fwplanes == 1)
    #   subhdr$w <- hdr$subhdr$w +  hdr$fwinc
    # else
    #   subhdr$w <- hdr$subhdr$w

[GegznaV]

@BenoitCarreres, thank you for the detailed investigation of the issue.

@cbeleites, are you aware of this?

[wincowgerDEV]

Another recent issue with read.spc() https://github.com/wincowgerDEV/OpenSpecy-package/issues/159

For some reason this spc file reads fine when on a shiny app but not when read from desktop. In shiny apps, they have a special file storage mechanism that I believe overwrites common naming issues. I did try to rename the file but that didn't seem to resolve the issue.

Here is the data and a renamed version: 10.7.zip

Code and error message:

> read.spc("10.7.spc")
Error in gsub(rawToChar(replace.nul), "\r\n", log.txt) : 
  'pattern' is invalid
In addition: Warning message:
In gsub(rawToChar(replace.nul), "\r\n", log.txt) :
  unable to translate '<ff>' to a wide string