Stream read error bug

[Claudia-CC]

Dear Xiuwen,

I identified a bug leading to the error "Stream read error" during manipulation of GDS objects : I have to open and close a gds file several successive times, sometimes adding new nodes, sometimes modifying existing nodes (using replace = TRUE option in add.gdsn)... At a certain point, I get the error "Stream read error" when I try to re-open a GDS file : _Error in openfn.gds(gdsfile, FALSE, TRUE, TRUE) : Stream read error. I tried to use delete.gdsn and add.gdsn instead of using the replace option but I did not succeed to fix the error. I also tried to use sync.gds just after adding/modifying a node but no success neither.

Please find in my github repository https://github.com/claudiaQB/gdsfmt_bugstream.git a reproducible example using the official docker R image and the latest gdsfmt github version. To run this example, just type:

git clone https://github.com/claudiaQB/gdsfmt_bugstream.git
cd gdsfmt_bugstream
make

Thank you for your help.

[zhengxwen]

Here is "bug_gdsfmt.R" in your gdsfmt_bugstream:

library(gdsfmt)

gds_file <- 'bug.gds'

unlink(gds_file)

cnts <- TRUE

createfn.gds(gds_file, allow.duplicate = TRUE)

####################################################################
#                       FIRST
gds <- openfn.gds(gds_file, FALSE, TRUE, TRUE)
add.gdsn(gds, 'SAMPLES', val = iris, compress = '')
print('1. new node ok')
closefn.gds(gds)

if (cnts) {
  gds <- openfn.gds(gds_file, FALSE, TRUE, TRUE)
  add.gdsn(gds, 'CNTS', val = matrix(1:100, ncol = 10), valdim = c(10, 10))
  closefn.gds(gds)
}
####################################################################

####################################################################
#                       SECOND (NEW NODE)
gds <- openfn.gds(gds_file, FALSE, TRUE, TRUE)
add.gdsn(gds, 'SAMPLES2', val = rbind(iris, iris), compress = '')
print('2. new node ok')
closefn.gds(gds)

if (cnts) {
  gds <- openfn.gds(gds_file, FALSE, TRUE, TRUE)
  add.gdsn(gds, 'CNTS2', val = matrix(1:100, ncol = 10), valdim = c(10, 10))
  print('CNTS2')
  closefn.gds(gds)
}
####################################################################

####################################################################
#                       THIRD (REPLACE NODE)
gds <- openfn.gds(gds_file, FALSE, TRUE, TRUE)
add.gdsn(gds, 'SAMPLES', val = rbind(iris, iris), compress = '', replace = TRUE)
print('3. replace ok')
closefn.gds(gds)

if (cnts) {
  gds <- openfn.gds(gds_file, FALSE, TRUE, TRUE)
add.gdsn(gds, 'CNTS3', val = matrix(1:100, ncol = 10), valdim = c(10, 10))
print('CNTS3')
closefn.gds(gds)
}
####################################################################

####################################################################
#                       FOURTH (REPLACE NODE)
gds <- openfn.gds(gds_file, FALSE, TRUE, TRUE)
add.gdsn(gds, 'SAMPLES', val = rbind(iris, iris), compress = '', replace = TRUE)
print('4. replace ok')
closefn.gds(gds)

if (cnts) {
gds <- openfn.gds(gds_file, FALSE, TRUE, TRUE)
add.gdsn(gds, 'CNTS4', val = matrix(1:100, ncol = 10), valdim = c(10, 10))
print('CNTS4')
closefn.gds(gds)
}
####################################################################

[zhengxwen]

At first, please close the gds file immediately after creating a new one, since you re-open it in the next steps:

library(gdsfmt)

gds <- createfn.gds(gds_file, allow.duplicate = TRUE)
closefn.gds(gds)

But the error "Stream read error" still occurs in THIRD and FOURTH chucks! I guess the bug occurs in delete.gdsn, since add.gdsn(..., replace=TRUE) is actually equivalent to the combination of delete.gdsn and add.gdsn(..., replace=FALSE).

[zhengxwen]

The severity of this bug is low: it does not affect the GDS files which has been created, and there is no damage in GDS files.

[Claudia-CC]

Dear Zhengxwen,

Thank you for your fix, it solved my issue. I updated my git repository https://github.com/claudiaQB/gdsfmt_bugstream.git with this new gdsfmt version.

Best, Claudia