search

nanoporetech / vbz_compression

VBZ compression plugin for nanopore signal data
https://nanoporetech.com/
Mozilla Public License 2.0
39 stars 9 forks source link

Using plugin from within R/RStudio on Windows #21

Open nemitheasura opened 2 years ago

nemitheasura commented 2 years ago

Hello, I am asking you for troubleshooting regarding using vbz plugin & R.

I want to access raw signal data stored within fast5 files and plot the signals in R (more specific: I want to create a Rmarkdown document with some plots). I have a vast collection of fast5 files- some are older, some are new, some were compressed with vbz algo and some are using older compression methods.

Before I decided to post an issue, I did my research - e.g. followed the hints posted there: https://github.com/nanoporetech/vbz_compression/issues/20 https://github.com/adnaniazi/tailfindr/issues/12

I want my document to be usable on Linux and Win operating systems. I have Linux Mint Vanessa and Win11.

All necessary dependencies to access and browse hdf5 files are installed on both systems - according to tutorials provided and to aforementioned threads (above). The hdf5 libraries, R packages (rhdf5, hdf5filters, Rhdf5lib) and Rtools in case of Windows are present. Off corse the plugin itself too. The plugin file was also copied to the rhdf5filters directory, so it would be accessible from within R. All software was installed in standard directories, no custom dirs were used.

This strategy allows me to access & plot signals from vbz compressed fast5 files only on Linux. On Windows it crashes R session immediately throwing a fatal error.

I am even unable to access the signal data from within R on Windows11, not to mention plotting. Just accessing & trying to store them as vec. Not a chance - fatal error, session restart.

Interestingly, the fast5 file itself is accessible, I can browse sections which were not subjected to vbz compression, so for instance, I can access and display configuration data, sampling rate, minKNOW version etc. I am only unable to access the signal data.

Off course I can change the compression algo and then plot my files under Win11 OS, but this is additional step, which I would like to avoid since there is a plugin which should solve the problem without using h5repack and convert the data back and forth with ont-fast5-api.

Also, solution provided in this thread https://github.com/adnaniazi/tailfindr/issues/12 allows me to produce a nonempty csv file using test data provided by tailfindr maintainer, while on my data, on Windows, the produced file is blank.

This is not a problem on Linux Mint and a few other Linux distros I have already tested. The problem seems to appear only on Windows. It is just like this plugin would require other actions to be recognizable/accessible by R.

Here is my session_info() on Windows (this was initially a Win10 upgraded to 11).

0x55555555 commented 2 years ago

Hi @nemitheasura ,

Hopefully some of the above points would provide clues to what is going on... I have not worked with R myself, but hdf should work transparently... Do you know what version of hdf is actualy used internally by the R package?

Thanks,

nemitheasura commented 2 years ago

Hi @jorj1988 ,

Ad1. The error message is rather vague, I'd say (screen attached). Bez tytułu

Ad2. I tried all 3 releases from the repo (1.0.0, 1.0.1 and 1.0.2) with the same result.

Ad3. Hdfview shows typical tree-like structure of multifast5 file. The vbz compressed signal data can be displayed too (screens attached). hdf_file signal

Ad4. normal, ordinary output. Nothing suspicious (screen attached, dumped with -H option) cmd

Ad5. yes ont_fast5_api

Ad.6 Currently, I use HDF5 1.12.2, but I also tested all releases from 1.10 to the newest one, hoping that it would help somehow. Nothing helped.

EDIT: SOLUTION

While attempting to debug this, I encountered a following thread on SO: https://stackoverflow.com/questions/72067335/r-session-aborted-and-r-encountered-a-fatal-error

The problem described here has nothing in common with the plugin and nanopore data, however this also concerns fatal error produced by R/Rstudio. In all previous attempts I used official R releases for Windows from recommended installation packages with default options. This time I gave a try and installed patched R 4.2.1 build for Windows from the following link mentioned in this thread: https://cran.r-project.org/bin/windows/base/rpatched.html

Then I installed compatible Rtools and devtools and rhdf5 package from bioconductor (as it is absent from CRAN for some reason). The session_info() showed me that there is some issue with rhdf5filters (md5sum differs) so I went to the github repository and installed rhdf5filters directly from there (https://github.com/grimbough/rhdf5filters).

After all of those steps I went to the plugin directory, copied the plugin and library files and placed them into the lib directory within rhdf5filters dir (in my case it was C:\Users\nemitheasura\AppData\Local\R\win-library\4.2\rhdf5filters\lib\x64)

This finally solved the issue. The plugin is now recognized properly by R and the fast5 files are completely browsable. I post it here in case if someone would encounter the same issue while working with vbz compressed nanopore data in R.

Best, N.