search

scenerygraphics / sciview

sciview is a tool for visualization and interaction with ND image and mesh data
BSD 2-Clause "Simplified" License
66 stars 17 forks source link

Automatic downloader for CREMI should catch invalid files and retry #453

Closed kephale closed 1 year ago

kephale commented 1 year ago

I closed sciview while downloading the CREMI sample, and the next time I ran it looked like it got stuck when reading the dataaset.

When I try to open the HDF5 through File > Open it reveals the issue

[ERROR] java.io.IOException: io.scif.img.ImgIOException: java.io.IOException: java.io.IOException: File is truncated should be= 175159177 actual = 22216441%nlocation= /Users/kharrington/.sciview/examples/sample_A_20160501.hdf
    at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:163)
    at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:133)
    at io.scif.io.DatasetIOPlugin.open(DatasetIOPlugin.java:94)
    at io.scif.io.DatasetIOPlugin.open(DatasetIOPlugin.java:52)
    at org.scijava.io.DefaultIOService.open(DefaultIOService.java:93)
    at org.scijava.io.DefaultIOService.open(DefaultIOService.java:68)
    at sc.iview.SciView.open(SciView.kt:744)
    at sc.iview.commands.file.Open.run(Open.java:73)
    at org.scijava.command.CommandModule.run(CommandModule.java:196)
    at org.scijava.module.ModuleRunner.run(ModuleRunner.java:163)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
    at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.scif.img.ImgIOException: java.io.IOException: java.io.IOException: File is truncated should be= 175159177 actual = 22216441%nlocation= /Users/kharrington/.sciview/examples/sample_A_20160501.hdf
    at io.scif.img.ImgOpener.createReader(ImgOpener.java:489)
    at io.scif.img.ImgOpener.openImgs(ImgOpener.java:242)
    at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:152)
    ... 16 more
Caused by: java.io.IOException: java.io.IOException: File is truncated should be= 175159177 actual = 22216441%nlocation= /Users/kharrington/.sciview/examples/sample_A_20160501.hdf
    at ucar.nc2.NetcdfFile.open(NetcdfFile.java:429)
    at ucar.nc2.NetcdfFile.open(NetcdfFile.java:396)
    at ucar.nc2.NetcdfFile.open(NetcdfFile.java:383)
    at ucar.nc2.NetcdfFile.open(NetcdfFile.java:371)
    at loci.formats.services.NetCDFServiceImpl.init(NetCDFServiceImpl.java:310)
    at loci.formats.services.NetCDFServiceImpl.setFile(NetCDFServiceImpl.java:103)
    at loci.formats.in.VeecoReader.initFile(VeecoReader.java:118)
    at loci.formats.FormatReader.setId(FormatReader.java:1421)
    at loci.formats.ImageReader.setId(ImageReader.java:849)
    at io.scif.bf.BioFormatsFormat$Parser.typedParse(BioFormatsFormat.java:472)
    at io.scif.bf.BioFormatsFormat$Parser.typedParse(BioFormatsFormat.java:450)
    at io.scif.AbstractParser.parse(AbstractParser.java:244)
    at io.scif.AbstractParser.parse(AbstractParser.java:314)
    at io.scif.AbstractParser.parse(AbstractParser.java:53)
    at io.scif.AbstractReader.setSource(AbstractReader.java:271)
    at io.scif.services.DefaultInitializeService.initializeReader(DefaultInitializeService.java:91)
    at io.scif.img.ImgOpener.createReader(ImgOpener.java:483)
    ... 18 more
Caused by: java.io.IOException: File is truncated should be= 175159177 actual = 22216441%nlocation= /Users/kharrington/.sciview/examples/sample_A_20160501.hdf
    at ucar.nc2.iosp.hdf5.H5header.readSuperBlock1(H5header.java:308)
    at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:208)
    at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:130)
    at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1566)
    at ucar.nc2.NetcdfFile.open(NetcdfFile.java:841)
    at ucar.nc2.NetcdfFile.open(NetcdfFile.java:426)
    ... 34 more
skalarproduktraum commented 1 year ago

I think just re-downloading automatically is not a good idea, because there's too many possible origins of the issue there. I'd rather catch the IOException and tell the user to delete the files manually. What do you think @kephale?

kephale commented 1 year ago

Even easier! That is fine with me.