search

clbarnes / jeiss-convert

Convert Jeiss .dat files to HDF5
MIT License
2 stars 2 forks source link

first run through pytests took an hour #11

Open trautmane opened 1 year ago

trautmane commented 1 year ago

My first run through this repo's pytests took almost an hour because (I think) of an extremely slow download of https://neurophyla.mrc-lmb.cam.ac.uk/share/fibsem_example/FIBdeSEMAna_21-12-26_005024_0-0-0.dat.

Once the file is cached locally, things are better.

I don't know if there was an intermittent problem with the neurophyla.mrc-lmb.cam.ac.uk server when I first ran, but if it is possible to host the example files in a faster location that would be a welcome change. Alternatively, maybe we could purposely image something very small to use for tests and just keep the tiny dat files in the repo directly.

clbarnes commented 1 year ago

Yes, downloading those files is slow, although it shouldn't be that slow. You can add the --skip-full argument to pytest, which will skip tests requiring the full .dat file, but still do the tests on the truncated headers vendorised with jeiss-specs.

But as you say, ensuring that the public exemplar of a particular version is a small one would be convenient; that v8 .dat was from the only stack I had easily available.

clbarnes commented 1 year ago

You're right, it's coming down at 1.5MB/s, that's no good.