ashvardanian / StringZilla

Up to 10x faster strings for C, C++, Python, Rust, and Swift, leveraging SWAR and SIMD on Arm Neon and x86 AVX2 & AVX-512-capable chips to accelerate search, sort, edit distances, alignment scores, etc 🦖
https://ashvardanian.com/posts/stringzilla/
Apache License 2.0
1.92k stars 64 forks source link

[CLI] sz_split error #139

Open lborcard opened 3 months ago

lborcard commented 3 months ago

Thank you very much for sharing this great project. I am using the cli tool sz_split and it's stopping after the first split when I specify -n 20 . It produces only 1 of the 20 files and stops with this error message:

An error occurred: [Errno 0] Error: 'X0'

I am splitting a .fast5 (hdf5 files for sequencing data). sz_split -n 20 gxb03325_20240327_1204_FAX92537_X5_sequencing_run_no_sample_ac8a3c2b_b3c8a5e7.fast5 best,

Loïc

ashvardanian commented 3 months ago

That's interesting, @lborcard, thank you for submitting the issue! Would it be possible to share the file URL or a simple test case that I can use to replicate the issue?

lborcard commented 3 months ago

thank your answer, yes I can give you an example file. here is an example: https://s3.amazonaws.com/nanopore-human-wgs/bulkfile/GXB02001_20230509_1250_FAW79338_X3_sequencing_run_NA12878_B1_19382aa5_ef4362cd.fast5

I will try to split and see if I run into the same error.

thanks for your reply

edit: the error is reproducible with the file posted

An error occurred: [Errno 0] Error: 'X0'
Usage example: split.py [-l LINES] [file] [prefix]
lborcard commented 3 months ago

I am pretty sure that the python split.py does not work with the -n flag because it does work without any flags

MarkReedZ commented 2 months ago

@lborcard The split works for me. Can you provide details on your machine? OS ?

MarkReedZ commented 2 months ago

We should perhaps use logging.exception or traceback. The error message above just prints the X0 filename - likely due to running out of disk space? Looking at the code its not obvious what possible exception was thrown with the filename as the message.

https://stackoverflow.com/questions/1483429/how-do-i-print-an-exception-in-python

ashvardanian commented 2 months ago

@MarkReedZ sounds reasonable.

lborcard commented 2 months ago

@lborcard The split works for me. Can you provide details on your machine? OS ? Sorry for the late reply, I am working on centos 8

lborcard commented 2 months ago

I tested again today after updating to 3.84 and I get the same error message