search

bio-guoda / preston

a biodiversity dataset tracker
MIT License
24 stars 1 forks source link

[preston ls] continues despite broken pipe #251

Open jhpoelen opened 1 year ago

jhpoelen commented 1 year ago

whereas:

preston ls\
 --anchor\
 hash://sha256/bc7368469e50020ce8ae27b9d6a9a869e0b9a2a0a9b5480c69ce6751fa4b870e\
 --remote\
 https://linker.bio,https://zenodo.org/record/8117720/files/,https://biokic6.rc.asu.edu/preston/gbpln\
 | grep urn:x-ncbi:gbpln.csv.gz

almost immediately produces: 

<urn:x-ncbi:gbpln.csv.gz> <http://purl.org/pav/hasVersion> <hash://sha256/18394187bb2fc70e6f2493e5654e0e9782292b4e1aa8988d35edf8ce9c4bd232> <urn:uuid:ba41a2d7-9cc0-468b-bd07-0018f6ed2581> .

a pipe with other downstream comments continues processing:

 --anchor\
 hash://sha256/bc7368469e50020ce8ae27b9d6a9a869e0b9a2a0a9b5480c69ce6751fa4b870e\
 --remote\
 https://linker.bio,https://zenodo.org/record/8117720/files/,https://biokic6.rc.asu.edu/preston/gbpln\
 | grep urn:x-ncbi:gbpln.csv.gz\
 | head -n1
[https://linker.bio/hash:...7e27cf0c2f4806771a82958] 72 kB at 0.32 MB/s completed in < 1 minute
[https://linker.bio/hash:...54f12f5758567e13f44d8bc] 72 kB at 0.34 MB/s completed in < 1 minute

Suggest to check whether preston ls is listening for a broken pipe, and verify that processing stop on detecting a broken pipe.

related to:

Poelen, Jorrit H. (2023). GenBank PLN (Plantae, Fungi, Algae) Sequence Index in TSV, CSV, JSONL formats hash://sha256/bc7368469e50020ce8ae27b9d6a9a869e0b9a2a0a9b5480c69ce6751fa4b870e hash://md5/f6f78f64e3b3ff06adc3229badbd578b (0.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.8117720