search

ncbi / datasets

NCBI Datasets is a new resource that lets you easily gather data from across NCBI databases.
https://www.ncbi.nlm.nih.gov/datasets
Other
355 stars 39 forks source link

`--debug` dumps a lot of binary to stderr #362

Closed corneliusroemer closed 4 months ago

corneliusroemer commented 4 months ago

When adding the --debug flag as in datasets download virus genome taxon 186538 --no-progressbar --filename results/ncbi_dataset.zip --debug the initial output (a log of HTTP requests and responses) is helpful - but at some point when one gets the actual results, the stderr gets flooded with binary (probably contents of the zip I'm requesting).

This makes it quite difficult to review the debug results post run. Would it be possible to truncate long HTTP responses after a certain threshold? Though I guess the following works: --debug >debug.log 2>&1

Debug logs are output to stderr, so one needs to make sure to redirect stderr into a file to prevent the flooding.

Example output (it's 100k more lines of binary like this)

image
ericcox1 commented 4 months ago

Hi @corneliusroemer,

Thanks for opening this issue. This is a known bug that should have been fixed as of 16.16.0.

Best, Eric

Eric Cox, PhD [Contractor] (he/him/his) NCBI Datasets NIH/NLM/NCBI eric.cox@nih.gov