search

ESGF / esgf-download

ESGF data transfer and replication tool
https://esgf.github.io/esgf-download/
BSD 3-Clause "New" or "Revised" License
15 stars 2 forks source link

Add a "filename" column to the download queue (feature request) #33

Closed meteorologist15 closed 1 hour ago

meteorologist15 commented 7 months ago

When an asynchronous download is being executed, there are 6 columns that are displayed and updated in real time: download #, queue ID, download progress bar, # MiB/GiB downloaded vs total, download rate, and the data node. This setup notably excludes the filename, a feature that the previous Synda contained while an asynchronous download took place. My request is to restore this previous feature, either by adding a column or replacing the data node column (is displaying the data node really necessary?). Thanks.

svenrdz commented 7 months ago

Hi !

To answer your question on the data_node column, it is intended for monitoring transfer speed during large batches of downloads and allows taking decisions such as stop and try another data node. Although this process is currently not as useful as I'd wish it to be, swapping data_node is still a manual process, so it is usually done through the python API rather than the CLI. I could make the column opt-out.

About the filename column, I've added it as an opt-in config in a local branch, which uncovered a bug with boolean options, I'll push it as soon as it's fixed properly.

Since I'm not totally sure how that feature looked with synda, does the following look good to you?

[1] <002fa1>  71% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━ · 128.0/180.6 MiB · 6.0 MB/s · vesg.ipsl.upmc.fr · co3satcalc_Oyr_IPSL-CM5B-LR_historical_r1i1p1_1950-2005.nc
[3] <008164>  40% ━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━ · 128.0/321.7 MiB · 6.0 MB/s · vesg.ipsl.upmc.fr · ph_Oyr_IPSL-CM5A-LR_historical_r6i1p1_1850-1949.nc        
[4] <0092b0>  71% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━ · 128.0/180.6 MiB · 5.4 MB/s · vesg.ipsl.upmc.fr · pcalc_Oyr_IPSL-CM5A-LR_historical_r4i1p1_1950-2005.nc     
[5] <014158>  40% ━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━ · 128.0/321.7 MiB · 6.4 MB/s · vesg.ipsl.upmc.fr · ph_Oyr_IPSL-CM5A-LR_historical_r1i1p1_1850-1949.nc
meteorologist15 commented 7 months ago

That is EXACTLY how I pictured it, yes. I understand that it does make the row display longer, but I personally feel that filenames are fairly important interactive tracking markers. I was just thinking to substitute it for the data_node to cut down the length of the row, but leaving it as is is fine with me. Perhaps indeed there could be some sort of "column addition" or "column subtraction" option (or as you said, 'opt-in'/'opt-out'), allowing users freedom to decide what info they'd like to monitor interactively (outside of a default number of columns).