search

fli-iam / shanoir-ng

Shanoir (SHAring iN vivO Imaging Resources)
GNU General Public License v3.0
21 stars 21 forks source link

Mass deletion of datasets #2138

Open chomo-ofsep opened 6 months ago

chomo-ofsep commented 6 months ago

adding a feature allowing mass deletion of datasets by giving the list of datasets to delete

michaelkain commented 6 months ago

Hi @chomo-ofsep, do you have a preference, to integrate this feature into shanoir-downloader.py, that you can drive it from an Excel sheet on your side, or if the web GUI would allow you to select a big list of datasetIds and create a job in Shanoir to delete it?

chomo-ofsep commented 6 months ago

Hi @michaelkain, for us it probably would be easier via downloader but to make it accessible, it would be better on the webGUI. I guess it also depends on how much time it takes for each solution to develop. How would the selection on the web GUI work ? through the SOLR search ? I discovered that if you want to search multiple datasets with the expert mode, you don't need to type "OR" between each dataset, you can just separate each dataset by a space and it works. So I can just copy the dataset ids from an excel sheet and copy in the search bar in Shanoir.

youennmerel commented 6 months ago

Hi @michaelkain, for us it probably would be easier via downloader but to make it accessible, it would be better on the webGUI. I guess it also depends on how much time it takes for each solution to develop. How would the selection on the web GUI work ? through the SOLR search ? I discovered that if you want to search multiple datasets with the expert mode, you don't need to type "OR" between each dataset, you can just separate each dataset by a space and it works. So I can just copy the dataset ids from an excel sheet and copy in the search bar in Shanoir.

Hi @chomo-ofsep,

Do you have an idea of the volume of dataset you would want to delete at once ? There is a query size limit in Solr, and If i remember well my test for OFSEP Sequence Identifier execution, you can input something like 500 datasets max in the research bar.

chomo-ofsep commented 6 months ago

it would definitely be a lot more than 500 at a time. might go over 10'000 at once.

michaelkain commented 5 months ago

I propose to do this deletion via a Python script? probably to discuss.

michaelkain commented 5 months ago

Python script is fine for CH

jcomedouteau commented 3 months ago

See https://github.com/Inria-Empenn/shanoir_downloader/pull/12 for review