genouest / biomaj-download

Download microservice for BioMAJ
GNU Affero General Public License v3.0
1 stars 7 forks source link

Issue if size is not present for http download #2

Closed osallou closed 7 years ago

osallou commented 7 years ago

When listing a remote http page for download (not directhttp), to parse an html page to get one or several files for example, and size is not present, workflow fails because size is mandatory with properties:

http.group.dir.name=1 http.group.dir.date=2 http.group.file.name=1 http.group.file.date=2 http.group.file.size=3

if http.group.file.size is not set, program should default to a 0 size in download/http.py at rfile['size'] = foundfile[int(self.config.get('http.group.file.size'))-1]

kinda:

if self.config.get('http.group.file.size', default=''): rfile['size'] = foundfile[int(self.config.get('http.group.file.size'))-1] else: rfile['size'] = 0

Refers to global biomaj issue https://github.com/genouest/biomaj/issues/76

osallou commented 7 years ago

proposal:

set field value to -1 if not set:

http.group.file.size=-1

Works for date and size. Size is set to 0 in this case and date is set to today.