Feature request - BioCache download improvements

[sat01a]

Raised by Rosemary:

We've had a support request re ALA downloads (biocache recs). The request is to have an option to exclude null columns or have a set of check boxes to select the fields required (urggh) for the CSV.

The client indicated it takes them a lot of time every time they download to clean up the CSV files - I'm guessing they do it manually.

So... Doug suggested I talk to you and raise a feature request. I have no idea where in github to do that specifically.

I figure it won't be a trivial exercise to implement the feature but good to have it flagged.

[adam-collins]

I don't think this is a high priority because it should not be difficult to remove empty columns if required. Users already need to deal with a very large number of columns.

The inclusion of empty columns makes it clear that there are no values for that column. Note that the assertions column has an option to include empty assertion related columns. The qa parameter controls this.

If going ahead with this change, there is a function that unpacks the zipped download file to update the header when columns vary, DownloadService.insertMiscHeader. This can be used to read the records file (tsv or csv), count the number of values per column and remove empty columns. The headings.csv should also be updated to remain in sync with the records file.

[djtfmartin]

I don't think this is a high priority because it should not be difficult to remove empty columns if required. Users already need to deal with a very large number of columns.

I agree with @adam-collins and this is tricky to implement

[nielsklazenga]

If they do it regularly, they should use the offline download (web service). That can be scheduled and lets you choose exactly the columns you want.

[adam-collins]

will not implement