Add support for CSV formatting - [merged] #90

Closed alexander-haller closed 2 months ago

In GitLab by @mhxion on Apr 12, 2024, 01:04

Merges csv-formatter -> new-bill-teams

For reading long list of response (like a list of teams or experiments) in JSON and YAML can be exhausting. A better solution that we've discussed before would be to introduce a ls/list sub-command for endpoints like teams, experiments, users etc. which will make finding a certain ID easier. For now, a cheap solution would be to add support for a more human-readable data format. This PR introduces CSV data format support across elAPI.

elAPI's layered design pattern makes it quite easier to add support for new data formats. We sub-class BaseFormat in src/elapi/styles/format.py to include support for a new data format. The new data format propagates across all elAPI commands that support --format/-F. That is, the developer would not need to update the CLI documentation or do anything else! The new CSVFormat essentially attempts to convert response from JSON to CSV. If it fails to do so—i.e., if the data structure is too complex to convert, it will show an error message and elAPI will fallback to JSON data format. So far, none of eLabFTW's responses are that complex.

Read experiments as `CSV`

The following command would print all experiments as CSV data on the terminal.

$ elapi get experiments -F csv

The printed CSV will not likely to be legible though. So piping the output to a command-like CSV viewer is recommended if one wishes to read the CSV output without leaving the terminal. Exporting the data would probably be better in this case.

Export teams as `CSV`

The following command would export all teams data as CSV to the $HOME directory.

$ elapi get teams -F csv --export ~

I believe this is something Philipp and hopefully other users can benefit from.

`elapi experiments get` sub-command with CSV formatting

The new CSVFormat essentially attempts to convert response from JSON to CSV.

elapi experiments get sub-command already supports formatting with CSV. It will still continue to use the CSV-formatted data received from eLabFTW instead of using this JSON-converted CSVFormat. This makes CSVFormat safe to use without worrying about conflict.

`bill-teams` plugin with CSV formatting

--format csv is disabled for bill-teams sub-commands teams-info, owners-info and consequently store-info. It is technically possible to convert the data generated by teams-info to CSV, however, the JSON data structure teams-info produces is really complex for a tabular data format like CSV, and converting it to CSV just gives broken and totally unreadable CSV data. Therefore, we disable CSV support for bill-teams, and as such the --help for bill-teams excludes CSV as one of the supported formats.

In GitLab by @mhxion on Apr 12, 2024, 01:05

requested review from @alexander-haller

In GitLab by @mhxion on Apr 12, 2024, 01:06

Most the commits are coming from new-bill-teams branch. Only the last commits (099c181 and 3d3e745) are the new commits for this branch.

In GitLab by @mhxion on Apr 12, 2024, 01:30

This makes CSVFormat safe to use without worrying about conflict.

Never mind. It does conflict with elapi experiments get --format csv 🤣. Well, that was a major find and a good lesson for me. I will work on fixing this later. new-bill-teams branch is unaffected of course. You can skip reviewing this for now.

Great - the only thing I'm unsure if elapi get experiments -F csv or elapi experiment get -F csv is better long-term.

low-prio

approved this merge request

In GitLab by @mhxion on Apr 12, 2024, 18:57

the only thing I'm unsure if elapi get experiments -F csv or elapi experiment get -F csv is better long-term.

We wouldn't be able to change elapi get as that's tied to the GET requests to eLabFTW. elapi get doesn't specifically target one endpoint (not hardcoded) like experiments but it just targets any supported endpoint (the list is documented in --help). elapi experiment get supports more functionalities like saving the experiment as PDF that elapi get cannot do. Naming is hard :D

Update: Fixed.

In GitLab by @project_994_bot_1c5bd6ac1fdbd740fbc4ed080ecff58d on Apr 14, 2024, 02:18

added 3 commits

f7c5997a - Move src/elapi/cli/helpers.py
08a50e3e - Fix how bill-teams gets CLI documentation
6b421ec5 - Fix how experiments plugin gets CLI documentation

Compare with previous version

In GitLab by @mhxion on Apr 14, 2024, 15:51

changed target branch from dev to new-bill-teams

In GitLab by @mhxion on Apr 14, 2024, 15:52

Fixed #27 with commit 08a50e3e and 6b421ec5.

In GitLab by @mhxion on Apr 14, 2024, 15:59

Thanks for approving btw. Will merge now.

uhd-urz / elAPI