search

linkml / linkml-runtime

Runtime support for linkml generated models
https://linkml.io/linkml/
Creative Commons Zero v1.0 Universal
24 stars 23 forks source link

Add new TSV loader/dumper classes #268

Closed pkalita-lbl closed 1 year ago

pkalita-lbl commented 1 year ago

This is related to https://github.com/linkml/linkml/issues/1337

Currently CSVDumper and CSVLoader, despite their names, are only capable of handling tab-separated strings. The underlying json-flattener library is capable of using either tabs or commas through its GlobalConfig object, but that option is not being explicitly set so the default '\t' is always used.

These changes repurpose the existing CSVDumper/Loader classes as DelimitedFileDumper/Loader classes with one abstract property, delimiter, which it does pass to json-flattener functions. The concrete subclasses CSVDumper/Loader and the new TSVDumper/Loader set the delimiter property appropriately.

This may be straddling the line between a bug fix and a breaking change. I think it's good that CSVDumper/Loader now handles, you know, actual CSV. But I'm slightly concerned that some people might be relying on the current TSV from CSVDumper/Loader.

codecov-commenter commented 1 year ago

Codecov Report

Merging #268 (a9bb788) into main (74d699c) will increase coverage by 0.13%. The diff coverage is 84.11%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##             main     #268      +/-   ##
==========================================
+ Coverage   65.86%   65.99%   +0.13%     
==========================================
  Files          56       60       +4     
  Lines        7613     7679      +66     
  Branches     2012     2022      +10     
==========================================
+ Hits         5014     5068      +54     
- Misses       2009     2017       +8     
- Partials      590      594       +4
Impacted Files Coverage Δ
linkml_runtime/utils/context_utils.py 68.33% <ø> (ø)
linkml_runtime/dumpers/rdf_dumper.py 65.90% <55.55%> (-5.89%) :arrow_down:
linkml_runtime/loaders/loader_root.py 63.63% <60.00%> (-3.04%) :arrow_down:
linkml_runtime/loaders/delimited_file_loader.py 70.58% <70.58%> (ø)
linkml_runtime/dumpers/json_dumper.py 77.77% <84.61%> (-1.54%) :arrow_down:
linkml_runtime/utils/schemaview.py 87.79% <88.23%> (ø)
linkml_runtime/dumpers/delimited_file_dumper.py 89.65% <89.65%> (ø)
linkml_runtime/dumpers/__init__.py 100.00% <100.00%> (ø)
linkml_runtime/dumpers/csv_dumper.py 100.00% <100.00%> (+8.69%) :arrow_up:
linkml_runtime/dumpers/dumper_root.py 100.00% <100.00%> (ø)
... and 11 more