search

OpenConceptLab / ocl_issues

Issues for all OCL repos. NOTE: Install ZenHub Browser Extension and request access to the OCL Roadmap board to view all issues and to contribute
4 stars 1 forks source link

OCL CSV exports truncated at 1000 #1911

Open chamiw2008 opened 1 month ago

chamiw2008 commented 1 month ago

Issue tracker is ONLY used for reporting bugs. New features should be discussed with us on OCL Chat or on one our slack channels.

When used to export the codes as CSV the list is truncated at 1000 There are 55066 codes https://api.openconceptlab.org/orgs/CIEL/sources/CIEL/ But, CSV has only 1000

Export also SHOULD have the CONCEPT NAME

paynejd commented 1 month ago

Thanks for reporting this @chamiw2008. Truncating CSV downloads at 1k was by design, because CSV downloads are generated on-the-fly and the same method is used for downloading content from a repo as from a search result that has no limit in the number of results, so we needed to set some arbitrary limit. This made the CSV download not very useful and poorly maintained, and at this point the data in the CSV download doesn't offer much value at all.

However, implementing a better CSV download has long been in the backlog, and it would be great to move this forward.

First step is to agree on the columns/format. The CSV download is NOT intended to be a full representation of the content, it is intended to be a useful summary of the content.

@chamiw2008 Would you be willing to add in your requirements for fields in a CSV download? I included OCL's current CSV download and the OpenMRS download as examples below.

CSV examples:

chamiw2008 commented 1 month ago

@paynejd Thanks for the update.

My requirement was to generate couple of code sets that can be used to capture a tentative diagnosis in OPD setup. So, the display name (Name/FSN) is required. Then when sharing with fhir the code and code system is needed.

Many code systems would have codes for findings, diagnosis, investigations, etc. But collecting apples oranges and bananas in one basket is not very useful. So I needed to filter by the OCL Class attribute.

For my exercise I had to find diagnosis codes from SNOMED-GPS, CIEL and ICPC-2.

If the OCL had a better working download option, two of the systems could have been retrieved from OCL.

Hope this explained my requirement better.

image