search

globalgov / manydata

The portal for global governance data
https://manydata.ch
GNU Affero General Public License v3.0
9 stars 0 forks source link

Set a text argument in consolidate() #227

Closed henriquesposito closed 1 year ago

henriquesposito commented 2 years ago

Now that we will have texts in HUGGO dataset, we should perhaps add an extra text = FALSE/TRUE argument to function so that users can decide if text variables should be used in consolidation. This should be false by default too make the process faster.

henriquesposito commented 2 years ago

Since function is complex (contain 5 arguments) and lintr complains about it already (cyclomatic complexity of 23), perhaps a better alternative is to either simply remove text observations (currently implemented) or to resolve them differently (separate, get first non-missing, and re-merge with output data in the end).

henriquesposito commented 1 year ago

I removed text variables altogether from ´consolidate()´. This, first, caused issues for resolving and for coalescing compatible rows. Besides, working with text variables would slow down the function greatly and not be a wise use of memory (might be causing memory issues mentioned in issue #230).

Now what I propose is create a function to re-join texts to consolidated data if users want to. I will open a new issue for this.