New Parser for MedAfriCarbon & Jomon Databases

[dirkseidensticker]

Created new parsers for MedAfriCarbon & Crema Jomon Japan datasets (see #2 ).

Known Issues:

• [x] MedAfriCarbon
  • get_medafricarbon.R#L35-L66: Cultural attribution and references are listed in separate tables as individual entries within the original dataset. The used for()-loops impose quite a performance bottleneck. How to do this better/faster?
• [x] Jomon
  • [x] site = .data[["Site_Name_JP"]] used, but encoding = "UTF-8" is not returning the desired output (only UTF-8 code). Checking the codes against compart.com shows that characters are partially encoded as 'Han Chinese', which seems odd for data on Japan.
  • [x] No coordinates found in data-folder of original data set. Spatial differentiation seems only based on site names and province(?).
• [x] need to resolve merge-conflicts due to new branch being based on outdated readme_overview_fig branch
• [x] Thesaurification failed, despite entering missing data in thesauri

@nevrome : Any suggestion how to improve this?

[codecov-io]

Codecov Report

Merging #95 into master will increase coverage by 0.31%. The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #95      +/-   ##
==========================================
+ Coverage   95.18%   95.49%   +0.31%     
==========================================
  Files          32       34       +2     
  Lines        1434     1533      +99     
==========================================
+ Hits         1365     1464      +99     
  Misses         69       69

Impacted Files	Coverage Δ
R/get_pacea.R	100% <ø> (ø)	:arrow_up:
R/get_medafricarbon.R	100% <100%> (ø)
R/get_c14data.R	76.19% <100%> (+0.78%)	:arrow_up:
R/get_jomon.R	100% <100%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 334c0e9...fb6362f. Read the comment docs.

[nevrome]

The Japanese characters look fine for me. At least as far as I can judge :smile:. Maybe a problem with your system? It's ok if there are no coordinates. We already have other databases without these. Maybe @ercrema wants to comment this?

[nevrome]

Concerning the thesaurification: The reference tables are always downloaded from the master branch. Ceramic Organic Residue is in here, but not in here yet. This is a constant source of confusion and I'm more and more convinced that we have to find a better solution for this.

[dirkseidensticker]

Concerning the thesaurification: The reference tables are always downloaded from the master branch. Ceramic Organic Residue is in here, but not in here yet. This is a constant source of confusion and I'm more and more convinced that we have to find a better solution for this.

Arg, damn ... you are so right ... okay

But ... please wait a second: please take a look at #96 ... there it seems the country_thesaurus is not respected either

[dirkseidensticker]

The Japanese characters look fine for me. At least as far as I can judge 😄. Maybe a problem with your system? It's ok if there are no coordinates. We already have other databases without these. Maybe @ercrema wants to comment this?

Okay, actually same here, if I print into the console. But I always checked using View() and that only returns UTF-8 codes. But so this isn't an issue on our side 👍

[dirkseidensticker]

thanks for b282e40 ... I need to up my dplyr-game for sure

[ercrema]

The Japanese look fine - coordinates are a bit of nightmare for various reasons. We are working on a larger dataset which will have some coordinates but this is still work in progress. Thanks for adding both datasets on your package btw!

[dirkseidensticker]

a62bb76 adds variables missed in the initial parser and lists the unused variable in the reference list.

@nevrome please test if still functional, after that I can resolve merge conflicts & merge

[nevrome]

Great! Works fine. Go ahead!

[dirkseidensticker]

The Japanese look fine - coordinates are a bit of nightmare for various reasons. We are working on a larger dataset which will have some coordinates but this is still work in progress. Thanks for adding both datasets on your package btw!

@ercrema many thanks for your explanations. Looking forward to future data ;)