CatalogueOfLife / data

Repository for COL content
7 stars 2 forks source link

Conversion report 2019-12-10 #93

Closed yroskov closed 4 years ago

yroskov commented 4 years ago

Test of 3 datasets (ICTV, Brachiopoda Database and FishBase-Elasmobranchii) at http://dev3.species.id:9191/col_plus/browse/tree?de7e51d53e54690914e3b49a4f1755dd

yroskov commented 4 years ago

@mdoering, @gdower My results: all data (incl. references with acc, syns & common names, sets of common names, LTS date) look fine. Virus presentation is also fine. The only remaining problem is encoding in entire database https://github.com/Sp2000/colplus-backend/issues/577. Example from com. names: ?ralok tmavonos? | Zralok tmavonosy | Czech | Czech Republic We need to adjust metadata before release. At the moment, GSD Citation looks like that: Froese R. & Pauly D. (eds). (2019). FishBase (version Feb 2018). In: Draft Catalogue of Life, 2019-12-09 (). Digital resource at www.catalogueoflife.org/col. Species 2000: Naturalis, Leiden, the Netherlands. ISSN 2405-8858.

mdoering commented 4 years ago

Good, these 2 are expected but on my list next. For the metadata we need to think how to best change that - as mentioned in our last call. Either on release time, pre release in the draft and then revert, or post release in the release dataset.

gdower commented 4 years ago

Here's output from the col-diff-tool for some names in the class Elasmobranchii:

http://ower.org/col_diff/2019-12-10/

gdower commented 4 years ago

@mdoering, that issue with the references that you pointed out yesterday might not be that big of a deal. It's for GSD 500 (management classification) and my best guess is that maybe it's exporting empty references for the species estimates? We don't actually have a 1500 dataset.

record_id,author,year,title,source,database_id,reference_code
1,\N,\N,\N,\N,500,1
2,\N,\N,\N,\N,500,2
3,\N,\N,\N,\N,500,3
4,\N,\N,\N,\N,500,4
5,\N,\N,\N,\N,500,5
6,\N,\N,\N,\N,500,6
7,\N,\N,\N,\N,500,7
8,\N,\N,\N,\N,500,8
9,\N,\N,\N,\N,500,9
10,\N,\N,\N,\N,500,10
11,\N,\N,\N,\N,500,11
12,\N,\N,\N,\N,500,12
13,\N,\N,\N,\N,500,13
14,\N,\N,\N,\N,500,14
15,\N,\N,\N,\N,500,15
16,\N,\N,\N,\N,500,16
17,\N,\N,\N,\N,500,17
18,\N,\N,\N,\N,500,18
19,\N,\N,\N,\N,500,19
20,\N,\N,\N,\N,500,20
21,\N,\N,\N,\N,500,21
22,\N,\N,\N,\N,500,22
23,\N,\N,\N,\N,500,23
24,\N,\N,\N,\N,500,24
25,\N,\N,\N,\N,500,25
26,\N,\N,\N,\N,500,26
27,\N,\N,\N,\N,500,27
28,\N,\N,\N,\N,500,28
29,\N,\N,\N,\N,500,29
30,\N,\N,\N,\N,500,30
31,\N,\N,\N,\N,500,31
32,\N,\N,\N,\N,500,32
33,\N,\N,\N,\N,500,33
34,\N,\N,\N,\N,500,34
35,\N,\N,\N,\N,500,35
36,\N,\N,\N,\N,500,36
37,\N,\N,\N,\N,500,37
38,\N,\N,\N,\N,500,38
39,\N,\N,\N,\N,500,39
40,\N,\N,\N,\N,500,40
41,\N,\N,\N,\N,500,41
42,\N,\N,\N,\N,500,42
43,\N,\N,\N,\N,500,43
44,\N,\N,\N,\N,500,44
45,\N,\N,\N,\N,500,45
46,\N,\N,\N,\N,500,46
47,\N,\N,\N,\N,500,47
48,\N,\N,\N,\N,500,48
49,\N,\N,\N,\N,500,49
50,\N,\N,\N,\N,500,50
51,\N,\N,\N,\N,500,51
52,\N,\N,\N,\N,500,52
53,\N,\N,\N,\N,500,53
54,\N,\N,\N,\N,500,54
55,\N,\N,\N,\N,500,55
56,\N,\N,\N,\N,500,56
57,\N,\N,\N,\N,500,57
58,\N,\N,\N,\N,500,58
59,\N,\N,\N,\N,500,59
60,\N,\N,\N,\N,500,60
61,\N,\N,\N,\N,500,61
62,\N,\N,\N,\N,500,62
63,\N,\N,\N,\N,500,63
64,\N,\N,\N,\N,500,64
65,\N,\N,\N,\N,500,65
66,\N,\N,\N,\N,500,66
67,\N,\N,\N,\N,500,67
68,\N,\N,\N,\N,500,68
69,\N,\N,\N,\N,500,69
70,\N,\N,\N,\N,500,70
71,\N,\N,\N,\N,500,71
72,\N,\N,\N,\N,500,72
73,\N,\N,\N,\N,500,73
74,\N,\N,\N,\N,500,74
mdoering commented 4 years ago

Probably. These are unparsed references like this one: http://api.catalogue.life/dataset/3/reference/1

Estimate refs are exported as full citations, so these here probably do not matter and will not be used anywhere

mdoering commented 4 years ago

@gdower @yroskov pointing to the actual export file used for the conversion would be helpful in the future. I guess it is this one here: http://api.dev.catalogue.life/download/export-30805.zip

yroskov commented 4 years ago

All references linked to estimate figures (like that one http://api.catalogue.life/dataset/3/reference/1) are unparsed.