some occurrences not being shown in the dataset

[gbif-portal]

some occurrences not being shown in the dataset

Hello, I just published a dataset with several events associated with occurrences (https://www.gbif.org/dataset/707036df-5053-40d0-a636-92f78c04cf90). However, some occurrences, specifically those belonging to the camera trap records, are not displayed. Can you help me figure out what is wrong? Perhaps it is related to the metadata term parentEventID in the occurrences? Thanks a lot for your help.

---

User: See in registry - Send email System: Safari 17.5.0 / Mac OS X 10.15.7 Referer: https://www.gbif.org/dataset/707036df-5053-40d0-a636-92f78c04cf90 Window size: width 1440 - height 724 API log&_a=(columns:!(_source),filters:!(),index:'3390a910-fcda-11ea-a9ab-4375f2a9d11c',interval:auto,query:(language:kuery,query:''),sort:!())) Site log&_a=(columns:!(_source),filters:!(),index:'5c73f360-fce3-11ea-a9ab-4375f2a9d11c',interval:auto,query:(language:kuery,query:''),sort:!())) System health at time of feedback: INFO

[Mesibov]

Another issue is that your footprintWKT entries are invalid. For example in https://www.gbif.org/occurrence/4879911587

POLYGON ((-53,79832 -34,20196, -53,79646 -34,20264, -53,79804 -34,20291, -53,80114 -34,20279, -53,79977 -34,20202, -53,79832 -34,20196))

is invalid because you have used "," as a decimal separator, and "," is the vertex separator in a WKT polygon. Change this to

POLYGON ((-53.79832 -34.20196, -53.79646 -34.20264, -53.79804 -34.20291, -53.80114 -34.20279, -53.79977 -34.20202, -53.79832 -34.20196))

and the WKT will be valid.

(P.S. I am not associated with GBIF)

[ManonGros]

It looks like many occurrences refer an eventID that isn't included in the event file.

┌─eventID─────────────────────┐
│ lc_001_2019_trailcam_44_801 │
│ lc_001_2019_trailcam_44_802 │
│ lc_002_2019_trailcam_45_1   │
│ lc_002_2019_trailcam_45_2   │
│ lc_002_2019_trailcam_45_3   │
│ lc_002_2019_trailcam_45_4   │
│ lc_002_2019_trailcam_45_5   │
│ lc_002_2019_trailcam_45_6   │
│ lc_002_2019_trailcam_45_7   │
│ lc_002_2019_trailcam_45_8   │
│ lc_002_2019_trailcam_45_9   │
│ lc_002_2019_trailcam_45_10  │
│ lc_002_2019_trailcam_45_11  │
│ lc_002_2019_trailcam_45_12  │
│ lc_002_2019_trailcam_45_13  │
│ lc_002_2019_trailcam_45_14  │
│ lc_002_2019_trailcam_45_15  │
│ lc_002_2019_trailcam_45_16  │
│ lc_002_2019_trailcam_45_17  │
│ lc_002_2019_trailcam_45_18  │
│ lc_002_2019_trailcam_45_19  │
│ lc_002_2019_trailcam_45_20  │
│ lc_002_2019_trailcam_45_21  │
│ lc_002_2019_trailcam_45_22  │
│ lc_002_2019_trailcam_45_23  │
│ lc_002_2019_trailcam_45_24  │
│ lc_002_2019_trailcam_45_25  │
│ lc_002_2019_trailcam_45_26  │
│ lc_002_2019_trailcam_45_27  │
│ lc_002_2019_trailcam_45_28  │
│ lc_002_2019_trailcam_45_29  │
│ lc_002_2019_trailcam_45_30  │
│ lc_002_2019_trailcam_45_31  │
│ lc_002_2019_trailcam_45_32  │
│ lc_002_2019_trailcam_45_33  │
│ lc_002_2019_trailcam_45_34  │
│ lc_002_2019_trailcam_45_35  │
│ lc_002_2019_trailcam_45_36  │
│ lc_002_2019_trailcam_45_37  │
│ lc_002_2019_trailcam_45_38  │
│ lc_002_2019_trailcam_45_39  │
│ lc_002_2019_trailcam_45_40  │
│ lc_002_2019_trailcam_45_41  │
│ lc_002_2019_trailcam_45_42  │
│ lc_002_2019_trailcam_45_43  │
│ lc_002_2019_trailcam_45_44  │
│ lc_002_2019_trailcam_45_45  │
│ lc_002_2019_trailcam_45_46  │
│ lc_002_2019_trailcam_45_47  │
│ lc_002_2019_trailcam_45_48  │
│ lc_003_2019_trailcam_46_1   │
│ lc_003_2019_trailcam_46_2   │
│ lc_003_2019_trailcam_46_3   │
│ lc_003_2019_trailcam_46_4   │
│ lc_003_2019_trailcam_46_5   │
│ lc_003_2019_trailcam_46_6   │
│ lc_003_2019_trailcam_46_7   │
│ lc_003_2019_trailcam_46_8   │
│ lc_003_2019_trailcam_46_9   │
│ lc_003_2019_trailcam_46_10  │
│ lc_003_2019_trailcam_46_11  │
│ lc_003_2019_trailcam_46_12  │
│ lc_003_2019_trailcam_46_13  │
│ lc_003_2019_trailcam_46_14  │
│ lc_003_2019_trailcam_46_15  │
│ lc_003_2019_trailcam_46_16  │
│ lc_003_2019_trailcam_46_17  │
│ lc_003_2019_trailcam_46_18  │
│ lc_003_2019_trailcam_46_19  │
│ lc_003_2019_trailcam_46_20  │
│ lc_003_2019_trailcam_46_21  │
│ lc_003_2019_trailcam_46_22  │
│ lc_003_2019_trailcam_46_23  │
│ lc_003_2019_trailcam_46_24  │
│ lc_003_2019_trailcam_46_25  │
│ lc_003_2019_trailcam_46_26  │
│ lc_003_2019_trailcam_46_27  │
│ lc_003_2019_trailcam_46_28  │
│ lc_003_2019_trailcam_46_29  │
│ lc_003_2019_trailcam_46_30  │
│ lc_003_2019_trailcam_46_31  │
│ lc_003_2019_trailcam_46_32  │
│ lc_003_2019_trailcam_46_33  │
│ lc_003_2019_trailcam_46_34  │
│ lc_003_2019_trailcam_46_35  │
│ lc_003_2019_trailcam_46_36  │
│ lc_003_2019_trailcam_46_37  │
│ lc_003_2019_trailcam_46_38  │
│ lc_003_2019_trailcam_46_39  │
│ lc_003_2019_trailcam_46_40  │
│ lc_003_2019_trailcam_46_41  │
│ lc_003_2019_trailcam_46_42  │
│ lc_003_2019_trailcam_46_43  │
│ lc_003_2019_trailcam_46_44  │
│ lc_003_2019_trailcam_46_45  │
│ lc_003_2019_trailcam_46_46  │
│ lc_003_2019_trailcam_46_47  │
│ lc_003_2019_trailcam_46_48  │
│ lc_003_2019_trailcam_46_49  │
│ lc_003_2019_trailcam_46_50  │
│ lc_004_2019_trailcam_47_1   │
│ lc_004_2019_trailcam_47_2   │
│ lc_004_2019_trailcam_47_3   │
│ lc_004_2019_trailcam_47_4   │
│ lc_004_2019_trailcam_47_5   │
│ lc_004_2019_trailcam_47_6   │
│ lc_004_2019_trailcam_47_7   │
│ lc_004_2019_trailcam_47_8   │
│ lc_004_2019_trailcam_47_9   │
│ lc_004_2019_trailcam_47_10  │
│ lc_005_2019_trailcam_48_1   │
│ lc_005_2019_trailcam_48_2   │
│ lc_005_2019_trailcam_48_3   │
│ lc_006_2019_trailcam_49_1   │
│ lc_006_2019_trailcam_49_2   │
│ lc_006_2019_trailcam_49_3   │
│ lc_006_2019_trailcam_49_4   │
│ lc_006_2019_trailcam_49_5   │
│ lc_006_2019_trailcam_49_6   │
│ lc_006_2019_trailcam_49_7   │
│ lc_006_2019_trailcam_49_8   │
│ lc_007_2019_trailcam_50_1   │
│ lc_007_2019_trailcam_50_2   │
│ lc_007_2019_trailcam_50_3   │
│ lc_008_2019_trailcam_51_1   │
│ lc_008_2019_trailcam_51_2   │
│ lc_008_2019_trailcam_51_3   │
│ lc_008_2019_trailcam_51_4   │
│ lc_008_2019_trailcam_51_5   │
│ lc_009_2019_trailcam_52_1   │
│ lc_009_2019_trailcam_52_2   │
│ lc_009_2019_trailcam_52_3   │
│ lc_010_2019_trailcam_53_1   │
│ lc_010_2019_trailcam_53_2   │
│ lc_010_2019_trailcam_53_3   │
│ lc_010_2019_trailcam_53_4   │
│ lc_010_2019_trailcam_53_5   │
│ lc_010_2019_trailcam_53_6   │
│ lc_011_2019_trailcam_54_1   │
│ lc_011_2019_trailcam_54_2   │
│ lc_011_2019_trailcam_54_3   │
│ lc_011_2019_trailcam_54_4   │
│ lc_011_2019_trailcam_54_5   │
│ lc_011_2019_trailcam_54_6   │
│ lc_011_2019_trailcam_54_7   │
│ lc_012_2019_trailcam_55_1   │
│ lc_012_2019_trailcam_55_2   │
│ lc_012_2019_trailcam_55_3   │
│ lc_012_2019_trailcam_55_4   │
│ lc_012_2019_trailcam_55_5   │
│ lc_012_2019_trailcam_55_6   │
│ lc_012_2019_trailcam_55_7   │
│ lc_013_2019_trailcam_56_1   │
│ lc_013_2019_trailcam_56_2   │
│ lc_013_2019_trailcam_56_3   │
│ lc_013_2019_trailcam_56_4   │
│ lc_013_2019_trailcam_56_5   │
│ lc_013_2019_trailcam_56_6   │
│ lc_013_2019_trailcam_56_7   │
│ lc_013_2019_trailcam_56_8   │
│ lc_013_2019_trailcam_56_9   │
│ lc_013_2019_trailcam_56_10  │
│ lc_013_2019_trailcam_56_11  │
│ lc_013_2019_trailcam_56_12  │
│ lc_013_2019_trailcam_56_13  │
│ lc_013_2019_trailcam_56_14  │
│ lc_013_2019_trailcam_56_15  │
│ lc_013_2019_trailcam_56_16  │
│ lc_013_2019_trailcam_56_17  │
│ lc_013_2019_trailcam_56_18  │
│ lc_014_2019_trailcam_57_1   │
│ lc_014_2019_trailcam_57_2   │
│ lc_015_2019_trailcam_58_1   │
│ lc_015_2019_trailcam_58_2   │
│ lc_015_2019_trailcam_58_3   │
│ lc_015_2019_trailcam_58_4   │
│ lc_015_2019_trailcam_58_5   │
│ lc_015_2019_trailcam_58_6   │
│ lc_015_2019_trailcam_58_7   │
│ lc_015_2019_trailcam_58_8   │
│ lc_015_2019_trailcam_58_9   │
│ lc_015_2019_trailcam_58_10  │
│ lc_015_2019_trailcam_58_11  │
│ lc_015_2019_trailcam_58_12  │
│ lc_015_2019_trailcam_58_13  │
│ lc_015_2019_trailcam_58_14  │
│ lc_015_2019_trailcam_58_15  │
│ lc_015_2019_trailcam_58_16  │
│ lc_015_2019_trailcam_58_17  │
│ lc_015_2019_trailcam_58_18  │
│ lc_015_2019_trailcam_58_19  │
│ lc_015_2019_trailcam_58_20  │
│ lc_015_2019_trailcam_58_21  │
│ lc_016_2019_trailcam_59_1   │
│ lc_016_2019_trailcam_59_2   │
│ lc_016_2019_trailcam_59_3   │
│ lc_016_2019_trailcam_59_4   │
│ lc_016_2019_trailcam_59_5   │
│ lc_016_2019_trailcam_59_6   │
│ lc_016_2019_trailcam_59_7   │
│ lc_016_2019_trailcam_59_8   │
│ lc_016_2019_trailcam_59_9   │
│ lc_017_2019_trailcam_60_1   │
│ lc_017_2019_trailcam_60_2   │
│ lc_017_2019_trailcam_60_3   │
│ lc_017_2019_trailcam_60_4   │
│ lc_017_2019_trailcam_60_5   │
│ lc_017_2019_trailcam_60_6   │
│ lc_017_2019_trailcam_60_7   │
│ lc_017_2019_trailcam_60_8   │
│ lc_017_2019_trailcam_60_9   │
│ lc_017_2019_trailcam_60_10  │
│ lc_017_2019_trailcam_60_11  │
│ lc_017_2019_trailcam_60_12  │
│ lc_017_2019_trailcam_60_13  │
│ lc_017_2019_trailcam_60_14  │
│ lc_017_2019_trailcam_60_15  │
│ lc_017_2019_trailcam_60_16  │
│ lc_017_2019_trailcam_60_17  │
│ lc_017_2019_trailcam_60_18  │
│ lc_017_2019_trailcam_60_19  │
│ lc_017_2019_trailcam_60_20  │
│ lc_018_2019_trailcam_61_1   │
│ lc_018_2019_trailcam_61_2   │
│ lc_018_2019_trailcam_61_3   │
│ lc_018_2019_trailcam_61_4   │
│ lc_018_2019_trailcam_61_5   │
│ lc_018_2019_trailcam_61_6   │
│ lc_018_2019_trailcam_61_7   │
│ lc_018_2019_trailcam_61_8   │
│ lc_018_2019_trailcam_61_9   │
│ lc_018_2019_trailcam_61_10  │
│ lc_018_2019_trailcam_61_11  │
│ lc_018_2019_trailcam_61_12  │
│ lc_018_2019_trailcam_61_13  │
│ lc_018_2019_trailcam_61_14  │
│ lc_018_2019_trailcam_61_15  │
│ lc_019_2019_trailcam_62_1   │
│ lc_019_2019_trailcam_62_2   │
│ lc_019_2019_trailcam_62_3   │
│ lc_019_2019_trailcam_62_4   │
│ lc_019_2019_trailcam_62_5   │
│ lc_019_2019_trailcam_62_6   │
│ lc_019_2019_trailcam_62_7   │
│ lc_019_2019_trailcam_62_8   │
│ lc_019_2019_trailcam_62_9   │
│ lc_019_2019_trailcam_62_10  │
│ lc_019_2019_trailcam_62_11  │
│ lc_019_2019_trailcam_62_12  │
│ lc_019_2019_trailcam_62_13  │
│ lc_019_2019_trailcam_62_14  │
│ lc_019_2019_trailcam_62_15  │
│ lc_019_2019_trailcam_62_16  │
│ lc_020_2019_trailcam_63_1   │
│ lc_020_2019_trailcam_63_2   │
│ lc_020_2019_trailcam_63_3   │
│ lc_020_2019_trailcam_63_4   │
│ lc_020_2019_trailcam_63_5   │
│ lc_020_2019_trailcam_63_6   │
│ lc_020_2019_trailcam_63_7   │
│ lc_020_2019_trailcam_63_8   │
│ lc_020_2019_trailcam_63_9   │
│ lc_020_2019_trailcam_63_10  │
│ lc_020_2019_trailcam_63_11  │
│ lc_020_2019_trailcam_63_12  │
│ lc_021_2019_trailcam_64_1   │
│ lc_021_2019_trailcam_64_2   │
│ lc_021_2019_trailcam_64_3   │
│ lc_021_2019_trailcam_64_4   │
│ lc_022_2019_trailcam_65_1   │
│ lc_022_2019_trailcam_65_2   │
│ lc_022_2019_trailcam_65_3   │
│ lc_022_2019_trailcam_65_4   │
│ lc_022_2019_trailcam_65_5   │
│ lc_022_2019_trailcam_65_6   │
│ lc_022_2019_trailcam_65_7   │
│ lc_022_2019_trailcam_65_8   │
│ lc_022_2019_trailcam_65_9   │
│ lc_022_2019_trailcam_65_10  │
│ lc_022_2019_trailcam_65_11  │
│ lc_022_2019_trailcam_65_12  │
│ lc_022_2019_trailcam_65_13  │
│ lc_022_2019_trailcam_65_14  │
│ lc_022_2019_trailcam_65_15  │
│ lc_022_2019_trailcam_65_16  │
│ lc_022_2019_trailcam_65_17  │
│ lc_022_2019_trailcam_65_18  │
│ lc_022_2019_trailcam_65_19  │
│ lc_023_2019_trailcam_66_1   │
│ lc_023_2019_trailcam_66_2   │
│ lc_023_2019_trailcam_66_3   │
│ lc_024_2019_trailcam_67_1   │
│ lc_024_2019_trailcam_67_2   │
│ lc_024_2019_trailcam_67_3   │
│ lc_024_2019_trailcam_67_4   │
│ lc_024_2019_trailcam_67_5   │
│ lc_025_2019_trailcam_68_1   │
│ lc_025_2019_trailcam_68_2   │
│ lc_025_2019_trailcam_68_3   │
│ lc_025_2019_trailcam_68_4   │
│ lc_025_2019_trailcam_68_5   │
│ lc_025_2019_trailcam_68_6   │
│ lc_026_2019_trailcam_69_1   │
│ lc_026_2019_trailcam_69_2   │
│ lc_026_2019_trailcam_69_3   │
│ lc_026_2019_trailcam_69_4   │
│ lc_026_2019_trailcam_69_5   │
│ lc_026_2019_trailcam_69_6   │
│ lc_026_2019_trailcam_69_7   │
│ lc_026_2019_trailcam_69_8   │
│ lc_026_2019_trailcam_69_9   │
│ lc_026_2019_trailcam_69_10  │
│ lc_026_2019_trailcam_69_11  │
│ lc_026_2019_trailcam_69_12  │
│ lc_026_2019_trailcam_69_13  │
│ lc_026_2019_trailcam_69_14  │
│ lc_026_2019_trailcam_69_15  │
│ lc_026_2019_trailcam_69_16  │
│ lc_026_2019_trailcam_69_17  │
│ lc_026_2019_trailcam_69_18  │
│ lc_026_2019_trailcam_69_19  │
│ lc_026_2019_trailcam_69_20  │
│ lc_027_2019_trailcam_70_1   │
│ lc_027_2019_trailcam_70_2   │
│ lc_027_2019_trailcam_70_3   │
│ lc_027_2019_trailcam_70_4   │
│ lc_027_2019_trailcam_70_5   │
│ lc_027_2019_trailcam_70_6   │
│ lc_027_2019_trailcam_70_7   │
│ lc_027_2019_trailcam_70_8   │
│ lc_027_2019_trailcam_70_9   │
│ lc_027_2019_trailcam_70_10  │
│ lc_027_2019_trailcam_70_11  │
│ lc_028_2019_trailcam_71_1   │
│ lc_028_2019_trailcam_71_2   │
│ lc_028_2019_trailcam_71_3   │
│ lc_028_2019_trailcam_71_4   │
│ lc_028_2019_trailcam_71_5   │
│ lc_029_2019_trailcam_72_1   │
│ lc_029_2019_trailcam_72_2   │
│ lc_029_2019_trailcam_72_3   │
│ lc_029_2019_trailcam_72_4   │
│ lc_029_2019_trailcam_72_5   │
│ lc_029_2019_trailcam_72_6   │
│ lc_029_2019_trailcam_72_7   │
│ lc_029_2019_trailcam_72_8   │
│ lc_029_2019_trailcam_72_9   │
│ lc_029_2019_trailcam_72_10  │
│ lc_030_2019_trailcam_73_1   │
│ lc_030_2019_trailcam_73_2   │
│ lc_030_2019_trailcam_73_3   │
│ lc_030_2019_trailcam_73_4   │
│ lc_030_2019_trailcam_73_5   │
│ lc_030_2019_trailcam_73_6   │
│ lc_030_2019_trailcam_73_7   │
│ lc_030_2019_trailcam_73_8   │
│ lc_030_2019_trailcam_73_9   │
│ lc_030_2019_trailcam_73_10  │
│ lc_030_2019_trailcam_73_11  │
│ lc_030_2019_trailcam_73_12  │
│ lc_030_2019_trailcam_73_13  │
│ lc_030_2019_trailcam_73_14  │
│ lc_030_2019_trailcam_73_15  │
│ lc_030_2019_trailcam_73_16  │
│ lc_030_2019_trailcam_73_17  │
│ lc_030_2019_trailcam_73_18  │
│ lc_030_2019_trailcam_73_19  │
│ lc_030_2019_trailcam_73_20  │
│ lc_030_2019_trailcam_73_21  │
│ lc_030_2019_trailcam_73_22  │
│ lc_030_2019_trailcam_73_23  │
│ lc_030_2019_trailcam_73_24  │
│ lc_002_2019_trailcam_74_49  │
│ lc_002_2019_trailcam_74_50  │
│ lc_002_2019_trailcam_74_51  │
│ lc_002_2019_trailcam_74_52  │
│ lc_002_2019_trailcam_74_53  │
│ lc_002_2019_trailcam_74_54  │
│ lc_002_2019_trailcam_74_55  │
│ lc_002_2019_trailcam_74_56  │
│ lc_002_2019_trailcam_74_57  │
│ lc_002_2019_trailcam_74_58  │
│ lc_002_2019_trailcam_74_59  │
│ lc_002_2019_trailcam_74_60  │
│ lc_002_2019_trailcam_74_61  │
│ lc_002_2019_trailcam_74_62  │
│ lc_002_2019_trailcam_74_63  │
│ lc_002_2019_trailcam_74_64  │
└─────────────────────────────┘

I will contact the publisher. Thank you Bob for pointing out the issue with the polygons as well!

[Mesibov]

@ManonGros, yes, this issue is found with the "chkevoc" script and I guess also with the GBIF Data Validator. In 2022 we discussed here whether GBIF could notify publishers about referential integrity problems. At the time you wrote "I understand that there are plans to integrate the data validator to the IPT in the future. That would allow publishers to see this type of issue before the data makes it to GBIF." So, not yet?

[bienflorencia]

Thank you so much for detecting this @Mesibov and @ManonGros! I have corrected the files and they should now be ok. I don't see the events for the trailcams being displayed, but perhaps it needs a bit of extra time to load the changes?

Indeed, it would be great to integrate the data validator to the IPT, as it is difficult to anticipate an issue such as the footprintWKT occurring. In my code, it worked and mapped correctly. Cheers!

[Mesibov]

@bienflorencia, happy to help. It sounds like your GIS software understands "," as a decimal separator, but this is not universal. Your WKTs fail to plot in these two online WKT plotters: https://wktmap.com/ https://clydedacruz.github.io/openstreetmap-wkt-playground/