search

openstate / open-raadsinformatie

Doorzoek vergaderstukken van gemeenten en provincies
https://openbesluitvorming.nl
MIT License
28 stars 12 forks source link

V0 vs V1 data quantity comparison #146

Closed joepio closed 4 years ago

joepio commented 5 years ago

When we make the V1 API the default one and use ori-search on zoek.openraadsinformatie.nl, we want to make sure that the data is at least as complete as in V0. In order to prove this, we need to provide some statistics about how many items are loaded per month, per type and per municipality.

jurrian commented 5 years ago

Here are some results of the Agenda/Meetings for a specified month, aggregated per index:

V0 event Agenda's oct 2018 ``` V0 { "took": 19, "timed_out": false, "_shards": { "total": 134, "successful": 134, "failed": 0 }, "hits": { "total": 1095, "max_score": 0.0, "hits": [] }, "aggregations": { "maanden": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [{ "key": "ori_maastricht_20180122161222", "doc_count": 86 }, { "key": "ori_zaanstad_20180122161607", "doc_count": 42 }, { "key": "ori_zwolle_20180125145125", "doc_count": 36 }, { "key": "ori_amersfoort_20180122161238", "doc_count": 32 }, { "key": "ori_voorst_20180122161636", "doc_count": 30 }, { "key": "ori_arnhem_20180124164533", "doc_count": 28 }, { "key": "ori_deventer_20180122161434", "doc_count": 25 }, { "key": "ori_enschede_20170309152431_migrated", "doc_count": 25 }, { "key": "ori_groningen_20180122161714", "doc_count": 24 }, { "key": "ori_zeist_20180122161742", "doc_count": 24 }, { "key": "ori_almere_20180122161542", "doc_count": 21 }, { "key": "ori_den_helder_20151221133407_new_migrated", "doc_count": 21 }, { "key": "ori_amsterdam_20180122161245", "doc_count": 19 }, { "key": "ori_hilvarenbeek_20180122161214", "doc_count": 18 }, { "key": "ori_raalte_20180123103236", "doc_count": 17 }, { "key": "ori_denhaag_20180122161458", "doc_count": 16 }, { "key": "ori_rhenen_20180122161649", "doc_count": 16 }, { "key": "ori_wageningen_20180122161620", "doc_count": 16 }, { "key": "ori_utrecht_20151221135818_new_migrated", "doc_count": 15 }, { "key": "ori_limburg_20180605132257", "doc_count": 14 }, { "key": "ori_edamvolendam_20180125150152", "doc_count": 13 }, { "key": "ori_flevoland_20180618120622", "doc_count": 13 }, { "key": "ori_zuid-holland_20181018132040", "doc_count": 13 }, { "key": "ori_almelo_20180123103246", "doc_count": 12 }, { "key": "ori_goirle_20170203151744_migrated", "doc_count": 12 }, { "key": "ori_medemblik_20170310150856_migrated", "doc_count": 12 }, { "key": "ori_oosterhout_20180123095212", "doc_count": 12 }, { "key": "ori_hollandskroon_20180125122446", "doc_count": 11 }, { "key": "ori_leiden_20180122161529", "doc_count": 11 }, { "key": "ori_leusden_20180122160946", "doc_count": 11 }, { "key": "ori_nieuwegein_20180124165712", "doc_count": 11 }, { "key": "ori_roermond_20180122161449", "doc_count": 11 }, { "key": "ori_zoetermeer_20170313132023_migrated", "doc_count": 11 }, { "key": "ori_maassluis_20180122161736", "doc_count": 10 }, { "key": "ori_zandvoort_20180122161629", "doc_count": 10 }, { "key": "ori_culemborg_20180122161617", "doc_count": 9 }, { "key": "ori_hilversum_20180123100207", "doc_count": 9 }, { "key": "ori_krimpenerwaard_20180124163621", "doc_count": 9 }, { "key": "ori_meierijstad_20180125140121", "doc_count": 9 }, { "key": "ori_rheden_20180627130051", "doc_count": 9 }, { "key": "ori_ede_20180519010322", "doc_count": 8 }, { "key": "ori_gouda_20170310143543_migrated", "doc_count": 8 }, { "key": "ori_hulst_20180122161510", "doc_count": 8 }, { "key": "ori_katwijk_20180122161656", "doc_count": 8 }, { "key": "ori_lingewaard_20180125140752", "doc_count": 8 }, { "key": "ori_oude_ijsselstreek_20151221134613_new_migrated", "doc_count": 8 }, { "key": "ori_provincie-utrecht_20180605132843", "doc_count": 8 }, { "key": "ori_waalwijk_20180122161153", "doc_count": 8 }, { "key": "ori_amsterdam_oost_20180122161601", "doc_count": 7 }, { "key": "ori_cappelleadijssel_20180122161444", "doc_count": 7 }, { "key": "ori_krimpenadijssel_20180122161548", "doc_count": 7 }, { "key": "ori_noord-holland_20180605132038", "doc_count": 7 }, { "key": "ori_amsterdam_noord_20180122161732", "doc_count": 6 }, { "key": "ori_beemster_20180122161041", "doc_count": 6 }, { "key": "ori_borne_20180122161105", "doc_count": 6 }, { "key": "ori_buren_20180123103402", "doc_count": 6 }, { "key": "ori_doetinchem_20170127130353_migrated", "doc_count": 6 }, { "key": "ori_ettenleur_20180123103555", "doc_count": 6 }, { "key": "ori_nieuwkoop_20180123103222", "doc_count": 6 }, { "key": "ori_roosendaal_20180122161053", "doc_count": 6 }, { "key": "ori_schagen_20180122161801", "doc_count": 6 }, { "key": "ori_soest_20180122161123", "doc_count": 6 }, { "key": "ori_teylingen_20180123114737", "doc_count": 6 }, { "key": "ori_wassenaar_20180126130808", "doc_count": 6 }, { "key": "ori_haarlemmermeer_20190111135004", "doc_count": 5 }, { "key": "ori_heemskerk_20180124171613", "doc_count": 5 }, { "key": "ori_leiderdorp_20180122161024", "doc_count": 5 }, { "key": "ori_nijkerk_20180523005116", "doc_count": 5 }, { "key": "ori_noordoostpolder_20170207165214_migrated", "doc_count": 5 }, { "key": "ori_noordwijk_20180122161839", "doc_count": 5 }, { "key": "ori_oss_20170310151053_migrated", "doc_count": 5 }, { "key": "ori_rijswijk_20180123103022", "doc_count": 5 }, { "key": "ori_veenendaal_20180122161208", "doc_count": 5 }, { "key": "ori_velsen_20180125172527", "doc_count": 5 }, { "key": "ori_vlaardingen_20180122161349", "doc_count": 5 }, { "key": "ori_aalsmeer_20180125122532", "doc_count": 4 }, { "key": "ori_amstelveen_20151221132805_new_migrated", "doc_count": 4 }, { "key": "ori_baarn_20180122161147", "doc_count": 4 }, { "key": "ori_barneveld_20180123112938", "doc_count": 4 }, { "key": "ori_binnenmaas_20180122161553", "doc_count": 4 }, { "key": "ori_dongen_20180122161626", "doc_count": 4 }, { "key": "ori_drimmelen_20180606004659", "doc_count": 4 }, { "key": "ori_epe_20180122161831", "doc_count": 4 }, { "key": "ori_hardenberg_20180122161828", "doc_count": 4 }, { "key": "ori_heerde_20151221134011_new_migrated", "doc_count": 4 }, { "key": "ori_leeuwarden_20180122161139", "doc_count": 4 }, { "key": "ori_molenwaard_20180122161049", "doc_count": 4 }, { "key": "ori_oisterwijk_20180122161234", "doc_count": 4 }, { "key": "ori_oostgelre_20180122161646", "doc_count": 4 }, { "key": "ori_rucphen_20180122161558", "doc_count": 4 }, { "key": "ori_schiedam_20180122161524", "doc_count": 4 }, { "key": "ori_staphorst_20180125140519", "doc_count": 4 }, { "key": "ori_steenbergen_20170207165350_migrated", "doc_count": 4 }, { "key": "ori_vlissingen_20180122161819", "doc_count": 4 }, { "key": "ori_zwartewaterland_20180519010027", "doc_count": 4 }, { "key": "ori_alkmaar_20180209111005", "doc_count": 3 }, { "key": "ori_amsterdam_zuid_20180122161726", "doc_count": 3 }, { "key": "ori_castricum_20180122161204", "doc_count": 3 }, { "key": "ori_hendrikidoambacht_20180122161441", "doc_count": 3 }, { "key": "ori_landsmeer_20180125145530", "doc_count": 3 }, { "key": "ori_loonopzand_20180123103353", "doc_count": 3 }, { "key": "ori_moerdijk_20180122161808", "doc_count": 3 }, { "key": "ori_noordwijkerhout_20180125144832", "doc_count": 3 }, { "key": "ori_oldambt_20180125142217", "doc_count": 3 }, { "key": "ori_steenwijkerland_20180523121516", "doc_count": 3 }, { "key": "ori_texel_20180125122155", "doc_count": 3 }, { "key": "ori_amsterdam_centrum_20180122161753", "doc_count": 2 }, { "key": "ori_amsterdam_nieuwwest_20180122161143", "doc_count": 2 }, { "key": "ori_amsterdam_west_20180122161623", "doc_count": 2 }, { "key": "ori_amsterdam_zuidoost_20180122161706", "doc_count": 2 }, { "key": "ori_dewolden_20180122155825", "doc_count": 2 }, { "key": "ori_diemen_20180125150906", "doc_count": 2 }, { "key": "ori_meppel_20180125121920", "doc_count": 2 }, { "key": "ori_ommen_20180122161518", "doc_count": 2 }, { "key": "ori_voorschoten_20180127104535", "doc_count": 2 }, { "key": "ori_bodegravenreeuwijk_20180605133139", "doc_count": 1 }, { "key": "ori_boxmeer_20180523121239", "doc_count": 1 }] } } } ```
V0 event Agenda's feb 2019 ``` { "took": 517, "timed_out": false, "_shards": { "total": 135, "successful": 135, "failed": 0 }, "hits": { "total": 1540, "max_score": 0.0, "hits": [] }, "aggregations": { "maanden": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [{ "key": "ori_combined_index_new_migrated", "doc_count": 769 }, { "key": "ori_maastricht_20180122161222", "doc_count": 38 }, { "key": "ori_denhaag_20180122161458", "doc_count": 23 }, { "key": "ori_deventer_20180122161434", "doc_count": 23 }, { "key": "ori_zwolle_20180125145125", "doc_count": 22 }, { "key": "ori_arnhem_20180124164533", "doc_count": 19 }, { "key": "ori_almere_20180122161542", "doc_count": 18 }, { "key": "ori_utrecht_20151221135818_new_migrated", "doc_count": 17 }, { "key": "ori_zeist_20180122161742", "doc_count": 16 }, { "key": "ori_enschede_20170309152431_migrated", "doc_count": 15 }, { "key": "ori_raalte_20180123103236", "doc_count": 15 }, { "key": "ori_voorst_20180122161636", "doc_count": 15 }, { "key": "ori_amersfoort_20180122161238", "doc_count": 14 }, { "key": "ori_amsterdam_20180122161245", "doc_count": 14 }, { "key": "ori_zaanstad_20180122161607", "doc_count": 13 }, { "key": "ori_hollandskroon_20180125122446", "doc_count": 12 }, { "key": "ori_limburg_20180605132257", "doc_count": 12 }, { "key": "ori_zuid-holland_20181018132040", "doc_count": 12 }, { "key": "ori_hilvarenbeek_20180122161214", "doc_count": 11 }, { "key": "ori_hilversum_20180123100207", "doc_count": 11 }, { "key": "ori_nieuwegein_20180124165712", "doc_count": 11 }, { "key": "ori_goirle_20170203151744_migrated", "doc_count": 10 }, { "key": "ori_rhenen_20180122161649", "doc_count": 10 }, { "key": "ori_teylingen_20180123114737", "doc_count": 10 }, { "key": "ori_alkmaar_20180209111005", "doc_count": 9 }, { "key": "ori_almelo_20180123103246", "doc_count": 9 }, { "key": "ori_culemborg_20180122161617", "doc_count": 9 }, { "key": "ori_meierijstad_20180125140121", "doc_count": 9 }, { "key": "ori_oss_20170310151053_migrated", "doc_count": 9 }, { "key": "ori_amsterdam_noord_20180122161732", "doc_count": 8 }, { "key": "ori_amsterdam_oost_20180122161601", "doc_count": 8 }, { "key": "ori_groningen_20180122161714", "doc_count": 8 }, { "key": "ori_heemskerk_20180124171613", "doc_count": 8 }, { "key": "ori_leiden_20180122161529", "doc_count": 8 }, { "key": "ori_maassluis_20180122161736", "doc_count": 8 }, { "key": "ori_medemblik_20170310150856_migrated", "doc_count": 8 }, { "key": "ori_roermond_20180122161449", "doc_count": 8 }, { "key": "ori_edamvolendam_20180125150152", "doc_count": 7 }, { "key": "ori_krimpenerwaard_20180124163621", "doc_count": 7 }, { "key": "ori_lingewaard_20180125140752", "doc_count": 7 }, { "key": "ori_noord-holland_20180605132038", "doc_count": 7 }, { "key": "ori_waalwijk_20180122161153", "doc_count": 7 }, { "key": "ori_zandvoort_20180122161629", "doc_count": 7 }, { "key": "ori_zoetermeer_20170313132023_migrated", "doc_count": 7 }, { "key": "ori_ede_20180519010322", "doc_count": 6 }, { "key": "ori_flevoland_20180618120622", "doc_count": 6 }, { "key": "ori_katwijk_20180122161656", "doc_count": 6 }, { "key": "ori_nieuwkoop_20180123103222", "doc_count": 6 }, { "key": "ori_nijkerk_20180523005116", "doc_count": 6 }, { "key": "ori_oosterhout_20180123095212", "doc_count": 6 }, { "key": "ori_oude_ijsselstreek_20151221134613_new_migrated", "doc_count": 6 }, { "key": "ori_provincie-utrecht_20180605132843", "doc_count": 6 }, { "key": "ori_rijswijk_20180123103022", "doc_count": 6 }, { "key": "ori_roosendaal_20180122161053", "doc_count": 6 }, { "key": "ori_wageningen_20180122161620", "doc_count": 6 }, { "key": "ori_buren_20180123103402", "doc_count": 5 }, { "key": "ori_cappelleadijssel_20180122161444", "doc_count": 5 }, { "key": "ori_castricum_20180122161204", "doc_count": 5 }, { "key": "ori_den_helder_20151221133407_new_migrated", "doc_count": 5 }, { "key": "ori_epe_20180122161831", "doc_count": 5 }, { "key": "ori_gouda_20170310143543_migrated", "doc_count": 5 }, { "key": "ori_haarlemmermeer_20190111135004", "doc_count": 5 }, { "key": "ori_hendrikidoambacht_20180122161441", "doc_count": 5 }, { "key": "ori_krimpenadijssel_20180122161548", "doc_count": 5 }, { "key": "ori_leeuwarden_20180122161139", "doc_count": 5 }, { "key": "ori_leiderdorp_20180122161024", "doc_count": 5 }, { "key": "ori_noordoostpolder_20170207165214_migrated", "doc_count": 5 }, { "key": "ori_schiedam_20180122161524", "doc_count": 5 }, { "key": "ori_vlissingen_20180122161819", "doc_count": 5 }, { "key": "ori_amsterdam_zuid_20180122161726", "doc_count": 4 }, { "key": "ori_barneveld_20180123112938", "doc_count": 4 }, { "key": "ori_beemster_20180122161041", "doc_count": 4 }, { "key": "ori_drimmelen_20180606004659", "doc_count": 4 }, { "key": "ori_hardenberg_20180122161828", "doc_count": 4 }, { "key": "ori_rheden_20180627130051", "doc_count": 4 }, { "key": "ori_rucphen_20180122161558", "doc_count": 4 }, { "key": "ori_velsen_20180125172527", "doc_count": 4 }, { "key": "ori_voorschoten_20180127104535", "doc_count": 4 }, { "key": "ori_wassenaar_20180126130808", "doc_count": 4 }, { "key": "ori_aalsmeer_20180125122532", "doc_count": 3 }, { "key": "ori_amstelveen_20151221132805_new_migrated", "doc_count": 3 }, { "key": "ori_amsterdam_west_20180122161623", "doc_count": 3 }, { "key": "ori_baarn_20180122161147", "doc_count": 3 }, { "key": "ori_diemen_20180125150906", "doc_count": 3 }, { "key": "ori_doetinchem_20170127130353_migrated", "doc_count": 3 }, { "key": "ori_dongen_20180122161626", "doc_count": 3 }, { "key": "ori_leusden_20180122160946", "doc_count": 3 }, { "key": "ori_meppel_20180125121920", "doc_count": 3 }, { "key": "ori_oisterwijk_20180122161234", "doc_count": 3 }, { "key": "ori_oldambt_20180125142217", "doc_count": 3 }, { "key": "ori_staphorst_20180125140519", "doc_count": 3 }, { "key": "ori_steenbergen_20170207165350_migrated", "doc_count": 3 }, { "key": "ori_vlaardingen_20180122161349", "doc_count": 3 }, { "key": "ori_zwartewaterland_20180519010027", "doc_count": 3 }, { "key": "ori_amsterdam_centrum_20180122161753", "doc_count": 2 }, { "key": "ori_amsterdam_nieuwwest_20180122161143", "doc_count": 2 }, { "key": "ori_amsterdam_zuidoost_20180122161706", "doc_count": 2 }, { "key": "ori_boxmeer_20180523121239", "doc_count": 2 }, { "key": "ori_ettenleur_20180123103555", "doc_count": 2 }, { "key": "ori_heerde_20151221134011_new_migrated", "doc_count": 2 }, { "key": "ori_landsmeer_20180125145530", "doc_count": 2 }, { "key": "ori_moerdijk_20180122161808", "doc_count": 2 }, { "key": "ori_soest_20180122161123", "doc_count": 2 }, { "key": "ori_steenwijkerland_20180523121516", "doc_count": 2 }, { "key": "ori_veenendaal_20180122161208", "doc_count": 2 }, { "key": "ori_borne_20180122161105", "doc_count": 1 }, { "key": "ori_dewolden_20180122155825", "doc_count": 1 }, { "key": "ori_hulst_20180122161510", "doc_count": 1 }, { "key": "ori_loonopzand_20180123103353", "doc_count": 1 }, { "key": "ori_molenwaard_20180122161049", "doc_count": 1 }, { "key": "ori_ommen_20180122161518", "doc_count": 1 }, { "key": "ori_oostgelre_20180122161646", "doc_count": 1 }, { "key": "ori_schagen_20180122161801", "doc_count": 1 }, { "key": "ori_texel_20180125122155", "doc_count": 1 }] } } } ```
V0 event Agenda's may 2019 ``` { "took": 349, "timed_out": false, "_shards": { "total": 135, "successful": 135, "failed": 0 }, "hits": { "total": 306, "max_score": 0.0, "hits": [] }, "aggregations": { "maanden": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [{ "key": "ori_combined_index_new_migrated", "doc_count": 155 }, { "key": "ori_arnhem_20180124164533", "doc_count": 14 }, { "key": "ori_utrecht_20151221135818_new_migrated", "doc_count": 12 }, { "key": "ori_provincie-utrecht_20180605132843", "doc_count": 8 }, { "key": "ori_enschede_20170309152431_migrated", "doc_count": 7 }, { "key": "ori_goirle_20170203151744_migrated", "doc_count": 6 }, { "key": "ori_oude_ijsselstreek_20151221134613_new_migrated", "doc_count": 6 }, { "key": "ori_raalte_20180123103236", "doc_count": 6 }, { "key": "ori_almelo_20180123103246", "doc_count": 5 }, { "key": "ori_oosterhout_20180123095212", "doc_count": 5 }, { "key": "ori_amstelveen_20151221132805_new_migrated", "doc_count": 4 }, { "key": "ori_beemster_20180122161041", "doc_count": 4 }, { "key": "ori_roermond_20180122161449", "doc_count": 4 }, { "key": "ori_zuid-holland_20181018132040", "doc_count": 4 }, { "key": "ori_barneveld_20180123112938", "doc_count": 3 }, { "key": "ori_den_helder_20151221133407_new_migrated", "doc_count": 3 }, { "key": "ori_flevoland_20180618120622", "doc_count": 3 }, { "key": "ori_hilversum_20180123100207", "doc_count": 3 }, { "key": "ori_hollandskroon_20180125122446", "doc_count": 3 }, { "key": "ori_krimpenerwaard_20180124163621", "doc_count": 3 }, { "key": "ori_leiderdorp_20180122161024", "doc_count": 3 }, { "key": "ori_nieuwegein_20180124165712", "doc_count": 3 }, { "key": "ori_oss_20170310151053_migrated", "doc_count": 3 }, { "key": "ori_teylingen_20180123114737", "doc_count": 3 }, { "key": "ori_borne_20180122161105", "doc_count": 2 }, { "key": "ori_buren_20180123103402", "doc_count": 2 }, { "key": "ori_ettenleur_20180123103555", "doc_count": 2 }, { "key": "ori_heemskerk_20180124171613", "doc_count": 2 }, { "key": "ori_landsmeer_20180125145530", "doc_count": 2 }, { "key": "ori_leusden_20180122160946", "doc_count": 2 }, { "key": "ori_medemblik_20170310150856_migrated", "doc_count": 2 }, { "key": "ori_meierijstad_20180125140121", "doc_count": 2 }, { "key": "ori_rheden_20180627130051", "doc_count": 2 }, { "key": "ori_rijswijk_20180123103022", "doc_count": 2 }, { "key": "ori_staphorst_20180125140519", "doc_count": 2 }, { "key": "ori_aalsmeer_20180125122532", "doc_count": 1 }, { "key": "ori_diemen_20180125150906", "doc_count": 1 }, { "key": "ori_doetinchem_20170127130353_migrated", "doc_count": 1 }, { "key": "ori_drimmelen_20180606004659", "doc_count": 1 }, { "key": "ori_gouda_20170310143543_migrated", "doc_count": 1 }, { "key": "ori_heerde_20151221134011_new_migrated", "doc_count": 1 }, { "key": "ori_lingewaard_20180125140752", "doc_count": 1 }, { "key": "ori_meppel_20180125121920", "doc_count": 1 }, { "key": "ori_nijkerk_20180523005116", "doc_count": 1 }, { "key": "ori_noordoostpolder_20170207165214_migrated", "doc_count": 1 }, { "key": "ori_roosendaal_20180122161053", "doc_count": 1 }, { "key": "ori_steenbergen_20170207165350_migrated", "doc_count": 1 }, { "key": "ori_steenwijkerland_20180523121516", "doc_count": 1 }, { "key": "ori_texel_20180125122155", "doc_count": 1 }] } } } ```
V1 Meeting's oct 2018 ``` { "took": 217, "timed_out": false, "_shards": { "total": 123, "successful": 123, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 518, "relation": "eq" }, "max_score": null, "hits": [] }, "aggregations": { "maanden": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "ori_maastricht_20190531095140", "doc_count": 85 }, { "key": "ori_zaanstad_20190531095152", "doc_count": 42 }, { "key": "ori_almere_20190531095211", "doc_count": 28 }, { "key": "ori_amersfoort_20190531095151", "doc_count": 28 }, { "key": "ori_voorst_20190531095203", "doc_count": 28 }, { "key": "ori_deventer_20190531095221", "doc_count": 25 }, { "key": "ori_zeist_20190531095154", "doc_count": 25 }, { "key": "ori_groningen_20190531095157", "doc_count": 24 }, { "key": "ori_maassluis_20190531095149", "doc_count": 13 }, { "key": "ori_wageningen_20190531095149", "doc_count": 13 }, { "key": "ori_zandvoort_20190531095217", "doc_count": 11 }, { "key": "ori_den_helder_20190531095229", "doc_count": 10 }, { "key": "ori_leiden_20190531095215", "doc_count": 10 }, { "key": "ori_ede_20190531095146", "doc_count": 9 }, { "key": "ori_amsterdam_noord_20190531095142", "doc_count": 8 }, { "key": "ori_capelle_ad_ijssel_20190531095210", "doc_count": 8 }, { "key": "ori_gouda_20190531095159", "doc_count": 8 }, { "key": "ori_hulst_20190531095133", "doc_count": 8 }, { "key": "ori_stichtse_vecht_20190531095157", "doc_count": 8 }, { "key": "ori_amsterdam_20190531095206", "doc_count": 7 }, { "key": "ori_krimpen_ad_ijssel_20190531095214", "doc_count": 7 }, { "key": "ori_lingewaard_20190531095215", "doc_count": 7 }, { "key": "ori_meierijstad_20190531095141", "doc_count": 6 }, { "key": "ori_overbetuwe_20190531095202", "doc_count": 6 }, { "key": "ori_dongen_20190531095205", "doc_count": 5 }, { "key": "ori_leiderdorp_20190531095139", "doc_count": 5 }, { "key": "ori_nijkerk_20190531095146", "doc_count": 5 }, { "key": "ori_oss_20190531095154", "doc_count": 5 }, { "key": "ori_vlaardingen_20190531095135", "doc_count": 5 }, { "key": "ori_aalsmeer_20190531095143", "doc_count": 4 }, { "key": "ori_bloemendaal_20190531095141", "doc_count": 4 }, { "key": "ori_emmen_20190531095227", "doc_count": 4 }, { "key": "ori_roosendaal_20190531095208", "doc_count": 4 }, { "key": "ori_soest_20190531095143", "doc_count": 4 }, { "key": "ori_veenendaal_20190531095155", "doc_count": 4 }, { "key": "ori_vlissingen_20190531095136", "doc_count": 4 }, { "key": "ori_alkmaar_20190531095208", "doc_count": 3 }, { "key": "ori_den_haag_20190531095142", "doc_count": 3 }, { "key": "ori_haarlemmermeer_20190531095155", "doc_count": 3 }, { "key": "ori_moerdijk_20190531095138", "doc_count": 3 }, { "key": "ori_waalwijk_20190531095212", "doc_count": 3 }, { "key": "ori_amsterdam_centrum_20190531095151", "doc_count": 2 }, { "key": "ori_amsterdam_nieuw-west_20190531095137", "doc_count": 2 }, { "key": "ori_amsterdam_west_20190531095134", "doc_count": 2 }, { "key": "ori_amsterdam_zuidoost_20190531095135", "doc_count": 2 }, { "key": "ori_binnenmaas_20190531095207", "doc_count": 2 }, { "key": "ori_de_wolden_20190531095202", "doc_count": 2 }, { "key": "ori_hendrik-ido-ambacht_20190531095218", "doc_count": 2 }, { "key": "ori_landsmeer_20190531095152", "doc_count": 2 }, { "key": "ori_rheden_20190531095139", "doc_count": 2 }, { "key": "ori_schiedam_20190531095206", "doc_count": 2 }, { "key": "ori_steenwijkerland_20190531095138", "doc_count": 2 }, { "key": "ori_epe_20190531095225", "doc_count": 1 }, { "key": "ori_nieuwkoop_20190531095201", "doc_count": 1 }, { "key": "ori_raalte_20190531095218", "doc_count": 1 }, { "key": "ori_steenbergen_20190531095200", "doc_count": 1 } ] } } ```
V1 Meeting's feb 2019 ``` { "took": 230, "timed_out": false, "_shards": { "total": 123, "successful": 123, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 383, "relation": "eq" }, "max_score": null, "hits": [] }, "aggregations": { "maanden": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "ori_maastricht_20190531095140", "doc_count": 28 }, { "key": "ori_zaanstad_20190531095152", "doc_count": 26 }, { "key": "ori_den_haag_20190531095142", "doc_count": 25 }, { "key": "ori_zeist_20190531095154", "doc_count": 25 }, { "key": "ori_deventer_20190531095221", "doc_count": 24 }, { "key": "ori_almere_20190531095211", "doc_count": 20 }, { "key": "ori_voorst_20190531095203", "doc_count": 20 }, { "key": "ori_amersfoort_20190531095151", "doc_count": 14 }, { "key": "ori_maassluis_20190531095149", "doc_count": 11 }, { "key": "ori_alkmaar_20190531095208", "doc_count": 9 }, { "key": "ori_oss_20190531095154", "doc_count": 9 }, { "key": "ori_amsterdam_noord_20190531095142", "doc_count": 8 }, { "key": "ori_overbetuwe_20190531095202", "doc_count": 8 }, { "key": "ori_zandvoort_20190531095217", "doc_count": 8 }, { "key": "ori_edam-volendam_20190531095220", "doc_count": 7 }, { "key": "ori_groningen_20190531095157", "doc_count": 7 }, { "key": "ori_leiden_20190531095215", "doc_count": 7 }, { "key": "ori_lingewaard_20190531095215", "doc_count": 7 }, { "key": "ori_stichtse_vecht_20190531095157", "doc_count": 7 }, { "key": "ori_vlissingen_20190531095136", "doc_count": 7 }, { "key": "ori_wageningen_20190531095149", "doc_count": 7 }, { "key": "ori_ede_20190531095146", "doc_count": 6 }, { "key": "ori_leiderdorp_20190531095139", "doc_count": 6 }, { "key": "ori_roosendaal_20190531095208", "doc_count": 6 }, { "key": "ori_vlaardingen_20190531095135", "doc_count": 6 }, { "key": "ori_capelle_ad_ijssel_20190531095210", "doc_count": 5 }, { "key": "ori_den_helder_20190531095229", "doc_count": 5 }, { "key": "ori_gouda_20190531095159", "doc_count": 5 }, { "key": "ori_veenendaal_20190531095155", "doc_count": 5 }, { "key": "ori_amsterdam_20190531095206", "doc_count": 4 }, { "key": "ori_hendrik-ido-ambacht_20190531095218", "doc_count": 4 }, { "key": "ori_nieuwkoop_20190531095201", "doc_count": 4 }, { "key": "ori_schiedam_20190531095206", "doc_count": 4 }, { "key": "ori_aalsmeer_20190531095143", "doc_count": 3 }, { "key": "ori_amsterdam_west_20190531095134", "doc_count": 3 }, { "key": "ori_dongen_20190531095205", "doc_count": 3 }, { "key": "ori_nijkerk_20190531095146", "doc_count": 3 }, { "key": "ori_zeewolde_20190531095209", "doc_count": 3 }, { "key": "ori_amsterdam_centrum_20190531095151", "doc_count": 2 }, { "key": "ori_amsterdam_nieuw-west_20190531095137", "doc_count": 2 }, { "key": "ori_amsterdam_zuidoost_20190531095135", "doc_count": 2 }, { "key": "ori_doetinchem_20190531095202", "doc_count": 2 }, { "key": "ori_emmen_20190531095227", "doc_count": 2 }, { "key": "ori_krimpen_ad_ijssel_20190531095214", "doc_count": 2 }, { "key": "ori_landsmeer_20190531095152", "doc_count": 2 }, { "key": "ori_moerdijk_20190531095138", "doc_count": 2 }, { "key": "ori_steenwijkerland_20190531095138", "doc_count": 2 }, { "key": "ori_de_wolden_20190531095202", "doc_count": 1 }, { "key": "ori_hulst_20190531095133", "doc_count": 1 }, { "key": "ori_ommen_20190531095148", "doc_count": 1 }, { "key": "ori_rheden_20190531095139", "doc_count": 1 }, { "key": "ori_soest_20190531095143", "doc_count": 1 }, { "key": "ori_steenbergen_20190531095200", "doc_count": 1 } ] } } } ```
V1 Meeting's may 2019 ``` { "took": 208, "timed_out": false, "_shards": { "total": 123, "successful": 123, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 715, "relation": "eq" }, "max_score": null, "hits": [] }, "aggregations": { "maanden": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "ori_maastricht_20190531095140", "doc_count": 75 }, { "key": "ori_deventer_20190531095221", "doc_count": 38 }, { "key": "ori_amsterdam_noord_20190531095142", "doc_count": 31 }, { "key": "ori_zeist_20190531095154", "doc_count": 31 }, { "key": "ori_voorst_20190531095203", "doc_count": 29 }, { "key": "ori_almere_20190531095211", "doc_count": 28 }, { "key": "ori_den_haag_20190531095142", "doc_count": 25 }, { "key": "ori_groningen_20190531095157", "doc_count": 23 }, { "key": "ori_arnhem_20190531095204", "doc_count": 16 }, { "key": "ori_zwolle_20190531095156", "doc_count": 16 }, { "key": "ori_amersfoort_20190531095151", "doc_count": 13 }, { "key": "ori_maassluis_20190531095149", "doc_count": 13 }, { "key": "ori_wageningen_20190531095149", "doc_count": 13 }, { "key": "ori_utrecht_20190531095223", "doc_count": 12 }, { "key": "ori_zaanstad_20190531095152", "doc_count": 12 }, { "key": "ori_amsterdam_oost_20190531095228", "doc_count": 10 }, { "key": "ori_culemborg_20190531095148", "doc_count": 10 }, { "key": "ori_enschede_20190531095140", "doc_count": 10 }, { "key": "ori_raalte_20190531095218", "doc_count": 10 }, { "key": "ori_rhenen_20190531095218", "doc_count": 10 }, { "key": "ori_stichtse_vecht_20190531095157", "doc_count": 10 }, { "key": "ori_den_helder_20190531095229", "doc_count": 9 }, { "key": "ori_amsterdam_20190531095206", "doc_count": 8 }, { "key": "ori_alkmaar_20190531095208", "doc_count": 7 }, { "key": "ori_ede_20190531095146", "doc_count": 7 }, { "key": "ori_goirle_20190531095158", "doc_count": 7 }, { "key": "ori_oisterwijk_20190531095144", "doc_count": 7 }, { "key": "ori_oss_20190531095154", "doc_count": 7 }, { "key": "ori_zandvoort_20190531095217", "doc_count": 7 }, { "key": "ori_capelle_ad_ijssel_20190531095210", "doc_count": 6 }, { "key": "ori_gouda_20190531095159", "doc_count": 6 }, { "key": "ori_hendrik-ido-ambacht_20190531095218", "doc_count": 6 }, { "key": "ori_katwijk_20190531095158", "doc_count": 6 }, { "key": "ori_leiden_20190531095215", "doc_count": 6 }, { "key": "ori_leiderdorp_20190531095139", "doc_count": 6 }, { "key": "ori_zoetermeer_20190531095223", "doc_count": 6 }, { "key": "ori_almelo_20190531095209", "doc_count": 5 }, { "key": "ori_beemster_20190531095200", "doc_count": 5 }, { "key": "ori_doetinchem_20190531095202", "doc_count": 5 }, { "key": "ori_krimpen_ad_ijssel_20190531095214", "doc_count": 5 }, { "key": "ori_lingewaard_20190531095215", "doc_count": 5 }, { "key": "ori_nijkerk_20190531095146", "doc_count": 5 }, { "key": "ori_vlissingen_20190531095136", "doc_count": 5 }, { "key": "ori_waalwijk_20190531095212", "doc_count": 5 }, { "key": "ori_amstelveen_20190531095207", "doc_count": 4 }, { "key": "ori_amsterdam_zuidoost_20190531095135", "doc_count": 4 }, { "key": "ori_bloemendaal_20190531095141", "doc_count": 4 }, { "key": "ori_etten-leur_20190531095222", "doc_count": 4 }, { "key": "ori_hilvarenbeek_20190531095153", "doc_count": 4 }, { "key": "ori_hollands_kroon_20190531095226", "doc_count": 4 }, { "key": "ori_landsmeer_20190531095152", "doc_count": 4 }, { "key": "ori_medemblik_20190531095216", "doc_count": 4 }, { "key": "ori_roosendaal_20190531095208", "doc_count": 4 }, { "key": "ori_schiedam_20190531095206", "doc_count": 4 }, { "key": "ori_vlaardingen_20190531095135", "doc_count": 4 }, { "key": "ori_zeewolde_20190531095209", "doc_count": 4 }, { "key": "ori_amsterdam_zuid_20190531095219", "doc_count": 3 }, { "key": "ori_barneveld_20190531095217", "doc_count": 3 }, { "key": "ori_emmen_20190531095227", "doc_count": 3 }, { "key": "ori_hilversum_20190531095144", "doc_count": 3 }, { "key": "ori_krimpenerwaard_20190531095222", "doc_count": 3 }, { "key": "ori_moerdijk_20190531095138", "doc_count": 3 }, { "key": "ori_nieuwegein_20190531095147", "doc_count": 3 }, { "key": "ori_noordoostpolder_20190531095221", "doc_count": 3 }, { "key": "ori_overbetuwe_20190531095202", "doc_count": 3 }, { "key": "ori_steenbergen_20190531095200", "doc_count": 3 }, { "key": "ori_teylingen_20190531095227", "doc_count": 3 }, { "key": "ori_veenendaal_20190531095155", "doc_count": 3 }, { "key": "ori_aalsmeer_20190531095143", "doc_count": 2 }, { "key": "ori_amsterdam_nieuw-west_20190531095137", "doc_count": 2 }, { "key": "ori_amsterdam_west_20190531095134", "doc_count": 2 }, { "key": "ori_borne_20190531095226", "doc_count": 2 }, { "key": "ori_buren_20190531095225", "doc_count": 2 }, { "key": "ori_castricum_20190531095212", "doc_count": 2 }, { "key": "ori_diemen_20190531095204", "doc_count": 2 }, { "key": "ori_dongen_20190531095205", "doc_count": 2 }, { "key": "ori_haarlemmermeer_20190531095155", "doc_count": 2 }, { "key": "ori_hardenberg_20190531095159", "doc_count": 2 }, { "key": "ori_heerde_20190531095205", "doc_count": 2 }, { "key": "ori_leusden_20190531095219", "doc_count": 2 }, { "key": "ori_losser_20190531095209", "doc_count": 2 }, { "key": "ori_meierijstad_20190531095141", "doc_count": 2 }, { "key": "ori_ommen_20190531095148", "doc_count": 2 }, { "key": "ori_oost_gelre_20190531095222", "doc_count": 2 }, { "key": "ori_rheden_20190531095139", "doc_count": 2 }, { "key": "ori_rijswijk_20190531095210", "doc_count": 2 }, { "key": "ori_staphorst_20190531095216", "doc_count": 2 }, { "key": "ori_steenwijkerland_20190531095138", "doc_count": 2 }, { "key": "ori_velsen_20190531095213", "doc_count": 2 }, { "key": "ori_amsterdam_centrum_20190531095151", "doc_count": 1 }, { "key": "ori_baarn_20190531095213", "doc_count": 1 }, { "key": "ori_de_wolden_20190531095202", "doc_count": 1 }, { "key": "ori_drimmelen_20190531095224", "doc_count": 1 }, { "key": "ori_epe_20190531095225", "doc_count": 1 }, { "key": "ori_houten_20190531095220", "doc_count": 1 }, { "key": "ori_meppel_20190531095150", "doc_count": 1 }, { "key": "ori_nieuwkoop_20190531095201", "doc_count": 1 }, { "key": "ori_oldambt_20190531095147", "doc_count": 1 }, { "key": "ori_oude_ijsselstreek_20190531095225", "doc_count": 1 }, { "key": "ori_schagen_20190531095228", "doc_count": 1 }, { "key": "ori_texel_20190531095150", "doc_count": 1 }, { "key": "ori_zwartewaterland_20190531095201", "doc_count": 1 } ] } } } ```

Some conclusions based on oct 2018:

jurrian commented 5 years ago

The indices in both V0 and V1 as of this moment. Beware that the index ori_combined_index_new_migrated contains everything double, so make sure to exclude it from comparison.

All indices V0 ``` green open ori_amsterdam_zuidoost_20180122161706 lbPH0DhWSPuZHzURGKmSOw 1 0 1995 146 48.4mb 48.4mb green open ori_amsterdam_noord_20180122161732 _zk8860hRyOLmSQ3CdnaJg 1 0 3553 196 83.6mb 83.6mb green open ori_emmen_20180125135538 9YCIFtCMSv2Qb-oCfI2Ujw 1 0 0 0 162b 162b green open ori_lingewaard_20180125140752 Q5F9YJpgQCWdLlt6WO7n_A 1 0 3876 1105 164.4mb 164.4mb green open ori_goirle_20170203151744_migrated aLSvts-vTjKzGvLLeOHQgg 1 0 3758 386 77.7mb 77.7mb green open ori_gouda_20170310143543_migrated qDN_UnRkTfaeQmd1UPraWA 1 0 3072 363 233.3mb 233.3mb green open ori_epe_20180122161831 wBNCCHt9SEauPw4gbYelUg 1 0 2174 152 104.9mb 104.9mb green open ori_noord-holland_20180605132038 K61yQqdlSzigFgh__Sx8XQ 1 0 21117 2492 1014.2mb 1014.2mb green open ori_velsen_20180125172527 4vLshAqQSLuOFw2EyUYNBA 1 0 1644 822 86.1mb 86.1mb green open ori_noordoostpolder_20170207165214_migrated 3_z0uPgcQbS8CU4WZ9MTzg 1 0 3599 1067 101.9mb 101.9mb green open ori_leiden_20180122161529 0qr4UvgJRmugZe6vkKNwoA 1 0 7392 521 291.8mb 291.8mb green open ori_meppel_20180125121920 F61DiOGCRy6ab5Wvcy2Xzg 1 0 3092 1429 111.2mb 111.2mb green open ori_borne_20180122161105 vIwbluDFQR-yAjWRuPPQwg 1 0 2810 171 47mb 47mb green open ori_oss_20170310151053_migrated Qash8qn1T0mFtotPWeH4NA 1 0 6017 1444 190mb 190mb green open ori_leusden_20180122160946 NoO4q-n3TfqlUQatZf5rBw 1 0 2875 362 61.6mb 61.6mb green open ori_heerde_20151221134011_new_migrated eACOTQ5ZRfWSZI1WjRTwkA 1 0 3762 880 174.1mb 174.1mb green open ori_teylingen_20180123114737 02FMKjl4SxuMwttwgn1YJQ 1 0 6261 2243 328.2mb 328.2mb green open ori_aalsmeer_20180125122532 y2hL_yqaRri7EtsOKX3aQA 1 0 2786 793 115mb 115mb green open ori_voorschoten_20180127104535 rR7fLFrSROOA32QayViDmQ 1 0 3817 505 200.3mb 200.3mb green open ori_hilversum_20180123100207 gX9BFWqTT3mB-4VqPlwDDA 1 0 7122 2428 387.9mb 387.9mb green open ori_amsterdam_nieuwwest_20180122161143 BEFnV7OvTSqx_T0LKt7W0A 1 0 3346 163 108.9mb 108.9mb green open ori_rheden_20180627130051 UZ2_1hOyT4yqHm0No2mXbg 1 0 3088 308 87.6mb 87.6mb green open ori_amstelveen_20151221132805_new_migrated siUyZhBhSRO8DAQF8b5VCg 1 0 7268 938 363.7mb 363.7mb green open ori_groningen_20180122161714 MkfYyWCLSSWLSuGdlEYpEQ 1 0 6841 769 228.6mb 228.6mb green open ori-combined-index w9HwNOlqR6u4uGXtcP3CXA 5 1 0 0 1.3kb 679b green open ori-zuid-holland 1-gdIqamQ9OEtlaftYrF3A 5 1 0 0 1.3kb 679b green open ori_amsterdam_centrum_20180122161753 7CzFJ3PFSBC0wWMffY4wxw 1 0 4175 247 110.5mb 110.5mb green open ori_schiedam_20180122161524 wEnwbI8mT3mlp1_zWow4TA 1 0 2751 201 165.5mb 165.5mb green open ori_noordwijk_20180122161839 L0i2e5-hTQO30b1hDt5OEg 1 0 6179 398 151.4mb 151.4mb green open ori_ettenleur_20180123103555 BER8MgoTSlODAvkcDb6U2g 1 0 3876 1101 147.4mb 147.4mb green open ori_schagen_20180122161801 FZTkfCQvSSSW2R7fnnYrFA 1 0 5125 303 148.6mb 148.6mb green open ori_noordwijkerhout_20180125144832 GGWGDS9YRDqrHGe0H-xEOw 1 0 2849 1112 113.3mb 113.3mb green open ori_barneveld_20180123112938 aF2ndMMeS12UbXkD5nY6Ug 1 0 2624 940 390.3mb 390.3mb green open ori_hulst_20180122161510 Ee5yb-LeS4ywRUrNRrJX6A 1 0 3866 339 78mb 78mb green open ori_zwolle_20180125145125 5lE_44p5RpmweMCpriftSw 1 0 10712 2005 346.8mb 346.8mb green open ori_voorst_20180122161636 m_xqtvw6QeO-m2D5cJHqGQ 1 0 2313 351 84.3mb 84.3mb green open ori_nieuwkoop_20180123103222 t-ZIQu8KTqWrjEP6bB_dkw 1 0 1682 250 84.6mb 84.6mb green open ori_enschede_20170309152431_migrated gK1qTDANRt-lw_kVdhQWCQ 1 0 7931 964 235.1mb 235.1mb green open ori_castricum_20180122161204 -uHLKtFIQHSjEBpKnx0Orw 1 0 1878 144 44.3mb 44.3mb green open ori_edamvolendam_20180125150152 nzRjcN5oTwqVdsyPCvpWqA 1 0 3695 718 134.9mb 134.9mb green open ori_limburg_20180605132257 wCrAbs6mTGKS1mZcqNBk6w 1 0 28510 3006 994.4mb 994.4mb green open ori_oldambt_20180125142217 e-J-hjivTeiI0ZhxmTaMKQ 1 0 2511 1454 72.3mb 72.3mb green open ori_nieuwegein_20180124165712 OxmaSV97SrSaaE0GjW19Cg 1 0 4467 1723 228.8mb 228.8mb green open ori_flevoland_20180618120622 0c2aXhKjQ7iCvxJUGrI3Mg 1 0 22425 393 398.1mb 398.1mb green open ori_molenwaard_20180122161049 nE2lxLQWQBqv0Q25JVfTKQ 1 0 1655 176 35.1mb 35.1mb green open ori_rucphen_20180122161558 0AW3TxPwQ4uCSNCxv8MmlA 1 0 2903 126 119.4mb 119.4mb green open ori_overbetuwe_20151221135216_new_migrated kyEvoQi2Tcmw-TiAq8fQ_Q 1 0 2374 0 143.7mb 143.7mb green open ori_rhenen_20180122161649 J6GxVXdhT5ezQMj91NNTZg 1 0 2288 243 66.6mb 66.6mb green open ori_buren_20180123103402 0tqbaARmQx26wYM_ZpE7Ww 1 0 2968 1033 69.7mb 69.7mb green open ori_maastricht_20180122161222 KTf1uja9TO26124lYuCJ_g 1 0 2117 16 19.1mb 19.1mb green open ori_harderwijk_20160602115248_migrated ELT18mSVQRSoupSJV48Vjw 1 0 301 3 38.5mb 38.5mb green open ori_heemskerk_20180124171613 46wVWLHASKK7AgBjzRZUGA 1 0 4717 1632 89.1mb 89.1mb green open ori_provincie-utrecht_20180605132843 lWi5IfUzSqq7f-LPyJaOCw 1 0 33473 282 644.4mb 644.4mb green open ori_resolver hRsFe5ehSqWqeJpPtUuoFw 1 0 429857 6301 60.6mb 60.6mb green open ori_boxmeer_20180523121239 WJPAlqneTVKigq4AzVGrkw 1 0 3610 362 104.5mb 104.5mb green open ori_steenwijkerland_20180523121516 VR9FeIaqTeGtbe7YKEcKtA 1 0 1456 559 48.4mb 48.4mb green open ori_bodegravenreeuwijk_20180605133139 7hnNIiQVRayVvNNR5ueC8g 1 0 5078 159 111.3mb 111.3mb green open ori_amersfoort_20180122161238 n1ceUYOPSjWlreEpZ3-Odw 1 0 1434 105 23.2mb 23.2mb green open ori_drimmelen_20180606004659 i7E7sOkdRJKa-b7g_vMV3w 1 0 2636 985 92.2mb 92.2mb green open ori_krimpenerwaard_20180124163621 t98PZjBZSHuU3jH0U2GrWw 1 0 3498 1232 161.1mb 161.1mb green open ori_vlissingen_20180122161819 VZX007EUTiiF2CgUdmkVNQ 1 0 4876 177 127.5mb 127.5mb green open ori_ommen_20180122161518 Kl1xwxEKTTyNLV39QEeVcQ 1 0 2676 220 97mb 97mb green open ori_soest_20180122161123 gNQ_zMywQrqbQhRiH9vIHQ 1 0 2001 96 129.1mb 129.1mb green open ori_rijswijk_20180123103022 TkNFT3iDQuaH-lOf36Mw8Q 1 0 4289 1506 151.6mb 151.6mb green open ori_landsmeer_20180125145530 -JVyQmqPRDCG_7Z8z0ZYUw 1 0 1394 128 44.8mb 44.8mb green open ori_utrecht_20151221135818_new_migrated 2pnH7E-NTbaGGZyMv-fpUw 1 0 27544 3897 6.6gb 6.6gb green open ori_oostgelre_20180122161646 DSi_vPHeSIq1bSgqq7FmmQ 1 0 2532 543 155.9mb 155.9mb green open ori_bloemendaal_20190111141248 Xb_gjQf0ToO3_-Z_BRFLIQ 1 0 2 0 10.6kb 10.6kb green open ori_medemblik_20170310150856_migrated QyRFeXkrTICSIG7cnlpT0A 1 0 12125 4039 637.6mb 637.6mb green open ori_zwartewaterland_20180519010027 sgjRC1iJTdK16j_TKrQDIQ 1 0 278 0 5.3mb 5.3mb green open ori_stichtsevecht_20180122161836 cnrkL6NwQi6yPxwXJ7NH7g 1 0 2352 118 89.8mb 89.8mb green open ori_arnhem_20180124164533 oOpLjCGeQWG59GI4ddT0TA 1 0 7029 3078 309.5mb 309.5mb green open ori_binnenmaas_20180122161553 hosWCJWoTBixDTiLrxBKIw 1 0 2551 174 128.9mb 128.9mb green open ori_oosterhout_20180123095212 OFOE6xJ8SaaYDFwP4HfLow 1 0 2340 1143 71.3mb 71.3mb green open ori_alkmaar_20180209111005 MlbNwrAYShq4WZU1qN4pgg 1 0 6039 146 206.4mb 206.4mb green open ori_den_helder_20151221133407_new_migrated voUqUvMJSVuTyIEx9ZiYSw 1 0 7935 538 204.1mb 204.1mb green open ori_combined_index_new_migrated KYzjzEypSWGEJe6RwU067w 1 0 563385 114291 17.5gb 17.5gb green open ori_texel_20180125122155 mUtDXSJSR6WGFUNSwpK7zg 1 0 4702 1330 210.9mb 210.9mb green open ori_nijkerk_20180523005116 LjjOsM6qSFm8xKRSltUs3g 1 0 2333 427 79.2mb 79.2mb green open ori_oude_ijsselstreek_20151221134613_new_migrated l2ZCbU0YSO-FqFDFaIy4Lg 1 0 3000 1018 178.9mb 178.9mb green open ori_zandvoort_20180122161629 w7epBk-rRFmCqiRNg7n0Bw 1 0 3588 204 122.6mb 122.6mb green open ori_hardenberg_20180122161828 DTnHjUnOQLiACLnwd9ehDA 1 0 3378 144 107.2mb 107.2mb green open ori_zeist_20180122161742 ViBV73-jQtavut3GdO0oPw 1 0 2526 263 75.2mb 75.2mb green open ori_zaanstad_20180122161607 YWd8UYhRQTaqyeH6RLV3bw 1 0 2998 167 137.9mb 137.9mb green open ori_waalwijk_20180122161153 F466Vbr1RI-opB51acvzdw 1 0 4041 615 182.8mb 182.8mb green open ori_krimpenadijssel_20180122161548 iXp4EV11T5moWHVctTMJdw 1 0 2283 117 40.1mb 40.1mb green open ori_loonopzand_20180123103353 5GiB49TWThCgIsnG7gdy4A 1 0 1390 170 55.4mb 55.4mb green open ori_hendrikidoambacht_20180122161441 Iw0ThYxpREipBNaL6rQnJA 1 0 4205 380 147.1mb 147.1mb green open ori_ede_20180519010322 LiiYAUgGQkWyQhfoHvFnUg 1 0 6440 31 149.2mb 149.2mb green open ori_almelo_20180123103246 LN75DZNaRqmdmxf-uVmqQw 1 0 3959 1367 241.8mb 241.8mb green open ori_hollandskroon_20180125122446 35kyvIsFQD6KBn02dJDuow 1 0 1564 365 52.9mb 52.9mb green open ori_leeuwarden_20180122161139 4xvPR7pTSbmpV0E07vdVzg 1 0 1238 225 55.3mb 55.3mb green open ori_leiderdorp_20180122161024 ekmdcKaEQYWQZ7oDD8Yfqg 1 0 3355 314 62.9mb 62.9mb green open ori_moerdijk_20180122161808 4DRlrSPmTMClgwbiL3f_nQ 1 0 3759 251 97.6mb 97.6mb green open ori_beemster_20180122161041 lS0-VMS3SQePJ3iybFnfxg 1 0 3305 47 35.3mb 35.3mb green open ori_staphorst_20180125140519 Gd_smNeNQ3mqZ-d5r8LhLw 1 0 3043 669 98.9mb 98.9mb green open ori_katwijk_20180122161656 XqZEERwOSlihJSWEQhO2aw 1 0 8842 658 482.3mb 482.3mb green open ori_wageningen_20180122161620 Kg6EkzL_RFWyv_xTeP8IPA 1 0 1444 90 50.1mb 50.1mb green open ori_veenendaal_20180122161208 T3XBQo-PS4K6UDUdWKCN7w 1 0 3312 462 209.8mb 209.8mb green open ori_zoetermeer_20170313132023_migrated -DajRrpXRquQJv0gMmbVOA 1 0 8847 3303 406.6mb 406.6mb green open ori_losser_20180126110130 iMT6yAxBScKI_OWrd8vaUw 1 0 1608 72 26.9mb 26.9mb green open ori_steenbergen_20170207165350_migrated JkuMvjF3QJC3t1LYKXnCAw 1 0 3759 931 128.6mb 128.6mb green open ori_amsterdam_20180122161245 pAiffpmYQWyIXmh7r1yzxQ 1 0 46579 7286 1.8gb 1.8gb green open ori_cappelleadijssel_20180122161444 PK8_T7K0REW3KLiNTAy9GQ 1 0 3367 517 156.2mb 156.2mb green open ori_roermond_20180122161449 lOxS1AaqR7uNP6cAPGutqw 1 0 5894 277 180.2mb 180.2mb green open ori_vlaardingen_20180122161349 Rm6WVpuaRCev7Dyy-JXR4w 1 0 3491 608 131.6mb 131.6mb green open ori_oisterwijk_20180122161234 8yLkGzr2TTuw-WQLw1IdMA 1 0 4373 183 129.5mb 129.5mb green open ori_maassluis_20180122161736 gLdjwHLXQi6H25cARap_zg 1 0 7649 197 195.1mb 195.1mb green open ori_dewolden_20180122155825 LDtz_IfWRcSexi2oUOPWdA 1 0 1737 59 32.6mb 32.6mb green open ori_meierijstad_20180125140121 E3nAJ8yRTSepHwi4KcRa4A 1 0 2895 1216 142.5mb 142.5mb green open ori_dongen_20180122161626 f5lPHhSJTWWg6qXbw2dBxg 1 0 1789 117 57.6mb 57.6mb green open ori_diemen_20180125150906 XCX6en3zSBiI4rPmZbZs3A 1 0 2600 502 91.2mb 91.2mb green open ori_denhaag_20180122161458 36lRkTcRRz-v356uO8hTag 1 0 9404 351 27.6mb 27.6mb green open ori_culemborg_20180122161617 WRmpd_TCRlOg0x9v_aHE5w 1 0 3084 178 74.6mb 74.6mb green open ori_zuid-holland_20181018132040 85RhTkYYTiO_be4yrbY-8g 1 0 6765 1282 541.2mb 541.2mb green open ori_amsterdam_oost_20180122161601 O7PovVsRThy5UGUYT1SaAg 1 0 3778 280 84.5mb 84.5mb green open ori_almere_20180122161542 UbVrP6waS9OUawddd9-tUg 1 0 4129 225 138.6mb 138.6mb green open ori_roosendaal_20180122161053 MBFZBCduSbikBwD1PjwEQw 1 0 955 4 13.6mb 13.6mb green open ori_amsterdam_zuid_20180122161726 vAFV5D0qTn2XkYwhqYzZxA 1 0 3353 117 135mb 135mb green open ori_raalte_20180123103236 xWu7kjYTSfOEbU-bkmv6nw 1 0 6630 666 228.2mb 228.2mb green open ori_wassenaar_20180126130808 2MaMJ0fwSeKSgtvXLim91w 1 0 3494 286 115.5mb 115.5mb green open ori_amsterdam_west_20180122161623 OR0YclE7QW-nFbL1iCxrNg 1 0 635 2 12.6mb 12.6mb green open ori_baarn_20180122161147 VEZ28qfoTSGGDqcLkY46Dg 1 0 2279 276 154.9mb 154.9mb green open ori_hilvarenbeek_20180122161214 jIyD086rRFq-fYCKxFQ2CA 1 0 3480 320 158.5mb 158.5mb green open ori_haarlemmermeer_20190111135004 9FQDSkBoSeuPUhptQqVKNA 1 0 3092 13 257.6mb 257.6mb green open ori_deventer_20180122161434 dC7du9SRTAiHcdM32m_yLQ 1 0 2087 307 95.6mb 95.6mb green open ori_doetinchem_20170127130353_migrated fWnMs9F0Td2AMvn8rBJSYQ 1 0 2254 256 73.4mb 73.4mb ```
All indices V1 ``` green open ori_almere_20190531095211 qFYfz5k_S6e80ia3EnczcA 1 1 2744 1279 30.5mb 15.2mb green open ori_goirle_20190531095158 EJAZW8g1TZe6Kx_SbPTF6A 1 1 546 57 726.1kb 363kb green open ori_katwijk_20190531095158 iYyQ2_89QT-foXk0gwHf-Q 1 1 217 0 2.4mb 1.2mb green open ori_hardenberg_20190531095159 xaaQYgaNSO-i90ktY_zsRQ 1 1 79 12 1.2mb 639.5kb green open ori_schiedam_20190531095206 NWR-rdhdSxayvW95sEFbMg 1 1 881 0 9.8mb 3.7mb green open ori_emmen_20190531095227 UAeGb7bTQo-qguw5vR6QLg 1 1 2950 0 27.7mb 13.8mb green open ori_krimpen_ad_ijssel_20190531095214 h_lSkVd8QCuzuz0CqCu1Nw 1 1 1690 333 20mb 10mb green open ori_hilvarenbeek_20190531095153 lfjw9O5USiCdOpGL8aKiyA 1 1 178 37 1.8mb 925kb green open ori_ommen_20190531095148 rC6vgQUuQfeEsXaDJ_MQ8Q 1 1 302 0 2.8mb 1.4mb green open ori_vlissingen_20190531095136 kWygIxHKQsGOx9eZsS9zlA 1 1 1088 403 6.9mb 3.4mb green open ori_binnenmaas_20190531095207 08h1lVxXQUm2M6Y5h00IUQ 1 1 51 24 563.5kb 281.6kb green open ori_moerdijk_20190531095138 GSl3-fHySuaryRQpNPh8Gg 1 1 568 227 1mb 578.6kb green open ori_heerde_20190531095205 sSHEoVUyQ3ict7XEU0E8Eg 1 1 959 254 4.7mb 2.3mb green open ori_maastricht_20190531095140 2Eee4688QFO6SgPbnVsz2w 1 1 1456 97 6mb 3mb green open ori_oss_20190531095154 igGfwmm3SAOxUEry-_mwIA 1 1 5394 1626 33mb 16.5mb green open ori_nijkerk_20190531095146 t-1aApaBQsm_Ia9TspsARg 1 1 1931 12 24.4mb 12.2mb green open osi_noord-holland_20190531095331 O1rCoFSpTS2sZ5Y-CNTngg 1 1 70 0 79.9kb 39.9kb green open ori_steenbergen_20190531095200 QF8_5oSBTJ2OHJQSMuo8vA 1 1 859 0 850.9kb 424.8kb green open ori_velsen_20190531095213 Nk6kF-vbSuOZy9triIX59A 1 1 1003 0 3.9mb 1.9mb green open ori_zeewolde_20190531095209 xZu0caAKTWCiO5PvPzvKdg 1 1 1001 4 8.5mb 4.2mb green open ori_oisterwijk_20190531095144 UIDkoFDoTDG3H09YV9lrSg 1 1 187 61 3.1mb 1.5mb green open ori_beemster_20190531095200 Zm0Oy8sMQ1-Iob_Ocekcpw 1 1 368 167 801.6kb 400.8kb green open ori_amsterdam_oost_20190531095228 f1rtmaWKQJ2hUW_nL3iasA 1 1 230 0 1.1mb 586.7kb green open ori_leiderdorp_20190531095139 aBCQyd02SriAdaa1UB_QhQ 1 1 2281 486 3.6mb 1.8mb green open ori_rucphen_20190531095145 KX7c7gd1SvCDHRTHBK1drw 1 1 89 1 2.3mb 1.1mb green open osi_utrecht_20190531095330 0dfFH6V_S62IkSQTqKmOUw 1 1 0 0 566b 283b green open ori_krimpenerwaard_20190531095222 DNBl6T0jTACnGpwztYlQwg 1 1 1034 505 5.7mb 2.8mb green open ori_leusden_20190531095219 _by2IIftQWS8bv8VrRUNLQ 1 1 540 0 532.3kb 266kb green open ori_almelo_20190531095209 QuirweaNQpOZKBDT6QZJmg 1 1 747 253 10.9mb 5.4mb green open osi_flevoland_20190531095330 TZvhQH54QFSU6wiGm33VTQ 1 1 465 0 227.2kb 113.6kb green open ori_nijmegen_20190531095156 gGGEj753SLSjtas30Ba_4w 1 1 12 4 70.5kb 49kb green open ori_waalwijk_20190531095212 tgJRijuoRXyFbYNxmFkS-A 1 1 283 9 1.4mb 741.5kb green open ori_castricum_20190531095212 jZuKrTbZQhW1DXqJGpdJuw 1 1 60 14 567.7kb 283.8kb green open ori_enschede_20190531095140 LFbnzZq7TSiGkFWLFivnbQ 1 1 2159 635 9.5mb 5.2mb green open ori_zwolle_20190531095156 Pw7X0UhgRKOI0yQWOdRSRg 1 1 4149 0 20mb 9.5mb green open ori_bloemendaal_20190531095141 7xi0WadiR7u1-9-S7LACHw 1 1 1596 592 2mb 1.1mb green open ori_zwartewaterland_20190531095201 jymXNu1ZTje2CXowiSecWA 1 1 158 3 1.7mb 896.1kb green open ori_barneveld_20190531095217 8CEjUe_NSD6in-dPvFzHhQ 1 1 544 260 8.6mb 4.3mb green open ori_hilversum_20190531095144 ZGHuM464Q3ahn_1adAeJbQ 1 1 1388 0 6.7mb 3.3mb green open ori_hendrik-ido-ambacht_20190531095218 8LsDphekTp6UKCxR7L4A7Q 1 1 1881 42 25.9mb 12.9mb green open ori_stichtse_vecht_20190531095157 Rkea-SiERsSZiobIH35gdg 1 1 3779 728 4.5mb 2.2mb green open ori_etten-leur_20190531095222 0DQHfCcXTFGe5m0e2ObpYw 1 1 671 260 5.1mb 2.8mb green open ori_wageningen_20190531095149 BivxXaObRBqwf4OeK9BGaQ 1 1 827 83 4.6mb 2.3mb green open ori_ede_20190531095146 uNK_Upv8SPe_jlyb_2e3Sg 1 1 1280 189 15.1mb 7.6mb green open ori_meppel_20190531095150 nHYKULFgSS23uMTj4IO_MA 1 1 838 305 6.1mb 3mb green open ori_voorschoten_20190531095136 Y2J5kFh4QUus9aEuS5dVRA 1 1 8 0 60.5kb 19.5kb green open ori_amersfoort_20190531095151 9hr2CffDRP-63fIeiX6Fow 1 1 751 152 2.7mb 1.3mb green open ori_leeuwarden_20190531095229 k2z38f-5RKSmbG0eHRg1OQ 1 1 37 36 77kb 38.5kb green open ori_raalte_20190531095218 Y-1kkZ57S42IudLatkdpZA 1 1 1002 0 9.5mb 4.7mb green open ori_amsterdam_centrum_20190531095151 ffEb805IQjKhKF0na7Pj1A 1 1 891 284 13mb 6.5mb green open ori_oost_gelre_20190531095222 vnkfw9uHSlyvC6sy6XQLBA 1 1 252 3 3.3mb 1.6mb green open ori_texel_20190531095150 2b7D4bqgTdGT52GROuiKOA 1 1 771 12 3.8mb 1.9mb green open ori_zandvoort_20190531095217 rUgruBqwQee1hiHcUAPSQg 1 1 2504 861 43.6mb 21.8mb green open ori_heemskerk_20190531095204 b2qoJEluTxSdsXOR1Iu4Kg 1 1 8 1 82.8kb 41.4kb green open ori_drimmelen_20190531095224 v5YSifuMSZq3etZC8fTqkw 1 1 459 226 1.5mb 798.4kb green open ori_gouda_20190531095159 qVEQHK0HSLmIpB9tJs02Wg 1 1 2445 0 21.5mb 9.6mb green open ori_zaanstad_20190531095152 UN3vaqJnTdeepMyw7KocDA 1 1 1745 183 20.3mb 10.2mb green open ori_diemen_20190531095204 4_3lj5epQI2U0jlMpovAeg 1 1 1051 236 4.9mb 2.4mb green open ori_arnhem_20190531095204 6vARiheiStiNi5OojL8FCQ 1 1 2307 2 9.7mb 4.9mb green open ori_amstelveen_20190531095207 o_ATCIN4RoGOJ-w3pzD_RA 1 1 1608 0 3.1mb 1.5mb green open ori_aalsmeer_20190531095143 lXj4xaKRTpScM04XtobHyA 1 1 1504 4 13.8mb 6.9mb green open ori_amsterdam_nieuw-west_20190531095137 pgcCYqpbTaG19tUtQaUDLA 1 1 1387 622 17.6mb 8.8mb green open ori_haarlemmermeer_20190531095155 jFhrSOunT5-nkQ4AwYj3ZA 1 1 2710 1141 16.6mb 8.3mb green open ori_dongen_20190531095205 K7ApENoQTTq2L6q4FyqRQA 1 1 1514 410 25.4mb 12.7mb green open ori_medemblik_20190531095216 L4vNXCOGSKC9SGNEty5tbg 1 1 929 4 6mb 2.1mb green open ori_de_wolden_20190531095202 r9VrcnFYQiKnERJFqAdnrA 1 1 830 33 9.1mb 4.5mb green open ori_noordwijk_20190531095224 buqMh5O7ROaISO2Rkr8TzA 1 1 33 32 71.8kb 35.9kb green open ori_soest_20190531095143 oKevO9TDRsGd3FbW7FxkvA 1 1 231 91 1.9mb 1009.1kb green open ori_maassluis_20190531095149 S62wD4_mQLOxgy-E_QlG1g 1 1 3030 757 42.3mb 21.1mb green open ori_teylingen_20190531095227 kRnKw9B5QsSGpYaLpsxIDg 1 1 2207 180 24.3mb 12.1mb green open ori_wassenaar_20190531095137 oClPcGZVQ8eAPo0wjTu0vg 1 1 11 3 103.6kb 51.8kb green open ori_nieuwegein_20190531095147 DlOgP_kRS62_8Gi0VB2lAg 1 1 1175 237 5.4mb 2.6mb green open ori_boxmeer_20190531095211 otxmP0zVQkOg03seWj1F0Q 1 1 593 136 2.2mb 1.1mb green open ori_leiden_20190531095215 jGcIjWfoSpKOHcNVre5Vhg 1 1 2790 939 48.6mb 24.3mb green open ori_roosendaal_20190531095208 K9SmxoY9ShWNsyR2sO-Nvw 1 1 2436 606 3.2mb 1.6mb green open ori_nieuwkoop_20190531095201 tH4b7y6gSUykbcI6VkI3dA 1 1 1679 0 25mb 10.7mb green open ori_deventer_20190531095221 Uf62HeynQxiguKdH7ewiQw 1 1 2363 323 31.9mb 15.9mb green open ori_oude_ijsselstreek_20190531095225 _cU2SDpNQrKWu0kGSpJDOQ 1 1 327 158 799.4kb 399.7kb green open osi_limburg_20190531095329 YKV1qqfnQ62IZJZap-JGLg 1 1 1 0 26.2kb 13.1kb green open ori_culemborg_20190531095148 5OqQpD2OTyGjNW3Vjuvv5A 1 1 562 83 3.9mb 2.4mb green open ori_losser_20190531095209 SC-SW5rLRlOyxVMPFYkRvQ 1 1 496 125 1.5mb 772.8kb green open ori_rheden_20190531095139 PVWZC5p7QSqoRPKAgRD6GA 1 1 2209 728 24.4mb 12.2mb green open ori_noordoostpolder_20190531095221 wyy0xm0WTYWZYiohsAVAMg 1 1 376 43 450.8kb 274.9kb green open ori_zoetermeer_20190531095223 4-z5vP6LQ9qVtiTl23eaaQ 1 1 2491 1 6mb 3mb green open ori_buren_20190531095225 _9AGyqhqTXWC8W_CwK3ofA 1 1 1101 154 1.9mb 1022.4kb green open ori_landsmeer_20190531095152 RZW4WEYRTreFROBsmQncAQ 1 1 1762 585 16.7mb 8.3mb green open ori_den_helder_20190531095229 1VHPSquET8KW8whbdfAGlQ 1 1 2809 1160 4.6mb 2.3mb green open ori_edam-volendam_20190531095220 Qgq9RBkBTKWb7ccOEs9-Rw 1 1 540 4 3.8mb 1.5mb green open osi_zuid-holland_20190531095331 ob_NiWQgSSyh6pAkYyMDBA 1 1 180 0 146.7kb 73.3kb green open ori_lingewaard_20190531095215 pg1cYaNKS2mEfe_RV0eFAA 1 1 3587 376 30.5mb 15.2mb green open ori_utrecht_20190531095223 2IseCtMoT6C_nvy9hKiJBw 1 1 2674 9182 16.7mb 8.4mb green open ori_zeist_20190531095154 mjWRrqjkRqumlbB4rsdFgg 1 1 2762 452 40.4mb 20.2mb green open ori_rijswijk_20190531095210 R6c1zWFhQNC7-Plrs1d5Nw 1 1 1668 792 10.2mb 5.1mb green open ori_meierijstad_20190531095141 nHDht32FTKOvWvlGvnbkJw 1 1 3202 1179 46.7mb 23.3mb green open ori_alkmaar_20190531095208 HL1OiK7TR5i-sgcwOhg7ew 1 1 2778 608 43.2mb 21.6mb green open ori_amsterdam_zuidoost_20190531095135 87T-T0VPTfuEjhfQq5l13A 1 1 834 78 8.2mb 4.1mb green open ori_roermond_20190531095203 M2TcYjKwSFiAYBWyL4fpGw 1 1 10 2 61.9kb 41.6kb green open ori_amsterdam_noord_20190531095142 kYHLAlIoRdeEXzEMcwQpcQ 1 1 1089 0 7.6mb 3.7mb green open ori_hulst_20190531095133 VzHy-x45T_mv7dZMkXrhGA 1 1 1832 0 18.2mb 9.1mb green open ori_steenwijkerland_20190531095138 NAEBbjK7R0S_oIkSNJe6pg 1 1 2656 946 25mb 12.6mb green open ori_doetinchem_20190531095202 Lxir7fdiT-SaB8lE093cQg 1 1 702 37 831.8kb 415.9kb green open ori_rhenen_20190531095218 HBrf7mvaS46Z9iUYfFLs-g 1 1 250 91 3mb 1.5mb green open ori_staphorst_20190531095216 wfwGyTc-R2uF40TbfWnBKg 1 1 562 228 5.2mb 2.6mb green open ori_houten_20190531095220 Ff2cURSnS822CDVOkOmfKw 1 1 831 3 2.8mb 1.3mb green open ori_voorst_20190531095203 nLYKARoMR5moUztzCDsbwQ 1 1 1252 161 7.4mb 3.7mb green open ori_capelle_ad_ijssel_20190531095210 t5324CDSQr-074U0bkmvCw 1 1 2322 837 55.4mb 27.7mb green open ori_den_haag_20190531095142 UtAMAibzS7m6m6VdKBKRIw 1 1 1334 253 2.6mb 1.3mb green open ori_epe_20190531095225 Hk6IGCkTRoSQ7ic0G10PmA 1 1 118 0 1mb 537.7kb green open ori_overbetuwe_20190531095202 mvQobC26SKaDlj_yGTbg9w 1 1 2466 213 11.3mb 5.6mb green open ori_amsterdam_west_20190531095134 gQo38heoTjW_VXlKurRtMg 1 1 187 13 1.2mb 646.2kb green open ori_oldambt_20190531095147 QLItkMC4TUqAZwInAF8r3w 1 1 1371 333 5.1mb 2.6mb green open ori_hollands_kroon_20190531095226 V6swg0WmTQOwi8yy4KmRZA 1 1 452 0 4.5mb 2.2mb green open ori_amsterdam_zuid_20190531095219 l13-qkF9QdeXXmMp5fbABA 1 1 45 0 337.1kb 82.1kb green open ori_groningen_20190531095157 VAnQCvjQT5WJQMxINj1byQ 1 1 2439 91 16.6mb 8.3mb green open ori_amsterdam_20190531095206 2bZTtsNpQ4enIu1hGLh4vw 1 1 5379 1787 74.3mb 37.1mb green open ori_baarn_20190531095213 xnpQQcugTHW5TjisRWkGNA 1 1 122 3 2.7mb 1.3mb green open ori_veenendaal_20190531095155 w1P_ZHt3QQyH4jx0ppC9Iw 1 1 2690 1251 58.6mb 29.3mb green open ori_schagen_20190531095228 kzrlpt9LTVaaD098oWi4Gw 1 1 170 0 2.9mb 1.4mb green open ori_loon_op_zand_20190531095158 6HFGJMgZSJS1xWAqxie2FA 1 1 447 0 4mb 2mb green open ori_oosterhout_20190531095152 2ONh41SdTaiajkp0wXHkRQ 1 1 9 0 40.2kb 20.1kb green open resolver Nm7lAEDrQQuFZsOULwAoCA 1 1 66910 26745 63.7mb 31.7mb green open ori_vlaardingen_20190531095135 BhdgBuOhQf6fm3toXoQCuA 1 1 1726 197 16mb 8mb green open ori_borne_20190531095226 Pb2l_hXmQAKtG59Zv3CFRA 1 1 205 29 694kb 347kb ```
jurrian commented 5 years ago
V0 all Agenda's per month ``` { "took": 59, "timed_out": false, "_shards": { "total": 134, "successful": 134, "failed": 0 }, "hits": { "total": 48579, "max_score": 0.0, "hits": [] }, "aggregations": { "maanden": { "buckets": [{ "key_as_string": "2000-09-01T00:00:00.000Z", "key": 967766400000, "doc_count": 2 }, { "key_as_string": "2000-10-01T00:00:00.000Z", "key": 970358400000, "doc_count": 2 }, { "key_as_string": "2000-11-01T00:00:00.000Z", "key": 973036800000, "doc_count": 2 }, { "key_as_string": "2000-12-01T00:00:00.000Z", "key": 975628800000, "doc_count": 13 }, { "key_as_string": "2001-01-01T00:00:00.000Z", "key": 978307200000, "doc_count": 12 }, { "key_as_string": "2001-02-01T00:00:00.000Z", "key": 980985600000, "doc_count": 12 }, { "key_as_string": "2001-03-01T00:00:00.000Z", "key": 983404800000, "doc_count": 12 }, { "key_as_string": "2001-04-01T00:00:00.000Z", "key": 986083200000, "doc_count": 14 }, { "key_as_string": "2001-05-01T00:00:00.000Z", "key": 988675200000, "doc_count": 12 }, { "key_as_string": "2001-06-01T00:00:00.000Z", "key": 991353600000, "doc_count": 14 }, { "key_as_string": "2001-07-01T00:00:00.000Z", "key": 993945600000, "doc_count": 12 }, { "key_as_string": "2001-08-01T00:00:00.000Z", "key": 996624000000, "doc_count": 0 }, { "key_as_string": "2001-09-01T00:00:00.000Z", "key": 999302400000, "doc_count": 12 }, { "key_as_string": "2001-10-01T00:00:00.000Z", "key": 1001894400000, "doc_count": 17 }, { "key_as_string": "2001-11-01T00:00:00.000Z", "key": 1004572800000, "doc_count": 15 }, { "key_as_string": "2001-12-01T00:00:00.000Z", "key": 1007164800000, "doc_count": 14 }, { "key_as_string": "2002-01-01T00:00:00.000Z", "key": 1009843200000, "doc_count": 11 }, { "key_as_string": "2002-02-01T00:00:00.000Z", "key": 1012521600000, "doc_count": 12 }, { "key_as_string": "2002-03-01T00:00:00.000Z", "key": 1014940800000, "doc_count": 15 }, { "key_as_string": "2002-04-01T00:00:00.000Z", "key": 1017619200000, "doc_count": 13 }, { "key_as_string": "2002-05-01T00:00:00.000Z", "key": 1020211200000, "doc_count": 11 }, { "key_as_string": "2002-06-01T00:00:00.000Z", "key": 1022889600000, "doc_count": 17 }, { "key_as_string": "2002-07-01T00:00:00.000Z", "key": 1025481600000, "doc_count": 0 }, { "key_as_string": "2002-08-01T00:00:00.000Z", "key": 1028160000000, "doc_count": 2 }, { "key_as_string": "2002-09-01T00:00:00.000Z", "key": 1030838400000, "doc_count": 11 }, { "key_as_string": "2002-10-01T00:00:00.000Z", "key": 1033430400000, "doc_count": 15 }, { "key_as_string": "2002-11-01T00:00:00.000Z", "key": 1036108800000, "doc_count": 15 }, { "key_as_string": "2002-12-01T00:00:00.000Z", "key": 1038700800000, "doc_count": 14 }, { "key_as_string": "2003-01-01T00:00:00.000Z", "key": 1041379200000, "doc_count": 14 }, { "key_as_string": "2003-02-01T00:00:00.000Z", "key": 1044057600000, "doc_count": 12 }, { "key_as_string": "2003-03-01T00:00:00.000Z", "key": 1046476800000, "doc_count": 8 }, { "key_as_string": "2003-04-01T00:00:00.000Z", "key": 1049155200000, "doc_count": 10 }, { "key_as_string": "2003-05-01T00:00:00.000Z", "key": 1051747200000, "doc_count": 13 }, { "key_as_string": "2003-06-01T00:00:00.000Z", "key": 1054425600000, "doc_count": 10 }, { "key_as_string": "2003-07-01T00:00:00.000Z", "key": 1057017600000, "doc_count": 0 }, { "key_as_string": "2003-08-01T00:00:00.000Z", "key": 1059696000000, "doc_count": 6 }, { "key_as_string": "2003-09-01T00:00:00.000Z", "key": 1062374400000, "doc_count": 12 }, { "key_as_string": "2003-10-01T00:00:00.000Z", "key": 1064966400000, "doc_count": 14 }, { "key_as_string": "2003-11-01T00:00:00.000Z", "key": 1067644800000, "doc_count": 13 }, { "key_as_string": "2003-12-01T00:00:00.000Z", "key": 1070236800000, "doc_count": 13 }, { "key_as_string": "2004-01-01T00:00:00.000Z", "key": 1072915200000, "doc_count": 15 }, { "key_as_string": "2004-02-01T00:00:00.000Z", "key": 1075593600000, "doc_count": 15 }, { "key_as_string": "2004-03-01T00:00:00.000Z", "key": 1078099200000, "doc_count": 17 }, { "key_as_string": "2004-04-01T00:00:00.000Z", "key": 1080777600000, "doc_count": 15 }, { "key_as_string": "2004-05-01T00:00:00.000Z", "key": 1083369600000, "doc_count": 25 }, { "key_as_string": "2004-06-01T00:00:00.000Z", "key": 1086048000000, "doc_count": 14 }, { "key_as_string": "2004-07-01T00:00:00.000Z", "key": 1088640000000, "doc_count": 6 }, { "key_as_string": "2004-08-01T00:00:00.000Z", "key": 1091318400000, "doc_count": 13 }, { "key_as_string": "2004-09-01T00:00:00.000Z", "key": 1093996800000, "doc_count": 15 }, { "key_as_string": "2004-10-01T00:00:00.000Z", "key": 1096588800000, "doc_count": 20 }, { "key_as_string": "2004-11-01T00:00:00.000Z", "key": 1099267200000, "doc_count": 16 }, { "key_as_string": "2004-12-01T00:00:00.000Z", "key": 1101859200000, "doc_count": 16 }, { "key_as_string": "2005-01-01T00:00:00.000Z", "key": 1104537600000, "doc_count": 12 }, { "key_as_string": "2005-02-01T00:00:00.000Z", "key": 1107216000000, "doc_count": 12 }, { "key_as_string": "2005-03-01T00:00:00.000Z", "key": 1109635200000, "doc_count": 13 }, { "key_as_string": "2005-04-01T00:00:00.000Z", "key": 1112313600000, "doc_count": 13 }, { "key_as_string": "2005-05-01T00:00:00.000Z", "key": 1114905600000, "doc_count": 23 }, { "key_as_string": "2005-06-01T00:00:00.000Z", "key": 1117584000000, "doc_count": 20 }, { "key_as_string": "2005-07-01T00:00:00.000Z", "key": 1120176000000, "doc_count": 13 }, { "key_as_string": "2005-08-01T00:00:00.000Z", "key": 1122854400000, "doc_count": 7 }, { "key_as_string": "2005-09-01T00:00:00.000Z", "key": 1125532800000, "doc_count": 19 }, { "key_as_string": "2005-10-01T00:00:00.000Z", "key": 1128124800000, "doc_count": 22 }, { "key_as_string": "2005-11-01T00:00:00.000Z", "key": 1130803200000, "doc_count": 20 }, { "key_as_string": "2005-12-01T00:00:00.000Z", "key": 1133395200000, "doc_count": 20 }, { "key_as_string": "2006-01-01T00:00:00.000Z", "key": 1136073600000, "doc_count": 25 }, { "key_as_string": "2006-02-01T00:00:00.000Z", "key": 1138752000000, "doc_count": 15 }, { "key_as_string": "2006-03-01T00:00:00.000Z", "key": 1141171200000, "doc_count": 14 }, { "key_as_string": "2006-04-01T00:00:00.000Z", "key": 1143849600000, "doc_count": 11 }, { "key_as_string": "2006-05-01T00:00:00.000Z", "key": 1146441600000, "doc_count": 15 }, { "key_as_string": "2006-06-01T00:00:00.000Z", "key": 1149120000000, "doc_count": 16 }, { "key_as_string": "2006-07-01T00:00:00.000Z", "key": 1151712000000, "doc_count": 5 }, { "key_as_string": "2006-08-01T00:00:00.000Z", "key": 1154390400000, "doc_count": 2 }, { "key_as_string": "2006-09-01T00:00:00.000Z", "key": 1157068800000, "doc_count": 10 }, { "key_as_string": "2006-10-01T00:00:00.000Z", "key": 1159660800000, "doc_count": 15 }, { "key_as_string": "2006-11-01T00:00:00.000Z", "key": 1162339200000, "doc_count": 15 }, { "key_as_string": "2006-12-01T00:00:00.000Z", "key": 1164931200000, "doc_count": 13 }, { "key_as_string": "2007-01-01T00:00:00.000Z", "key": 1167609600000, "doc_count": 21 }, { "key_as_string": "2007-02-01T00:00:00.000Z", "key": 1170288000000, "doc_count": 21 }, { "key_as_string": "2007-03-01T00:00:00.000Z", "key": 1172707200000, "doc_count": 7 }, { "key_as_string": "2007-04-01T00:00:00.000Z", "key": 1175385600000, "doc_count": 10 }, { "key_as_string": "2007-05-01T00:00:00.000Z", "key": 1177977600000, "doc_count": 21 }, { "key_as_string": "2007-06-01T00:00:00.000Z", "key": 1180656000000, "doc_count": 25 }, { "key_as_string": "2007-07-01T00:00:00.000Z", "key": 1183248000000, "doc_count": 8 }, { "key_as_string": "2007-08-01T00:00:00.000Z", "key": 1185926400000, "doc_count": 1 }, { "key_as_string": "2007-09-01T00:00:00.000Z", "key": 1188604800000, "doc_count": 24 }, { "key_as_string": "2007-10-01T00:00:00.000Z", "key": 1191196800000, "doc_count": 23 }, { "key_as_string": "2007-11-01T00:00:00.000Z", "key": 1193875200000, "doc_count": 22 }, { "key_as_string": "2007-12-01T00:00:00.000Z", "key": 1196467200000, "doc_count": 18 }, { "key_as_string": "2008-01-01T00:00:00.000Z", "key": 1199145600000, "doc_count": 31 }, { "key_as_string": "2008-02-01T00:00:00.000Z", "key": 1201824000000, "doc_count": 18 }, { "key_as_string": "2008-03-01T00:00:00.000Z", "key": 1204329600000, "doc_count": 23 }, { "key_as_string": "2008-04-01T00:00:00.000Z", "key": 1207008000000, "doc_count": 23 }, { "key_as_string": "2008-05-01T00:00:00.000Z", "key": 1209600000000, "doc_count": 28 }, { "key_as_string": "2008-06-01T00:00:00.000Z", "key": 1212278400000, "doc_count": 32 }, { "key_as_string": "2008-07-01T00:00:00.000Z", "key": 1214870400000, "doc_count": 0 }, { "key_as_string": "2008-08-01T00:00:00.000Z", "key": 1217548800000, "doc_count": 1 }, { "key_as_string": "2008-09-01T00:00:00.000Z", "key": 1220227200000, "doc_count": 26 }, { "key_as_string": "2008-10-01T00:00:00.000Z", "key": 1222819200000, "doc_count": 28 }, { "key_as_string": "2008-11-01T00:00:00.000Z", "key": 1225497600000, "doc_count": 24 }, { "key_as_string": "2008-12-01T00:00:00.000Z", "key": 1228089600000, "doc_count": 20 }, { "key_as_string": "2009-01-01T00:00:00.000Z", "key": 1230768000000, "doc_count": 24 }, { "key_as_string": "2009-02-01T00:00:00.000Z", "key": 1233446400000, "doc_count": 23 }, { "key_as_string": "2009-03-01T00:00:00.000Z", "key": 1235865600000, "doc_count": 29 }, { "key_as_string": "2009-04-01T00:00:00.000Z", "key": 1238544000000, "doc_count": 28 }, { "key_as_string": "2009-05-01T00:00:00.000Z", "key": 1241136000000, "doc_count": 33 }, { "key_as_string": "2009-06-01T00:00:00.000Z", "key": 1243814400000, "doc_count": 29 }, { "key_as_string": "2009-07-01T00:00:00.000Z", "key": 1246406400000, "doc_count": 4 }, { "key_as_string": "2009-08-01T00:00:00.000Z", "key": 1249084800000, "doc_count": 5 }, { "key_as_string": "2009-09-01T00:00:00.000Z", "key": 1251763200000, "doc_count": 32 }, { "key_as_string": "2009-10-01T00:00:00.000Z", "key": 1254355200000, "doc_count": 34 }, { "key_as_string": "2009-11-01T00:00:00.000Z", "key": 1257033600000, "doc_count": 31 }, { "key_as_string": "2009-12-01T00:00:00.000Z", "key": 1259625600000, "doc_count": 27 }, { "key_as_string": "2010-01-01T00:00:00.000Z", "key": 1262304000000, "doc_count": 42 }, { "key_as_string": "2010-02-01T00:00:00.000Z", "key": 1264982400000, "doc_count": 22 }, { "key_as_string": "2010-03-01T00:00:00.000Z", "key": 1267401600000, "doc_count": 34 }, { "key_as_string": "2010-04-01T00:00:00.000Z", "key": 1270080000000, "doc_count": 34 }, { "key_as_string": "2010-05-01T00:00:00.000Z", "key": 1272672000000, "doc_count": 26 }, { "key_as_string": "2010-06-01T00:00:00.000Z", "key": 1275350400000, "doc_count": 38 }, { "key_as_string": "2010-07-01T00:00:00.000Z", "key": 1277942400000, "doc_count": 8 }, { "key_as_string": "2010-08-01T00:00:00.000Z", "key": 1280620800000, "doc_count": 5 }, { "key_as_string": "2010-09-01T00:00:00.000Z", "key": 1283299200000, "doc_count": 41 }, { "key_as_string": "2010-10-01T00:00:00.000Z", "key": 1285891200000, "doc_count": 24 }, { "key_as_string": "2010-11-01T00:00:00.000Z", "key": 1288569600000, "doc_count": 30 }, { "key_as_string": "2010-12-01T00:00:00.000Z", "key": 1291161600000, "doc_count": 19 }, { "key_as_string": "2011-01-01T00:00:00.000Z", "key": 1293840000000, "doc_count": 43 }, { "key_as_string": "2011-02-01T00:00:00.000Z", "key": 1296518400000, "doc_count": 29 }, { "key_as_string": "2011-03-01T00:00:00.000Z", "key": 1298937600000, "doc_count": 30 }, { "key_as_string": "2011-04-01T00:00:00.000Z", "key": 1301616000000, "doc_count": 30 }, { "key_as_string": "2011-05-01T00:00:00.000Z", "key": 1304208000000, "doc_count": 35 }, { "key_as_string": "2011-06-01T00:00:00.000Z", "key": 1306886400000, "doc_count": 46 }, { "key_as_string": "2011-07-01T00:00:00.000Z", "key": 1309478400000, "doc_count": 11 }, { "key_as_string": "2011-08-01T00:00:00.000Z", "key": 1312156800000, "doc_count": 10 }, { "key_as_string": "2011-09-01T00:00:00.000Z", "key": 1314835200000, "doc_count": 41 }, { "key_as_string": "2011-10-01T00:00:00.000Z", "key": 1317427200000, "doc_count": 39 }, { "key_as_string": "2011-11-01T00:00:00.000Z", "key": 1320105600000, "doc_count": 47 }, { "key_as_string": "2011-12-01T00:00:00.000Z", "key": 1322697600000, "doc_count": 19 }, { "key_as_string": "2012-01-01T00:00:00.000Z", "key": 1325376000000, "doc_count": 48 }, { "key_as_string": "2012-02-01T00:00:00.000Z", "key": 1328054400000, "doc_count": 43 }, { "key_as_string": "2012-03-01T00:00:00.000Z", "key": 1330560000000, "doc_count": 64 }, { "key_as_string": "2012-04-01T00:00:00.000Z", "key": 1333238400000, "doc_count": 52 }, { "key_as_string": "2012-05-01T00:00:00.000Z", "key": 1335830400000, "doc_count": 61 }, { "key_as_string": "2012-06-01T00:00:00.000Z", "key": 1338508800000, "doc_count": 80 }, { "key_as_string": "2012-07-01T00:00:00.000Z", "key": 1341100800000, "doc_count": 23 }, { "key_as_string": "2012-08-01T00:00:00.000Z", "key": 1343779200000, "doc_count": 8 }, { "key_as_string": "2012-09-01T00:00:00.000Z", "key": 1346457600000, "doc_count": 80 }, { "key_as_string": "2012-10-01T00:00:00.000Z", "key": 1349049600000, "doc_count": 91 }, { "key_as_string": "2012-11-01T00:00:00.000Z", "key": 1351728000000, "doc_count": 82 }, { "key_as_string": "2012-12-01T00:00:00.000Z", "key": 1354320000000, "doc_count": 50 }, { "key_as_string": "2013-01-01T00:00:00.000Z", "key": 1356998400000, "doc_count": 417 }, { "key_as_string": "2013-02-01T00:00:00.000Z", "key": 1359676800000, "doc_count": 351 }, { "key_as_string": "2013-03-01T00:00:00.000Z", "key": 1362096000000, "doc_count": 399 }, { "key_as_string": "2013-04-01T00:00:00.000Z", "key": 1364774400000, "doc_count": 503 }, { "key_as_string": "2013-05-01T00:00:00.000Z", "key": 1367366400000, "doc_count": 424 }, { "key_as_string": "2013-06-01T00:00:00.000Z", "key": 1370044800000, "doc_count": 503 }, { "key_as_string": "2013-07-01T00:00:00.000Z", "key": 1372636800000, "doc_count": 193 }, { "key_as_string": "2013-08-01T00:00:00.000Z", "key": 1375315200000, "doc_count": 103 }, { "key_as_string": "2013-09-01T00:00:00.000Z", "key": 1377993600000, "doc_count": 482 }, { "key_as_string": "2013-10-01T00:00:00.000Z", "key": 1380585600000, "doc_count": 543 }, { "key_as_string": "2013-11-01T00:00:00.000Z", "key": 1383264000000, "doc_count": 475 }, { "key_as_string": "2013-12-01T00:00:00.000Z", "key": 1385856000000, "doc_count": 353 }, { "key_as_string": "2014-01-01T00:00:00.000Z", "key": 1388534400000, "doc_count": 478 }, { "key_as_string": "2014-02-01T00:00:00.000Z", "key": 1391212800000, "doc_count": 453 }, { "key_as_string": "2014-03-01T00:00:00.000Z", "key": 1393632000000, "doc_count": 435 }, { "key_as_string": "2014-04-01T00:00:00.000Z", "key": 1396310400000, "doc_count": 393 }, { "key_as_string": "2014-05-01T00:00:00.000Z", "key": 1398902400000, "doc_count": 517 }, { "key_as_string": "2014-06-01T00:00:00.000Z", "key": 1401580800000, "doc_count": 601 }, { "key_as_string": "2014-07-01T00:00:00.000Z", "key": 1404172800000, "doc_count": 239 }, { "key_as_string": "2014-08-01T00:00:00.000Z", "key": 1406851200000, "doc_count": 164 }, { "key_as_string": "2014-09-01T00:00:00.000Z", "key": 1409529600000, "doc_count": 674 }, { "key_as_string": "2014-10-01T00:00:00.000Z", "key": 1412121600000, "doc_count": 672 }, { "key_as_string": "2014-11-01T00:00:00.000Z", "key": 1414800000000, "doc_count": 617 }, { "key_as_string": "2014-12-01T00:00:00.000Z", "key": 1417392000000, "doc_count": 464 }, { "key_as_string": "2015-01-01T00:00:00.000Z", "key": 1420070400000, "doc_count": 540 }, { "key_as_string": "2015-02-01T00:00:00.000Z", "key": 1422748800000, "doc_count": 443 }, { "key_as_string": "2015-03-01T00:00:00.000Z", "key": 1425168000000, "doc_count": 591 }, { "key_as_string": "2015-04-01T00:00:00.000Z", "key": 1427846400000, "doc_count": 630 }, { "key_as_string": "2015-05-01T00:00:00.000Z", "key": 1430438400000, "doc_count": 488 }, { "key_as_string": "2015-06-01T00:00:00.000Z", "key": 1433116800000, "doc_count": 713 }, { "key_as_string": "2015-07-01T00:00:00.000Z", "key": 1435708800000, "doc_count": 262 }, { "key_as_string": "2015-08-01T00:00:00.000Z", "key": 1438387200000, "doc_count": 177 }, { "key_as_string": "2015-09-01T00:00:00.000Z", "key": 1441065600000, "doc_count": 692 }, { "key_as_string": "2015-10-01T00:00:00.000Z", "key": 1443657600000, "doc_count": 714 }, { "key_as_string": "2015-11-01T00:00:00.000Z", "key": 1446336000000, "doc_count": 670 }, { "key_as_string": "2015-12-01T00:00:00.000Z", "key": 1448928000000, "doc_count": 482 }, { "key_as_string": "2016-01-01T00:00:00.000Z", "key": 1451606400000, "doc_count": 556 }, { "key_as_string": "2016-02-01T00:00:00.000Z", "key": 1454284800000, "doc_count": 528 }, { "key_as_string": "2016-03-01T00:00:00.000Z", "key": 1456790400000, "doc_count": 599 }, { "key_as_string": "2016-04-01T00:00:00.000Z", "key": 1459468800000, "doc_count": 638 }, { "key_as_string": "2016-05-01T00:00:00.000Z", "key": 1462060800000, "doc_count": 563 }, { "key_as_string": "2016-06-01T00:00:00.000Z", "key": 1464739200000, "doc_count": 722 }, { "key_as_string": "2016-07-01T00:00:00.000Z", "key": 1467331200000, "doc_count": 289 }, { "key_as_string": "2016-08-01T00:00:00.000Z", "key": 1470009600000, "doc_count": 172 }, { "key_as_string": "2016-09-01T00:00:00.000Z", "key": 1472688000000, "doc_count": 784 }, { "key_as_string": "2016-10-01T00:00:00.000Z", "key": 1475280000000, "doc_count": 749 }, { "key_as_string": "2016-11-01T00:00:00.000Z", "key": 1477958400000, "doc_count": 810 }, { "key_as_string": "2016-12-01T00:00:00.000Z", "key": 1480550400000, "doc_count": 498 }, { "key_as_string": "2017-01-01T00:00:00.000Z", "key": 1483228800000, "doc_count": 725 }, { "key_as_string": "2017-02-01T00:00:00.000Z", "key": 1485907200000, "doc_count": 695 }, { "key_as_string": "2017-03-01T00:00:00.000Z", "key": 1488326400000, "doc_count": 838 }, { "key_as_string": "2017-04-01T00:00:00.000Z", "key": 1491004800000, "doc_count": 714 }, { "key_as_string": "2017-05-01T00:00:00.000Z", "key": 1493596800000, "doc_count": 811 }, { "key_as_string": "2017-06-01T00:00:00.000Z", "key": 1496275200000, "doc_count": 851 }, { "key_as_string": "2017-07-01T00:00:00.000Z", "key": 1498867200000, "doc_count": 484 }, { "key_as_string": "2017-08-01T00:00:00.000Z", "key": 1501545600000, "doc_count": 195 }, { "key_as_string": "2017-09-01T00:00:00.000Z", "key": 1504224000000, "doc_count": 849 }, { "key_as_string": "2017-10-01T00:00:00.000Z", "key": 1506816000000, "doc_count": 834 }, { "key_as_string": "2017-11-01T00:00:00.000Z", "key": 1509494400000, "doc_count": 977 }, { "key_as_string": "2017-12-01T00:00:00.000Z", "key": 1512086400000, "doc_count": 626 }, { "key_as_string": "2018-01-01T00:00:00.000Z", "key": 1514764800000, "doc_count": 844 }, { "key_as_string": "2018-02-01T00:00:00.000Z", "key": 1517443200000, "doc_count": 889 }, { "key_as_string": "2018-03-01T00:00:00.000Z", "key": 1519862400000, "doc_count": 961 }, { "key_as_string": "2018-04-01T00:00:00.000Z", "key": 1522540800000, "doc_count": 762 }, { "key_as_string": "2018-05-01T00:00:00.000Z", "key": 1525132800000, "doc_count": 826 }, { "key_as_string": "2018-06-01T00:00:00.000Z", "key": 1527811200000, "doc_count": 932 }, { "key_as_string": "2018-07-01T00:00:00.000Z", "key": 1530403200000, "doc_count": 456 }, { "key_as_string": "2018-08-01T00:00:00.000Z", "key": 1533081600000, "doc_count": 210 }, { "key_as_string": "2018-09-01T00:00:00.000Z", "key": 1535760000000, "doc_count": 982 }, { "key_as_string": "2018-10-01T00:00:00.000Z", "key": 1538352000000, "doc_count": 1095 }, { "key_as_string": "2018-11-01T00:00:00.000Z", "key": 1541030400000, "doc_count": 1199 }, { "key_as_string": "2018-12-01T00:00:00.000Z", "key": 1543622400000, "doc_count": 403 }, { "key_as_string": "2019-01-01T00:00:00.000Z", "key": 1546300800000, "doc_count": 771 }, { "key_as_string": "2019-02-01T00:00:00.000Z", "key": 1548979200000, "doc_count": 734 }, { "key_as_string": "2019-03-01T00:00:00.000Z", "key": 1551398400000, "doc_count": 958 }, { "key_as_string": "2019-04-01T00:00:00.000Z", "key": 1554076800000, "doc_count": 683 }, { "key_as_string": "2019-05-01T00:00:00.000Z", "key": 1556668800000, "doc_count": 151 }, { "key_as_string": "2019-06-01T00:00:00.000Z", "key": 1559347200000, "doc_count": 257 }, { "key_as_string": "2019-07-01T00:00:00.000Z", "key": 1561939200000, "doc_count": 1 }, { "key_as_string": "2019-08-01T00:00:00.000Z", "key": 1564617600000, "doc_count": 0 }, { "key_as_string": "2019-09-01T00:00:00.000Z", "key": 1567296000000, "doc_count": 0 }, { "key_as_string": "2019-10-01T00:00:00.000Z", "key": 1569888000000, "doc_count": 0 }, { "key_as_string": "2019-11-01T00:00:00.000Z", "key": 1572566400000, "doc_count": 0 }, { "key_as_string": "2019-12-01T00:00:00.000Z", "key": 1575158400000, "doc_count": 0 }, { "key_as_string": "2020-01-01T00:00:00.000Z", "key": 1577836800000, "doc_count": 1 }] } } } ```
V1 all meetings per month ``` { "took": 332, "timed_out": false, "_shards": { "total": 123, "successful": 123, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 7822, "relation": "eq" }, "max_score": null, "hits": [] }, "aggregations": { "maanden": { "buckets": [ { "key_as_string": "2017-12-01T00:00:00.000Z", "key": 1512086400000, "doc_count": 3 }, { "key_as_string": "2018-01-01T00:00:00.000Z", "key": 1514764800000, "doc_count": 483 }, { "key_as_string": "2018-02-01T00:00:00.000Z", "key": 1517443200000, "doc_count": 481 }, { "key_as_string": "2018-03-01T00:00:00.000Z", "key": 1519862400000, "doc_count": 580 }, { "key_as_string": "2018-04-01T00:00:00.000Z", "key": 1522540800000, "doc_count": 386 }, { "key_as_string": "2018-05-01T00:00:00.000Z", "key": 1525132800000, "doc_count": 457 }, { "key_as_string": "2018-06-01T00:00:00.000Z", "key": 1527811200000, "doc_count": 504 }, { "key_as_string": "2018-07-01T00:00:00.000Z", "key": 1530403200000, "doc_count": 229 }, { "key_as_string": "2018-08-01T00:00:00.000Z", "key": 1533081600000, "doc_count": 136 }, { "key_as_string": "2018-09-01T00:00:00.000Z", "key": 1535760000000, "doc_count": 474 }, { "key_as_string": "2018-10-01T00:00:00.000Z", "key": 1538352000000, "doc_count": 518 }, { "key_as_string": "2018-11-01T00:00:00.000Z", "key": 1541030400000, "doc_count": 506 }, { "key_as_string": "2018-12-01T00:00:00.000Z", "key": 1543622400000, "doc_count": 345 }, { "key_as_string": "2019-01-01T00:00:00.000Z", "key": 1546300800000, "doc_count": 383 }, { "key_as_string": "2019-02-01T00:00:00.000Z", "key": 1548979200000, "doc_count": 338 }, { "key_as_string": "2019-03-01T00:00:00.000Z", "key": 1551398400000, "doc_count": 351 }, { "key_as_string": "2019-04-01T00:00:00.000Z", "key": 1554076800000, "doc_count": 369 }, { "key_as_string": "2019-05-01T00:00:00.000Z", "key": 1556668800000, "doc_count": 715 }, { "key_as_string": "2019-06-01T00:00:00.000Z", "key": 1559347200000, "doc_count": 564 } ] } } } ```
jurrian commented 5 years ago

I've used Zwolle oct 2018 meetings as test case to see which data in V1 is still missing. I tried extracting only Zwolle in V1, but without result. However, in local development I have managed to extract the 36 meetings that were missing, so it must be possible in production too. Most likely there is a problem in the ETL on production causing gaps like these. Hopefully fixes in #147 might help to prevent these gaps.

joepio commented 5 years ago

This comparison should be repeated when Neo4j has been replaced by Postgres (#175)

joepio commented 5 years ago

The new import has started. When it has finished, we will continue working on this comparison.

jurrian commented 5 years ago

Current results for V1:

V1 all meetings per month ``` { "took": 1501, "timed_out": false, "_shards": { "total": 125, "successful": 125, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 4286, "relation": "eq" }, "max_score": null, "hits": [] }, "aggregations": { "maanden": { "buckets": [ { "key_as_string": "2018-01-01T00:00:00.000Z", "key": 1514764800000, "doc_count": 32 }, { "key_as_string": "2018-02-01T00:00:00.000Z", "key": 1517443200000, "doc_count": 23 }, { "key_as_string": "2018-03-01T00:00:00.000Z", "key": 1519862400000, "doc_count": 16 }, { "key_as_string": "2018-04-01T00:00:00.000Z", "key": 1522540800000, "doc_count": 4 }, { "key_as_string": "2018-05-01T00:00:00.000Z", "key": 1525132800000, "doc_count": 12 }, { "key_as_string": "2018-06-01T00:00:00.000Z", "key": 1527811200000, "doc_count": 36 }, { "key_as_string": "2018-07-01T00:00:00.000Z", "key": 1530403200000, "doc_count": 8 }, { "key_as_string": "2018-08-01T00:00:00.000Z", "key": 1533081600000, "doc_count": 5 }, { "key_as_string": "2018-09-01T00:00:00.000Z", "key": 1535760000000, "doc_count": 31 }, { "key_as_string": "2018-10-01T00:00:00.000Z", "key": 1538352000000, "doc_count": 40 }, { "key_as_string": "2018-11-01T00:00:00.000Z", "key": 1541030400000, "doc_count": 18 }, { "key_as_string": "2018-12-01T00:00:00.000Z", "key": 1543622400000, "doc_count": 8 }, { "key_as_string": "2019-01-01T00:00:00.000Z", "key": 1546300800000, "doc_count": 17 }, { "key_as_string": "2019-02-01T00:00:00.000Z", "key": 1548979200000, "doc_count": 505 }, { "key_as_string": "2019-03-01T00:00:00.000Z", "key": 1551398400000, "doc_count": 414 }, { "key_as_string": "2019-04-01T00:00:00.000Z", "key": 1554076800000, "doc_count": 410 }, { "key_as_string": "2019-05-01T00:00:00.000Z", "key": 1556668800000, "doc_count": 542 }, { "key_as_string": "2019-06-01T00:00:00.000Z", "key": 1559347200000, "doc_count": 557 }, { "key_as_string": "2019-07-01T00:00:00.000Z", "key": 1561939200000, "doc_count": 269 }, { "key_as_string": "2019-08-01T00:00:00.000Z", "key": 1564617600000, "doc_count": 119 }, { "key_as_string": "2019-09-01T00:00:00.000Z", "key": 1567296000000, "doc_count": 367 }, { "key_as_string": "2019-10-01T00:00:00.000Z", "key": 1569888000000, "doc_count": 308 }, { "key_as_string": "2019-11-01T00:00:00.000Z", "key": 1572566400000, "doc_count": 274 }, { "key_as_string": "2019-12-01T00:00:00.000Z", "key": 1575158400000, "doc_count": 207 }, { "key_as_string": "2020-01-01T00:00:00.000Z", "key": 1577836800000, "doc_count": 37 }, { "key_as_string": "2020-02-01T00:00:00.000Z", "key": 1580515200000, "doc_count": 27 } ] } } } ```

2018 in V1 seems to be very incomplete cause it has been partially processed. Focus has been on completing 2019. V0 shows a steep decline after the first three months of 2019, this is due to the fact that V0 stopped importing new stuff. Comparing to the first months of V0 all Agenda's per month (see earlier) we see that V1 contains about half the numbers of V0. In V0 the normal minimum counts start around 600 meetings averaging 850, except for summer recess months (holidays) which average at 200.

V1 Meeting's feb 2019 ``` { "took": 678, "timed_out": false, "_shards": { "total": 125, "successful": 125, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 505, "relation": "eq" }, "max_score": null, "hits": [] }, "aggregations": { "maanden": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "ori_maastricht_20190809125059", "doc_count": 55 }, { "key": "osi_zuid-holland_20190809125059", "doc_count": 54 }, { "key": "ori_deventer_20190809125059", "doc_count": 26 }, { "key": "ori_zaanstad_20190809125043", "doc_count": 22 }, { "key": "ori_voorst_20190809125059", "doc_count": 21 }, { "key": "ori_zeist_20190809125053", "doc_count": 21 }, { "key": "ori_den_haag_20190809125112", "doc_count": 19 }, { "key": "ori_hilvarenbeek_20190809125023", "doc_count": 15 }, { "key": "ori_almere_20190809125031", "doc_count": 13 }, { "key": "ori_amsterdam_noord_20190809125133", "doc_count": 12 }, { "key": "ori_rhenen_20190809125040", "doc_count": 12 }, { "key": "osi_overijssel_20190822111458", "doc_count": 11 }, { "key": "ori_amersfoort_20190809125113", "doc_count": 10 }, { "key": "ori_amsterdam_20190809125033", "doc_count": 10 }, { "key": "ori_roosendaal_20190809125037", "doc_count": 8 }, { "key": "ori_waalwijk_20190809125138", "doc_count": 8 }, { "key": "ori_amsterdam_oost_20190809125046", "doc_count": 7 }, { "key": "ori_goirle_20190809125132", "doc_count": 7 }, { "key": "ori_maassluis_20190809125107", "doc_count": 7 }, { "key": "ori_moerdijk_20190809125042", "doc_count": 7 }, { "key": "ori_stichtse_vecht_20190809125106", "doc_count": 7 }, { "key": "ori_den_helder_20190809125127", "doc_count": 6 }, { "key": "ori_ede_20190809125045", "doc_count": 6 }, { "key": "ori_groningen_20190809125025", "doc_count": 6 }, { "key": "ori_katwijk_20190809125133", "doc_count": 6 }, { "key": "ori_krimpen_ad_ijssel_20190809125113", "doc_count": 6 }, { "key": "ori_leiden_20190809125124", "doc_count": 6 }, { "key": "osi_flevoland_20190809125113", "doc_count": 6 }, { "key": "osi_provincie-utrecht_20190809125121", "doc_count": 6 }, { "key": "ori_amsterdam_zuidoost_20190809125101", "doc_count": 5 }, { "key": "ori_doetinchem_20190809125054", "doc_count": 5 }, { "key": "ori_vlissingen_20190809125043", "doc_count": 5 }, { "key": "ori_wageningen_20190809125043", "doc_count": 5 }, { "key": "ori_zandvoort_20190809125106", "doc_count": 5 }, { "key": "ori_alkmaar_20190809125035", "doc_count": 4 }, { "key": "ori_amsterdam_zuid_20190809125107", "doc_count": 4 }, { "key": "ori_baarn_20190809125121", "doc_count": 4 }, { "key": "ori_beemster_20190809125040", "doc_count": 4 }, { "key": "ori_bloemendaal_20190809125121", "doc_count": 4 }, { "key": "ori_castricum_20190809125052", "doc_count": 4 }, { "key": "ori_dongen_20190809125121", "doc_count": 4 }, { "key": "ori_hardenberg_20190809125038", "doc_count": 4 }, { "key": "ori_oude_ijsselstreek_20190809125121", "doc_count": 4 }, { "key": "ori_schagen_20190809125124", "doc_count": 4 }, { "key": "ori_capelle_ad_ijssel_20190809125112", "doc_count": 3 }, { "key": "ori_culemborg_20190809125124", "doc_count": 3 }, { "key": "ori_hendrik-ido-ambacht_20190809125056", "doc_count": 3 }, { "key": "ori_oisterwijk_20190809125116", "doc_count": 3 }, { "key": "ori_rucphen_20190809125124", "doc_count": 3 }, { "key": "ori_staphorst_20190809125107", "doc_count": 3 }, { "key": "ori_veenendaal_20190809125132", "doc_count": 3 }, { "key": "ori_amsterdam_nieuw-west_20190809125033", "doc_count": 2 }, { "key": "ori_amsterdam_west_20190809125108", "doc_count": 2 }, { "key": "ori_borne_20190809125124", "doc_count": 2 }, { "key": "ori_leusden_20190809125117", "doc_count": 2 }, { "key": "ori_schiedam_20190809125052", "doc_count": 2 }, { "key": "ori_vlaardingen_20190809125128", "doc_count": 2 }, { "key": "ori_zwartewaterland_20190809125059", "doc_count": 2 }, { "key": "ori_de_wolden_20190809125117", "doc_count": 1 }, { "key": "ori_epe_20190809125047", "doc_count": 1 }, { "key": "ori_leiderdorp_20190809125020", "doc_count": 1 }, { "key": "ori_ommen_20190809125052", "doc_count": 1 }, { "key": "ori_oost_gelre_20190809125133", "doc_count": 1 } ] } } } ```

On 5th of June 2019 the total count for V1 Meeting's feb 2019 was 383 and has increased now to 505, however the equivalent count for V0 for that period was 771. On closer look it seems that V1 is actually has more complete municipalities (Maastricht V1: 55 vs V0: 38). However, as remarked earlier on the 5th of June, quite some municipalities are still missing! Except for Hilvarenbeek, the municipalities: zwolle, arnhem, enschede, utrecht are still not to be found in Februari. Municipality count Februari V0: 113, V1 back then: 53, V1 now: 63. This does not mean that these lacking municipalities are completely missing for all months, zwolle for example currently only shows (a few) meetings for may and june 2018.

joepio commented 5 years ago

The missing municipalities could relate to #229 , since iBabs isn't running. Are the missing municipalities iBabs municipalities @jurrian?

jurrian commented 5 years ago

It certainly looks that way, the named municipalities above are all iBabs municipalities. The lack of those on June the 5th suggests that the problem in #229 persisted longer than we thought. When this problem is fixed, I predict that V1 will be more complete than V0 for the months that have been imported.

Taluke commented 5 years ago

Hi there! We want to go live with the implementation of ORI within 1848.nl on monday. But I have a questions about the current ORI API and the further development. 1) If I search within ori.argu.co there are hardly any results for Utrecht or Provincie Utrecht. How is this possible and will this improve shortly? 2) The API currently contains 118 municipalities and 5 provinces. When will the database be complete? 3) Can you give information about what documenttypes are included? For example: collegebrieven, stemmingsuitslagen, debatverslagen, etc

Many thanks! PS: I'm new to Github so correct me if I fail to apply the etiquette :)

joepio commented 5 years ago

Hi Taluke,

  1. Municipalities using ibabs (#229), such as Utrecht, cannot be imported due to an unexpected api change from ibabs. We're working on a fix.
  2. Currently, local governments must explicitly ask for being added to ORI. Hopefully, it will become mandatory and we can add all. The Actieplan Open Overheid explicitly mentions ORI and it is is designated as a high value open dataset, so we're hopeful in that regard. Up until then, we need to convince governments to join ORI.
  3. We mostly distinguish meetings, agenda items and documents. Other kind of semantic data is pretty much impossible to standardize at this point moment.

Good luck with the launch!

Taluke commented 5 years ago

Thanks joepio :) If I can help with pushing the local governments I'll be happy to do so. I bet the iBabs fix will not be ready before monday?

Taluke commented 5 years ago

And another question: How far do the API-documents go back?

joepio commented 5 years ago

Maybe the ibabs fix will be implemented this week, but the documents still need to be imported. This will take more time. However, I'll let the devs know that the most important import task is the daily one, so your notification system will work as soon as possible!

Tell governments to get in touch with the ORI project lead from VNG, Sander Bakker. Sander.Bakker@vng.nl

We'll be importing as far back as we can! We don't know how far that is yet :).

On Thu, Sep 12, 2019, 11:27 Taluke notifications@github.com wrote:

Thanks joepio :) If I can help with pushing the local governments I'll be happy to do so. I bet the iBabs fix will not be ready before monday?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/openstate/open-raadsinformatie/issues/146?email_source=notifications&email_token=AAQVBEJITREEAUYZN66GBSDQJIKQ3A5CNFSM4HTVUDQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6RNHZA#issuecomment-530764772, or mute the thread https://github.com/notifications/unsubscribe-auth/AAQVBEJOW7ODW5OK2JMGRS3QJIKQ3ANCNFSM4HTVUDQA .

aolieman commented 5 years ago

I'm glad to hear you're on top of things @joepio!

For our work on HierOverheid we were also wondering when the v1 collection would become more complete. We'll keep working with the documents we can currently access and hope to know soon how our system will do on the as-of-yet unloaded docs.

Since we're primarily focused on the provincial docs, we currently don't have very much to work with:

green  open   osi_zuid-holland_20190809125059         llsuyALZSTqSpzFOGLv3Lg   1   1        441            0      6.1mb            3mb
green  open   osi_limburg_20190809125018              Ck9RDxk1Ty2J671pLW7tNQ   1   1         38            0     71.5kb         35.7kb
green  open   osi_flevoland_20190809125113            MLp1DKVbQteFdbg5dg2WIA   1   1         52            0    256.1kb        138.3kb
green  open   osi_noord-holland_20190809125038        PS8mYzsnTh2legb4st0CGA   1   1          2            0     49.6kb         24.8kb
green  open   osi_provincie-utrecht_20190809125121    JiGLqlFMRrCnEKmnuLFTcQ   1   1         61            0    396.6kb          200kb

Could you say a bit more about why there are so few documents in these indices?

joepio commented 5 years ago

Logs indicate that some meetings take up an absurd amount of time (hours instead of a minute, even with extreme specs and instance counts). Both @jurrian and @jorritb have spent quite some time on trying to find the underlying issue, but it turns out to be very difficult to identify the root cause. I think #224 is suspicious.

If the root cause of the performance issue is solved, we can quickly increase the document counts.

Taluke commented 4 years ago

We're live with ORI on 1848.nl! From now we'll monitor the user feedback. Most important is that we have to be complete. @joepio Can you give an update on the iBabs-bug? Plus: What actions are being taken to add more provinces and municipalities? I'm pretty sure our users will ask me :)

Taluke commented 4 years ago

This document fails to download: https://www.raadleiderdorp.nl/api/v1/meetings/1588/documents/37265

joepio commented 4 years ago

@Taluke The iBabs bug can be followed here: https://github.com/openstate/open-raadsinformatie/issues/240. Please create a new issue for the failing document (although I suspect that it is not caused by IRU, since the link is external).

[edit: correct issue]

joepio commented 4 years ago

Performance issues have been solved, imports are 10 - 100x as fast now. This means that quantity will very soon be way better :)

jurrian commented 4 years ago

Current results for V1:

V1 all meetings per month ``` { "took": 800, "timed_out": false, "_shards": { "total": 125, "successful": 125, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 8815, "relation": "eq" }, "max_score": null, "hits": [] }, "aggregations": { "maanden": { "buckets": [ { "key_as_string": "2018-01-01T00:00:00.000Z", "key": 1514764800000, "doc_count": 38 }, { "key_as_string": "2018-02-01T00:00:00.000Z", "key": 1517443200000, "doc_count": 27 }, { "key_as_string": "2018-03-01T00:00:00.000Z", "key": 1519862400000, "doc_count": 18 }, { "key_as_string": "2018-04-01T00:00:00.000Z", "key": 1522540800000, "doc_count": 8 }, { "key_as_string": "2018-05-01T00:00:00.000Z", "key": 1525132800000, "doc_count": 17 }, { "key_as_string": "2018-06-01T00:00:00.000Z", "key": 1527811200000, "doc_count": 41 }, { "key_as_string": "2018-07-01T00:00:00.000Z", "key": 1530403200000, "doc_count": 8 }, { "key_as_string": "2018-08-01T00:00:00.000Z", "key": 1533081600000, "doc_count": 5 }, { "key_as_string": "2018-09-01T00:00:00.000Z", "key": 1535760000000, "doc_count": 35 }, { "key_as_string": "2018-10-01T00:00:00.000Z", "key": 1538352000000, "doc_count": 44 }, { "key_as_string": "2018-11-01T00:00:00.000Z", "key": 1541030400000, "doc_count": 22 }, { "key_as_string": "2018-12-01T00:00:00.000Z", "key": 1543622400000, "doc_count": 12 }, { "key_as_string": "2019-01-01T00:00:00.000Z", "key": 1546300800000, "doc_count": 481 }, { "key_as_string": "2019-02-01T00:00:00.000Z", "key": 1548979200000, "doc_count": 622 }, { "key_as_string": "2019-03-01T00:00:00.000Z", "key": 1551398400000, "doc_count": 722 }, { "key_as_string": "2019-04-01T00:00:00.000Z", "key": 1554076800000, "doc_count": 844 }, { "key_as_string": "2019-05-01T00:00:00.000Z", "key": 1556668800000, "doc_count": 1010 }, { "key_as_string": "2019-06-01T00:00:00.000Z", "key": 1559347200000, "doc_count": 1079 }, { "key_as_string": "2019-07-01T00:00:00.000Z", "key": 1561939200000, "doc_count": 549 }, { "key_as_string": "2019-08-01T00:00:00.000Z", "key": 1564617600000, "doc_count": 219 }, { "key_as_string": "2019-09-01T00:00:00.000Z", "key": 1567296000000, "doc_count": 1047 }, { "key_as_string": "2019-10-01T00:00:00.000Z", "key": 1569888000000, "doc_count": 693 }, { "key_as_string": "2019-11-01T00:00:00.000Z", "key": 1572566400000, "doc_count": 525 }, { "key_as_string": "2019-12-01T00:00:00.000Z", "key": 1575158400000, "doc_count": 357 }, { "key_as_string": "2020-01-01T00:00:00.000Z", "key": 1577836800000, "doc_count": 150 }, { "key_as_string": "2020-02-01T00:00:00.000Z", "key": 1580515200000, "doc_count": 121 }, { "key_as_string": "2020-03-01T00:00:00.000Z", "key": 1583020800000, "doc_count": 121 } ] } } } ```

In my last comment the total number of meetings was 4286, this is now 8815. The last time has been about importing 2019. Comparing the first three months 2019 of V0 all Agenda's per month we see that V1 is still missing 100-300 meetings, however the months that follow show to be much more complete than V0, up to March 2020.

V1 Meeting's feb 2019 ``` { "took": 501, "timed_out": false, "_shards": { "total": 125, "successful": 125, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 622, "relation": "eq" }, "max_score": null, "hits": [] }, "aggregations": { "maanden": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "osi_zuid-holland_20190809125059", "doc_count": 58 }, { "key": "ori_maastricht_20190809125059", "doc_count": 57 }, { "key": "ori_deventer_20190809125059", "doc_count": 26 }, { "key": "ori_zwolle_20190809125022", "doc_count": 23 }, { "key": "ori_zaanstad_20190809125043", "doc_count": 22 }, { "key": "ori_voorst_20190809125059", "doc_count": 21 }, { "key": "ori_zeist_20190809125053", "doc_count": 21 }, { "key": "ori_den_haag_20190809125112", "doc_count": 19 }, { "key": "ori_hilvarenbeek_20190809125023", "doc_count": 16 }, { "key": "ori_rhenen_20190809125040", "doc_count": 14 }, { "key": "ori_almere_20190809125031", "doc_count": 13 }, { "key": "ori_amersfoort_20190809125113", "doc_count": 13 }, { "key": "ori_amsterdam_noord_20190809125133", "doc_count": 12 }, { "key": "ori_goirle_20190809125132", "doc_count": 12 }, { "key": "osi_overijssel_20190822111458", "doc_count": 12 }, { "key": "ori_amsterdam_20190809125033", "doc_count": 10 }, { "key": "ori_groningen_20190809125025", "doc_count": 10 }, { "key": "ori_utrecht_20190809125127", "doc_count": 10 }, { "key": "ori_waalwijk_20190809125138", "doc_count": 9 }, { "key": "ori_maassluis_20190809125107", "doc_count": 8 }, { "key": "ori_roosendaal_20190809125037", "doc_count": 8 }, { "key": "ori_amsterdam_oost_20190809125046", "doc_count": 7 }, { "key": "ori_moerdijk_20190809125042", "doc_count": 7 }, { "key": "ori_nijmegen_20190809125121", "doc_count": 7 }, { "key": "ori_stichtse_vecht_20190809125106", "doc_count": 7 }, { "key": "ori_vlissingen_20190809125043", "doc_count": 7 }, { "key": "ori_zandvoort_20190809125106", "doc_count": 7 }, { "key": "ori_amsterdam_zuidoost_20190809125101", "doc_count": 6 }, { "key": "ori_den_helder_20190809125127", "doc_count": 6 }, { "key": "ori_ede_20190809125045", "doc_count": 6 }, { "key": "ori_hulst_20190809125116", "doc_count": 6 }, { "key": "ori_katwijk_20190809125133", "doc_count": 6 }, { "key": "ori_krimpen_ad_ijssel_20190809125113", "doc_count": 6 }, { "key": "ori_leiden_20190809125124", "doc_count": 6 }, { "key": "osi_flevoland_20190809125113", "doc_count": 6 }, { "key": "osi_provincie-utrecht_20190809125121", "doc_count": 6 }, { "key": "ori_alkmaar_20190809125035", "doc_count": 5 }, { "key": "ori_castricum_20190809125052", "doc_count": 5 }, { "key": "ori_doetinchem_20190809125054", "doc_count": 5 }, { "key": "ori_schagen_20190809125124", "doc_count": 5 }, { "key": "ori_wageningen_20190809125043", "doc_count": 5 }, { "key": "ori_amsterdam_zuid_20190809125107", "doc_count": 4 }, { "key": "ori_baarn_20190809125121", "doc_count": 4 }, { "key": "ori_beemster_20190809125040", "doc_count": 4 }, { "key": "ori_bloemendaal_20190809125121", "doc_count": 4 }, { "key": "ori_culemborg_20190809125124", "doc_count": 4 }, { "key": "ori_dongen_20190809125121", "doc_count": 4 }, { "key": "ori_drimmelen_20190809125033", "doc_count": 4 }, { "key": "ori_enschede_20190809125113", "doc_count": 4 }, { "key": "ori_epe_20190809125047", "doc_count": 4 }, { "key": "ori_hardenberg_20190809125038", "doc_count": 4 }, { "key": "ori_medemblik_20190809125134", "doc_count": 4 }, { "key": "ori_oude_ijsselstreek_20190809125121", "doc_count": 4 }, { "key": "ori_capelle_ad_ijssel_20190809125112", "doc_count": 3 }, { "key": "ori_heemskerk_20190809125050", "doc_count": 3 }, { "key": "ori_hendrik-ido-ambacht_20190809125056", "doc_count": 3 }, { "key": "ori_noordoostpolder_20190809125124", "doc_count": 3 }, { "key": "ori_oisterwijk_20190809125116", "doc_count": 3 }, { "key": "ori_rucphen_20190809125124", "doc_count": 3 }, { "key": "ori_staphorst_20190809125107", "doc_count": 3 }, { "key": "ori_veenendaal_20190809125132", "doc_count": 3 }, { "key": "ori_zwartewaterland_20190809125059", "doc_count": 3 }, { "key": "ori_amsterdam_centrum_20190809125121", "doc_count": 2 }, { "key": "ori_amsterdam_nieuw-west_20190809125033", "doc_count": 2 }, { "key": "ori_amsterdam_west_20190809125108", "doc_count": 2 }, { "key": "ori_borne_20190809125124", "doc_count": 2 }, { "key": "ori_leusden_20190809125117", "doc_count": 2 }, { "key": "ori_nieuwkoop_20190809125047", "doc_count": 2 }, { "key": "ori_ommen_20190809125052", "doc_count": 2 }, { "key": "ori_oost_gelre_20190809125133", "doc_count": 2 }, { "key": "ori_overbetuwe_20190809125117", "doc_count": 2 }, { "key": "ori_schiedam_20190809125052", "doc_count": 2 }, { "key": "ori_vlaardingen_20190809125128", "doc_count": 2 }, { "key": "ori_voorschoten_20190809125037", "doc_count": 2 }, { "key": "ori_zeewolde_20190809125026", "doc_count": 2 }, { "key": "ori_almelo_20190809125121", "doc_count": 1 }, { "key": "ori_de_wolden_20190809125117", "doc_count": 1 }, { "key": "ori_emmen_20190809125028", "doc_count": 1 }, { "key": "ori_gouda_20190809125124", "doc_count": 1 }, { "key": "ori_harderwijk_20190809125106", "doc_count": 1 }, { "key": "ori_leiderdorp_20190809125020", "doc_count": 1 } ] } } } ```

For a complete comparison of the feb 2019 sample see: https://docs.google.com/spreadsheets/d/1a1C6--9H0ERPsTR45SdMF4tDVKxsvf6NNBfwJWsyI-g/edit?usp=sharing

On 9th of sept the total count for feb 2019 was 505 which has increased to 622, but still less than 771 in V0. The comparison report above shows that a total of 44 municipalities are still missing for the month feb 2019. However, this is for the feb 2019 sample, municipalities missing in this month might be included in other months. The missing municipalities are almost all iBabs (40), this has probably to do with the problems we encountered with iBabs earlier.

jurrian commented 4 years ago

The problem why we are falling short is probably because of the processing capacity of the ORI system. We have a daily cronjob which starts at midnight fetching the last month and coming month, the entities are mainly meetings and reports. Persons, organizations and such are processed in the monthly cron. The daily cron seems to be taking up 19 hours already (encounting) and, as the name suggests is ran every night. But if the daily cron already takes up this amount of time, there is no time to process previous years. Therefore I am thinking about making the daily cron weekly, which has as consequence that the data will be less current.

tomkunzler commented 4 years ago

I don't think switching to a weekly cron job is wise. That will mean that the information is not very up to date and the use for ORI is quite dependent on getting all data asap (like for the purpose of 1848.nl re-use users want to have new information when it comes up). So going to weekly will make the dataset a lot less valuable.

Isn't it an option to get more processing capacity to make the daily cron job feasible?

breyten commented 4 years ago

Also, sinceit's running every day, it ould also help to make the period shorter, Ie. only last week and next week. Or maybe next two weeks, since next is usually not that much data ... (But then again, not sure what kind of impact that has on completeness)

jurrian commented 4 years ago

Thanks for your input, I agree that making the range shorter (week instead of month) might be the best solution here. We will keep the daily.

jurrian commented 4 years ago

Updated comparison Februari 2019: https://docs.google.com/spreadsheets/d/1a1C6--9H0ERPsTR45SdMF4tDVKxsvf6NNBfwJWsyI-g/edit?usp=sharing

Last time not all municipalities were indexed in the Feb 2019 sample, now they are all indexed. In most occurrences V1 is now equally as or more complete than V0 in this sample, which should be reflect the rest of 2019.

jurrian commented 4 years ago

It took just a few days to import the whole of 2018. Here the comparison V0 vs V1 comparison for October 2018: https://docs.google.com/spreadsheets/d/1a1C6--9H0ERPsTR45SdMF4tDVKxsvf6NNBfwJWsyI-g/edit#gid=128243340

6 municipalities/provinces appear missing, I will reprocess them manually.

joepio commented 4 years ago

Over 1.000.000 resources have been loaded into v1!

jurrian commented 4 years ago

A day later Elasticsearch now contains almost 1.500.000 resources. Over the whole period of 2000-2020, V1 now contains 78754 meetings vs 48579 in V0. See the report below for the monthly comparison: https://docs.google.com/spreadsheets/d/1a1C6--9H0ERPsTR45SdMF4tDVKxsvf6NNBfwJWsyI-g/edit#gid=83625813

The problems fixed most notably in milestones v1.3.4, v1.3.5 and v1.4.1 prevented reaching this goal earlier due to really slow processing. With these problems gone, importing until 2000 appeared to be much easier.

The goal of this issue was to measure completeness of V1 against V0 to determine whether V1 could be used and V0 deprecated. At this point we can say this goal has now been reached. Therefore this issue can be closed.

joepio commented 4 years ago

Awesome! Feels good to confidently close this one :)