search

mundanelunacy / covidmaps

Website were people view locations where confirmed COVID19 virus cases had visited on a Google Map
https://www.covidmaps.org
Other
5 stars 0 forks source link

How and where is the data stored? #16

Open E3V3A opened 4 years ago

E3V3A commented 4 years ago

How and where is the data stored?

mundanelunacy commented 4 years ago

Data is stored on realtime database called Firestore (Google Firebase)

E3V3A commented 4 years ago

Data is stored on realtime database called Firestore (Google Firebase)

ok, So on google servers?

mundanelunacy commented 4 years ago

Correct. Data transfer to the covidmaps client is a bit mysterious as it uses a Websockets to access this database. That's the preferred way to access this database but standard REST API via OAUTH2.0 is supported as well, i.e.

GET https://firestore.googleapis.com/v1/projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0

{ "documents": [ { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/0006c45ad4e1d2aa9e961f8026d93fcd5111a8e5", "fields": { "name": { "stringValue": "סופר-פארם יפו" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585560600000" }, "placeId": { "stringValue": "ChIJaSQ3S7pMHRURyr-7GTlpNq4" }, "address": { "stringValue": "שדרות ירושלים 49, תל אביב יפו, ישראל" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 32.0504862, "longitude": 34.7592203 } }, "geohash": { "stringValue": "sv8wqfwgs" } } } }, "startTimestampMs": { "integerValue": "1585558800000" } }, "createTime": "2020-04-13T19:14:32.653660Z", "updateTime": "2020-04-13T19:14:32.653660Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/000ea81a93e7deff5e0a07ea0db80ff7e89cc086", "fields": { "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 31.7051188, "longitude": 35.1168587 } }, "geohash": { "stringValue": "sv8ur5erd" } } } }, "startTimestampMs": { "integerValue": "1586237400000" }, "name": { "stringValue": "יש חסד ביתר עילית" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1586239200000" }, "placeId": { "stringValue": "ChIJXZXNw-fbAhURztrswifvzYk" }, "address": { "stringValue": "הר\"ן 6, ביתר עילית" } }, "createTime": "2020-04-12T15:12:55.969413Z", "updateTime": "2020-04-12T15:12:55.969413Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/0010cf743c3b76a2c4cf9ead2d357d39a32f3e78", "fields": { "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 31.2264211, "longitude": 34.8080059 } }, "geohash": { "stringValue": "sv8c8hd7j" } } } }, "startTimestampMs": { "integerValue": "1585818000000" }, "name": { "stringValue": "קונדיטוריית מרטין (סניף עמק שרה)" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585828800000" }, "placeId": { "stringValue": "ChIJSTAeuyhmAhURYOzk0wCHvfw" }, "address": { "stringValue": "הפועלים 28, באר שבע, ישראל" } }, "createTime": "2020-04-11T19:17:54.751618Z", "updateTime": "2020-04-11T19:17:54.751618Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00135e2dcdec6a0998a48b29b9b9809ef57b3b80", "fields": { "placeId": { "stringValue": "ChIJjUgk_eeffDURzsD2DfPzOOA" }, "address": { "stringValue": "대한민국 서울특별시 관악구 신림동 호암로 605" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 37.4713408, "longitude": 126.9336826 } }, "geohash": { "stringValue": "wydm0mqq3" } } } }, "startTimestampMs": { "integerValue": "1585839540000" }, "name": { "stringValue": "다이소 신림점" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585843140000" } }, "createTime": "2020-04-09T09:29:22.261024Z", "updateTime": "2020-04-09T09:29:22.261024Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00285f3674de7102e35741fd12d2c197eef307f3", "fields": { "name": { "stringValue": "בית אבות יוקרה יבנאל" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585594800000" }, "placeId": { "stringValue": "ChIJqZYyAdxAHBURCXHDhy6yR7U" }, "address": { "stringValue": "יבנאל, ישראל" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 32.7219273, "longitude": 35.4949277 } }, "geohash": { "stringValue": "svc4psye5" } } } }, "startTimestampMs": { "integerValue": "1585566000000" } }, "createTime": "2020-04-13T16:10:54.401095Z", "updateTime": "2020-04-13T16:10:54.401095Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/0033ecdb7e658d668e0cec67b779170b87c3573d", "fields": { "address": { "stringValue": "רחוב ירושלים 2, נתיבות, ישראל" }, "position": { "mapValue": { "fields": { "geohash": { "stringValue": "sv8dg2g9s" }, "geopoint": { "geoPointValue": { "latitude": 31.4253171, "longitude": 34.6009283 } } } } }, "startTimestampMs": { "integerValue": "1585472400000" }, "name": { "stringValue": "דלק - נתיבות" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585474200000" }, "placeId": { "stringValue": "ChIJHXghR959AhURNkQEYFnG3tU" } }, "createTime": "2020-04-12T19:18:41.769657Z", "updateTime": "2020-04-12T19:18:41.769657Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/003c7a16958ca33e1f2092d3dab81f22f18ea150", "fields": { "address": { "stringValue": "" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 34.948924727000076, "longitude": 32.594282776000057 } }, "geohash": { "stringValue": "swpnvdq2j" } } } }, "startTimestampMs": { "integerValue": "1585740600000" }, "name": { "stringValue": "אלמשהדאוי סניף פוריידיס - פוריידיס" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585741500000" }, "placeId": { "stringValue": "" } }, "createTime": "2020-04-11T11:17:41.090229Z", "updateTime": "2020-04-11T11:17:41.090229Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00656d8155b312ea4243a6c4a1ffaa860332e24d", "fields": { "name": { "stringValue": "מעוף מעיין יסכה" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585837800000" }, "placeId": { "stringValue": "ChIJVTotSou9AhURhnngE6QedC4" }, "address": { "stringValue": "א.ת רחוב תנובה, באר טוביה, ישראל" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 31.7245995, "longitude": 34.7516712 } }, "geohash": { "stringValue": "sv8sqz33h" } } } }, "startTimestampMs": { "integerValue": "1585795200000" } }, "createTime": "2020-04-12T19:16:00.681587Z", "updateTime": "2020-04-12T19:16:00.681587Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/006d69145b4eb32f74f832cfb7faa039f80330ae", "fields": { "placeId": { "stringValue": "ChIJhT2mVP8zHRURUcmRYqlvaFo" }, "address": { "stringValue": "רב ניסים גאון 79, אלעד, ישראל" }, "position": { "mapValue": { "fields": { "geohash": { "stringValue": "sv8y76fxw" }, "geopoint": { "geoPointValue": { "latitude": 32.052559, "longitude": 34.9512194 } } } } }, "startTimestampMs": { "integerValue": "1585485000000" }, "name": { "stringValue": "יש חסד אלעד" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585513800000" } }, "createTime": "2020-04-12T16:12:14.265792Z", "updateTime": "2020-04-12T16:12:14.265792Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/007d612bc6c9be3a1e924d8b7122d5583e479b49", "fields": { "name": { "stringValue": "בר כל" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585672200000" }, "placeId": { "stringValue": "ChIJtTI7kqe3AhURwUwI3ok-Tq4" }, "address": { "stringValue": "זכריה מדאר פינת כובשי החרמון, רחובות, ישראל" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 31.8881216, "longitude": 34.8190833 } }, "geohash": { "stringValue": "sv8v2m43z" } } } }, "startTimestampMs": { "integerValue": "1585668600000" } }, "createTime": "2020-04-09T06:52:06.646702Z", "updateTime": "2020-04-09T06:52:06.646702Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/009dff1c40e5150393a58bd93c84368ffabccbd3", "fields": { "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 37.4601908, "longitude": 126.4406957 } }, "geohash": { "stringValue": "wy9vj5rm3" } } } }, "startTimestampMs": { "integerValue": "1586012340000" }, "name": { "stringValue": "인천국제공항" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1586015940000" }, "placeId": { "stringValue": "ChIJWfpeOoOaezUR1L5cy5agS40" }, "address": { "stringValue": "대한민국 인천광역시 중구 공항로 272" } }, "createTime": "2020-04-09T09:20:46.281022Z", "updateTime": "2020-04-09T09:20:46.281022Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00bc97028a1a15e61a446aa36f4581744d71a770", "fields": { "name": { "stringValue": "האחים יעקובי" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1586030400000" }, "placeId": { "stringValue": "ChIJJ7HDhcHXAhURGjkHn69LT_8" }, "address": { "stringValue": "הרב חיים הלר 22, ירושלים, ישראל" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 31.7627267, "longitude": 35.197891100000007 } }, "geohash": { "stringValue": "sv9h8ynr6" } } } }, "startTimestampMs": { "integerValue": "1586019600000" } }, "createTime": "2020-04-11T19:12:28.983645Z", "updateTime": "2020-04-11T19:12:28.983645Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00c335aaf850a78adf8cb0bae011f89e468306f6", "fields": { "position": { "mapValue": { "fields": { "geohash": { "stringValue": "sv8tz8h9c" }, "geopoint": { "geoPointValue": { "latitude": 31.9485812, "longitude": 34.7889725 } } } } }, "startTimestampMs": { "integerValue": "1585886400000" }, "name": { "stringValue": "מרכז גריאטרי" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585915200000" }, "placeId": { "stringValue": "ChIJ2crT-xm0AhUR7U_NHPRZY5o" }, "address": { "stringValue": "דוד רמז 95, ראשון לציון, ישראל" } }, "createTime": "2020-04-10T15:11:59.317448Z", "updateTime": "2020-04-10T15:11:59.317448Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00d7f3c06d24eb01c08dcb2736795c23dfb392f0", "fields": { "address": { "stringValue": "קניון בית הדר, הרכבים 8, ירושלים, ישראל" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 31.7534878, "longitude": 35.2130043 } }, "geohash": { "stringValue": "sv9h9k936" } } } }, "startTimestampMs": { "integerValue": "1585566000000" }, "name": { "stringValue": "אושר עד" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585580400000" }, "placeId": { "stringValue": "ChIJQXz-suHXAhURN-uju7t6w-w" } }, "createTime": "2020-04-13T16:11:22.014091Z", "updateTime": "2020-04-13T16:11:22.014091Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00e12c87cb1a44b4adb0bbb03eb60f7b51ca6a71", "fields": { "placeId": { "stringValue": "ChIJ30iSbYc1HRUROqX4LerD9wI" }, "address": { "stringValue": "השיקמה 2, קרית אונו, ישראל" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 32.0521651, "longitude": 34.8635526 } }, "geohash": { "stringValue": "sv8y36fv7" } } } }, "startTimestampMs": { "integerValue": "1585801800000" }, "name": { "stringValue": "קופת חולים כללית" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585825200000" } }, "createTime": "2020-04-09T06:54:52.846702Z", "updateTime": "2020-04-09T06:54:52.846702Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00e83db072d7ce61e0f2d3c21b7e791fea91aa9c", "fields": { "placeId": { "stringValue": "ChIJ1-uetBkXZDUR9EeLhbylX8Q" }, "address": { "stringValue": "대한민국 예천읍 남본리 222-58번지 세종목욕탕세종프라자 예천군 경상북도 KR" }, "position": { "mapValue": { "fields": { "geohash": { "stringValue": "wye2dbzz2" }, "geopoint": { "geoPointValue": { "latitude": 36.6557758, "longitude": 128.4518173 } } } } }, "startTimestampMs": { "integerValue": "1586185140000" }, "name": { "stringValue": "세종목욕탕" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1586188740000" } }, "createTime": "2020-04-12T03:10:06.240296Z", "updateTime": "2020-04-12T03:10:06.240296Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00ea29c7b2a564b171044db20e28b6bc8029439c", "fields": { "name": { "stringValue": "מחסני השוק" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585576800000" }, "placeId": { "stringValue": "ChIJiw0Zikt6AhURCNWicKPaJEI" }, "address": { "stringValue": "שדרות יהדות דרום אפריקה 14, אופקים, ישראל" }, "position": { "mapValue": { "fields": { "geohash": { "stringValue": "sv8d5uubq" }, "geopoint": { "geoPointValue": { "latitude": 31.3152192, "longitude": 34.6247327 } } } } }, "startTimestampMs": { "integerValue": "1585571400000" } }, "createTime": "2020-04-13T16:10:34.373235Z", "updateTime": "2020-04-13T16:10:34.373235Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/00fb5aff9f9948b4745ce5afd420f1c4a7a2809e", "fields": { "placeId": { "stringValue": "ChIJ16uLFRlKHRURBz_IfqS8ul0" }, "address": { "stringValue": "הרב פוברסקי 17, בני ברק, ישראל" }, "position": { "mapValue": { "fields": { "geohash": { "stringValue": "sv8y8busq" }, "geopoint": { "geoPointValue": { "latitude": 32.084959, "longitude": 34.844088 } } } } }, "startTimestampMs": { "integerValue": "1586145600000" }, "name": { "stringValue": "מרכז רפואי מעייני הישועה" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1586188800000" } }, "createTime": "2020-04-11T20:11:48.404002Z", "updateTime": "2020-04-11T20:11:48.404002Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/0125f86c915d048e77ee81cb4fb9a79a692760c9", "fields": { "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 37.7513397, "longitude": 127.1193025 } }, "geohash": { "stringValue": "wydqu8mmq" } } } }, "startTimestampMs": { "integerValue": "1585666740000" }, "name": { "stringValue": "코스트코 의정부점" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585670340000" }, "placeId": { "stringValue": "ChIJSwVmlj7GfDUR8lMgE3Wczf8" }, "address": { "stringValue": "대한민국 경기도 의정부시 송산2동 용민로489번길 9" } }, "createTime": "2020-04-09T09:32:04.286776Z", "updateTime": "2020-04-09T09:32:04.286776Z" }, { "name": "projects/coronamaps-9e44a/databases/(default)/documents/incidents-schemaV1.0/013c2a43b4c73b16701d79d4ed6c939b3a59ccf0", "fields": { "address": { "stringValue": "אוגדה 30, מעלה אדומים" }, "position": { "mapValue": { "fields": { "geopoint": { "geoPointValue": { "latitude": 31.7932316, "longitude": 35.3288288 } }, "geohash": { "stringValue": "sv9hggv8v" } } } }, "startTimestampMs": { "integerValue": "1585630800000" }, "name": { "stringValue": "הר - שפי בע\"מ" }, "validated": { "booleanValue": true }, "endTimestampMs": { "integerValue": "1585663200000" }, "placeId": { "stringValue": "ChIJnT0o9vQM9xQRcJ-9Hz16cs4" } }, "createTime": "2020-04-10T11:14:23.879893Z", "updateTime": "2020-04-10T11:14:23.879893Z" } ], "nextPageToken": "ALumMsXQOmZNnuh2iUb65yEJyWP72GP4twi4IzfErsX9MERB3W8ceJmmuQbAfHZTcV81HJLNnbN523ip4KXJBXtLfmgp4DgC3UgIuXW3GeM4IADab8HzT0JlPPiMNfdseOY9X2Lb2zb5sS0H1jlnxdSuQzJpCbbWgDHsFm34DqaqsuA" }

E3V3A commented 4 years ago

Wow, this is pretty cool. This format also look the same as that used for the Israeli app...

Any idea what happens when that "file" get very large? I.e. when you have 100,000's of points?

mundanelunacy commented 4 years ago

https://github.com/codediodeio/geofirex

Covidmaps does not download the whole payload from Firebase every time. It uses geofire library to geo-query data points within the bounds of the Google Maps Viewport.