InternetHealthReport / internet-yellow-pages

A knowledge graph for Internet resources
GNU General Public License v3.0
39 stars 16 forks source link

Clarification on Database Dump Timelines at https://exp1.iijlab.net/wip/iyp/dumps/ #84

Closed mohamedawnallah closed 9 months ago

mohamedawnallah commented 9 months ago

Problem

I'm attempting to install Internet Yellow Pages on my local machine for experimentation. The project is intriguing, but I'm uncertain about the content of the database dumps available here. Specifically, I'm unsure whether these dumps contain all accumulated relationships since the initial publication of the database dump on 2022/12/09, or if they only include the new relationships added between the previous dump and the latest one.

Additional Context

My query stems from a desire to understand how the network relationships have evolved from, for instance, 2022/12/09until the present. To achieve this, I'm seeking clarification on whether I should download all the database dumps or solely the most recent one.

m-appel commented 9 months ago

Thanks for your interest in IYP!

Each dump represents only a snapshot of all datasets. Keeping track of changes within the database, i.e., continuously updating it, would be too complicated with the amount of data we have and would also complicate queries.

So if you want to do a longitudinal analysis, you currently have to download multiple dumps.

mohamedawnallah commented 9 months ago

Thank you for answering my query; it makes sense!