For the sake of interoperability, it is important for TDM Agents to know where the should look for information first, then second, then third. It will avoid cases where the information has been left in several places by publishers, with different values for tdm-reservation and tdm-policy.
My proposal is :
1- TDM File on the Origin Server -> the TDM Agent can check this before starting scraping a website, cache the information if any, and check the rules against any content he fetches.
2- TDM Header Field for HTTP Requests -> this is the first thing the TDM Agent will see when it fetches content and it is easy to process. Info found here supercedes previous info.
3- TDM Metadata in HTML Content -> this can only be done after the HTML content has been parsed. Therefore later and after more processing. Info found here supercedes previous info.
For the sake of interoperability, it is important for TDM Agents to know where the should look for information first, then second, then third. It will avoid cases where the information has been left in several places by publishers, with different values for tdm-reservation and tdm-policy.
My proposal is : 1- TDM File on the Origin Server -> the TDM Agent can check this before starting scraping a website, cache the information if any, and check the rules against any content he fetches. 2- TDM Header Field for HTTP Requests -> this is the first thing the TDM Agent will see when it fetches content and it is easy to process. Info found here supercedes previous info. 3- TDM Metadata in HTML Content -> this can only be done after the HTML content has been parsed. Therefore later and after more processing. Info found here supercedes previous info.