Display of the Data Policy before starting the download of data

[joerg-halo]

The WLA decided that before the start of a download, users have to be made aware of the HALO Data Policy and the Data Protocol of the respective mission. This should happen via a popup window or popup website.

[d70-t]

I think this statement could be highly problematic.

This raises at least the question what exactly is called a "download" in the terms of the WLA. Then there is the question what is meant by "before" i.e. is that once or every time or once per month? And in consequence, how this interacts with

• the meaning of “publicly available” data: "The HALO-WLA clarified, that this excludes every form of mandatory registration."
• accessing subsets of datasets i.e. #51
• script based access #48

[joerg-halo]

This raises at least the question what exactly is called a "download" in the terms of the WLA.

I am not sure if I get this point. In my simple view, a download is everything when 'actual' data (not metadata) are transferred to a computer which is not part of the DB. Exceptions for visualization tools might be discussed, depending on the tool. A display of a quicklook might not require the acceptance of the data policy, more advanced features might require it.

Then there is the question what is meant by "before" i.e. is that once or every time or once per month?

"Every time" is meant. In the WLA I mentioned Aeronet as an example for this scheme: https://aeronet.gsfc.nasa.gov/

the meaning of “publicly available” data: "The HALO-WLA clarified, that this excludes every form of mandatory registration."

I don't get this obstacle. @d70-t, could you help me?

accessing subsets of datasets i.e. #51

The data policy could be displayed with the retrieval of each subset, couldn't it?

script based access #48

Yepp, I think that this is the main issue. However, I advocate that this shouldn't influence us, when dealing with the script based access. If this feature is about to become adequately implemented, the WLA has to discuss how to deal with this contradiction.

[d70-t]

In my simple view, a download is everything when 'actual' data (not metadata) are transferred to a computer which is not part of the DB.

This raises the question about what is a "computer which is not part of the DB". If we have support to store datasets on machines which are not provided by DLR (or whoever runs the "core" of HALO-DB), are then all of these computers "part of the DB", or are they not?

• If they are not "part of the DB", then any process which transfers data to those machines must show the HALO Data Policy, even if those transfers are e.g. for distributed caching of data to make the database more reliable.
• If they are "part of the DB", then every data transfer from one of those outside machines to a users laptop must result in a display of such a message. But as these machines are probably not really in control of the HALO-DB, it is hard to enforce this policy.

-> probably we can't have any external data store if we want to enforce the rule.

"Every time" is meant.

This raises the question about what is "a time". One could be pedantic and refer to single IP-Packets, but that might be too picky. However one thing which we surely want to do is access subsets of the data (otherwise larger datasets are definitely out of reach). In that case, it is really common that a user / client application / etc.. creates many (could easily be 100s or 1000s) requests to different parts of the same dataset in order to create a single figure. Each of those requests look like the first request to the server / database and thus is per se not distinguishable from the second or any other request. If a message must be displayed "every time" then this would necessarily create a mess on the user's computer.

If we would force the user in stead to bundle all requests into a single big one, the whole thing would be really complicated to use any many (especially interactive) ways to work with the data would become impossible, as often one does not yet know which subsets of the data are needed before retrieving the first parts. (Think of downloading the time axis first to determine which data indices are within your study time, then request the rest only for the subsets in that time period).

"The HALO-WLA clarified, that this excludes every form of mandatory registration."

If we do not want to display a message "every time" (see above why this is a bad idea), but in stead only some times, then we basically have two options:

• Track the user and enforce the display of a message whenever the server likes to do this. This contradicts the exclusion of mandatory registration.
• Ask the user if the user has seen the message already. This is technically equivalent to let the user choose if they want to see the message, so we can not enforce it. Thus it contradicts the goal of this use case.

The data policy could be displayed with the retrieval of each subset, couldn't it?

As mentioned above, it will become very common to access 100s or 1000s of subsets per day per user, I don't want to imagine how much time I'll be spending clicking pop up windows away.

script based access #48

Yepp, I think that this is the main issue. However, I advocate that this shouldn't influence us, when dealing with the script based access. If this feature is about to become adequately implemented, the WLA has to discuss how to deal with this contradiction.

See above. Also we have plenty of use cases which will likely require some form of script based access, e.g.: #56, #53 (at least the reverse), #51, #50, #48, #47, #43 ?, #37, #20, #17?. If we should have a design which decouples the data storage from the user interface (which is highly desirable), then even our own user interface will be "script based access".

In the WLA I mentioned Aeronet as an example for this scheme: https://aeronet.gsfc.nasa.gov/

They are not enforcing the display. A user has several possibilities to obtain the data:

• The download tool asks the user a couple of questions, which determine a selection of data (sub-)sets in advance. Once the user has chosen everything they want, a data policy message is shown.
• The download all page provides a link to big files with everything. No message is shown.
• The Web Service which describes how a user can build a command to download specific data. No message is shown.

-> The Aeronet site has a user interface which makes it easy for a user to stumble across the data policy, but it does not enforce it for all downloads, because this would make very useful use cases impossible.

---

I think this all boils down to an analysis of costs versus benefits of the enforcement.

• Enforcing the display of a message is extremely hard and we would likely have to disable a large suite of important use cases to do so. Additionally, if we don't want to extremely annoy people, enforcing the display will require to register and track users only for that purpose. Apart from those drawbacks, I really would question the usefulness of this approach. If someone really wants to do bad / inappropriate things with the data, people will do that, no matter if the have clicked such a message away or not. Changes are high that they don't even read the message, as no one does e.g. for user agreements on social media platforms or even some contracts.
• On the other hand, we could easily attach license information / usage policy information to the datasets. We could create applications which by default would tell people to be nice and have a look at those things. We would make it really easy to people who want to comply to the rules to do so. We can also provide teaching resources with the datasets which can contain hints that those policies are important to observe. In my opinion, this will encourage more people to actually have a look at those things than any enforcement is able to do. By not enforcing this rule, we will be able to provide a significantly better service and don't need to track everyone.

[joerg-halo]

Alright, thank you @d70-t for the comprehensive answer. My conclusion is the following:

When data are downloaded via the web-surface, the data policy has to be displayed. This is the actual use case which the WLA had in mind, when making the respective decision.

I understand now, that a number of use case would be inhibited by a strong interpretation of the WLA decision. However, this should not influence our ways of thinking about solutions for the HALO-DB. Upcoming contradictions can be discussed within the WLA.