Download of documents and RDF files

sorenroug commented 9 years ago

It is time to come up with a strategy on how to support the documents and dataset that the users can download from the website.

It is possible to just place the documents on the server in a dedicated folder just like it is done on the old system, but that would mean that data is stored on the server side, and that isn't cloud friendly as the system is no longer horisontally scalable and we'll have to take daily backups of the server.

It would be preferable to use Azure's BLOB storage. There is sample code at https://github.com/azure/azure-storage-java. The storage connection string should be loaded from JNDI as in:

<Context>
    <Environment name="blobstorage"
                 value="DefaultEndpointsProtocol=http;AccountName=..."
                 type="java.lang.String" override="false"/>
</Context>

Requests should then be mapped to /doc/{filename} and /rdf/{filename}

sorenroug commented 9 years ago

When reading How to use Blob storage from Java it looks like all you have to do is to make a HTTP redirect to `http://eprtr.blob.core.windows.net/doc/filename.pdf.

sorenroug commented 9 years ago

I have now uploaded some files to a real Azure blob storage. You can do:

sorenroug commented 9 years ago

The web UI for Azure blob storage doesn't have upload functionality, so I have decided to use the other cloud provider. You can browse the files by going to https://00a58390727c49d28d3f65e78186ac81.objectstore.eu/E-PRTR/ and the examples above are

https://00a58390727c49d28d3f65e78186ac81.objectstore.eu/E-PRTR/rdf/Activity.rdf.gz https://00a58390727c49d28d3f65e78186ac81.objectstore.eu/E-PRTR/doc/EN_E-PRTR_fin.pdf

alvardan commented 9 years ago

Hi, DG ENV received a suggestion from a user that relates to this issue. Please take it into account.

Shortly after sending you my suggestion and while continuing extracting info from the E-PRTR web site, I realised that it would be better or even best to provide ALL meta-information as download to CSV. That means all meta-information that is presented on the views in the website should best be made available INTREGAL as downloadable to the CSV. That way you serve the user best and fully, as he has as such all freedom and all options to further select info to his own needs and taste. (And as it concerns, I think, a relative limited bulk of additional information, which nowadays does not put serious burden on the file size and its download transfer).

This came from a previous suggestion that reads like this:

In a project on air pollution in relation to spatial allocated emissions we needed the spatial related component of point source emissions. E-PRTR provides the geographical coordinates of the selected facilities at the view with the meta-information per facility. We found out that the long/lat coordinates as presented in the web page overview per facitlity are not included in these CSV downloads unfortunately. Our urgent suggestion would be to improve the downloads by including these geogr. coordinates in the CSV as well.

eea / eprtr2

Download of documents and RDF files #24