weso / weso-scholia

Collaboration between Scholia and WESO
GNU General Public License v2.0
3 stars 3 forks source link

Procedure to pubish dumps? #18

Open labra opened 3 years ago

labra commented 3 years ago

I would like to share some of the dumps that I have created with wdsub. How can I do it?

Maybe creating a folder in some place with external read-access?

By the way, the reason is that I have created a subset based on this ShEx which obtains all wikidata compounds:

prefix :  <http://www.wikidata.org/entity/>

# Expression that represents all compounds
# This query requires a 150000 limit in WDQS
<compounds> {
    :P31 [ :Q11173 ]
}

The result of the dump is a subset with 1201905 chemical compounds. The original query gave a timeout.

thewillyhuman commented 3 years ago

This is an interesting question. I've taking a look to what others do in this case and looks like almost all of them create a public shared directory called public_html under the /home/<user> directory. This is accessible from the www as http://hpc.weso.es/~<user>. And the webpage looks as follows:

Enable-Apache-UserDir-in-CentOS

I think we has something very similar at the EII with Ritchie server. My concerns are about how much space would be required for those directories. How much space does the subset requires?

Daniel-Mietchen commented 3 years ago

I think a better option would be to upload these dumps to Zenodo.

thewillyhuman commented 3 years ago

Right now we are publishing the dumps at http://files.hpc.weso.es:8080/. We're using the https://hub.docker.com/r/halverneus/static-file-server docker container.