search

agroportal / project-management

Repository used to consolidate documentation about the AgroPortal project and track content related issues.
http://agroportal.lirmm.fr
7 stars 0 forks source link

Do an adhoc script to remove unzipped folder from archived submissions #305

Closed syphax-bouazzouni closed 2 months ago

syphax-bouazzouni commented 1 year ago

Requirement

Now when we archive an ontology, we remove all the generated files and folders, and if the original ontology file is big (>= 100 mo) we zipped it. See details in https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/87

Use cases

All the below submissions have been re-achieved to remove old files and zip them. 

[6] pry(main)> s.archive
=> "/srv/ontoportal/data/repository/AGROVOC/36/agrovoc_lod.nt.zip"
[7] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/AGROVOC/submissions/37")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/AGROVOC/37/agrovoc_lod.nt.zip"
[8] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/AGROVOC/submissions/38")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/AGROVOC/38/agrovoc_lod.nt.zip"
[9] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/AGROVOC/submissions/39")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/AGROVOC/39/agrovoc_lod.nt.zip"
[10] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/AGROVOC/submissions/34")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/AGROVOC/34/agrovoc_lod.nt.zip"
[11] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/AGROVOC/submissions/35")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/AGROVOC/35/agrovoc_lod.nt.zip"
[12] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/5")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/5/NCBITAXON_2019AB.ttl.zip"
[13] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/6")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/6/NCBITAXON.ttl.zip"
[14] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/7")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/7/NCBITAXON_2020AB.ttl.zip"
[15] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/1")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/1/NCBITAXON.ttl.zip"
[16] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/9")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/9/NCBITAXON.ttl.zip"
[17] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/10")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/10/NCBITAXON.ttl.zip"
[18] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/11")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/11/NCBITAXON.ttl.zip"
[19] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/2")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/2/NCBITAXON_2016AB.ttl.zip"
[20] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/3")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/3/NCBITAXON_2017AB.ttl.zip"
[21] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/NCBITAXON/submissions/4")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/NCBITAXON/4/NCBITAXON_2018AA.ttl.zip"
[22] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologied/TAXREF-LD/submissions/1")).first.bring_remaining.archive
NoMethodError: undefined method `bring_remaining' for nil:NilClass
from (pry):22:in `block in <main>'
[23] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/TAXREF-LD/submissions/1")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/TAXREF-LD/1/Taxrefld10_BN_explicit-with-AgroportalMetadata.rdf.zip"
[24] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/TAXREF-LD/submissions/2")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/TAXREF-LD/2/taxrefld_singlefile_agropportal.ttl.zip"
[25] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/TAXREF-LD/submissions/3")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/TAXREF-LD/3/taxrefld_singlefile_agropportal.ttl.zip"
[26] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/TAXREF-LD/submissions/4")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/TAXREF-LD/4/taxrefld_singlefile_agropportal.ttl.zip"
[27] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/AGROVOC/submissions/19")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/AGROVOC/19/agrovoc_2022-03-02_lod.nt.zip"
[28] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/AGROVOC/submissions/23")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/AGROVOC/23/agrovoc_core.nt.zip"
[29] pry(main)> s = LinkedData::Models::OntologySubmission.find(RDF::URI.new("http://data.bioontology.org/ontologies/AGROVOC/submissions/24")).first.bring_remaining.archive
=> "/srv/ontoportal/data/repository/AGROVOC/24/agrovoc_core.nt.zip"
syphax-bouazzouni commented 2 months ago

All big unnecessary files have been cleaned, it remains only small unnecessary files that do not need to be cleaned for now.

image