search

OpenCS-ontology / OpenCS

Main OpenCS ontology repository
Creative Commons Attribution 4.0 International
5 stars 7 forks source link

Consider RAM-backed storage when combining the ontology #43

Open niegrzybkowski opened 8 months ago

niegrzybkowski commented 8 months ago

Combining all the components of the ontology into one file is an operation that requires quite a lot of I/O operations. I think it might be possible do speed it up, if the repository were to be cloned into a tmpfs, which is RAM-backed, and should theoretically handle a large amount of small files better. That being said, the SSDs used for runners in Github actions seem fairly fast, so it would have to be analysed. Also make sure the runner has enough RAM.

Someone asked about it on SO: https://stackoverflow.com/questions/75726151/is-it-possible-to-use-tmpfs-on-a-github-workflow-service

Ostrzyciel commented 2 months ago

This is kind of related to #42 – if we had it as custom code in Java, we could better control I/O operations.

Another thing to consider is that the runner is on Linux, and Linux already does file caching in RAM... so when we clone the repo, it's most likely still in RAM, so read access to it should be very fast. We'd have to carefully consider if a RAM disk would help anything here.