search

arquillian / arquillian-spacelift

Arquillian process and package manager. Makes executing an external process or fetching platform depends dependencies an easier job
4 stars 7 forks source link

Caching DownloadToolResult #14

Open lordofthejars opened 9 years ago

lordofthejars commented 9 years ago

DownloadTool can be an expensive operative (for example you may download a database engine (MySQL for instance) and install it. You can achive the same result by implementing by yourself, but I think it would be great that before downloading something we could check if the file is already downloaded and if it is the case skip the download. Of course we could add a flag to remove this behavior.

WDYT?

kpiwko commented 9 years ago

Absolutely. Or timestamp based check.

Side note, plenty of these ideas are already implemented in our Gradle Test Env Plugin - this one for instance here https://github.com/aerogear/aerogear-testing-tools/blob/master/aerogear-test-env-plugin/src/main/groovy/org/jboss/aerogear/test/Installation.groovy#L131-L142

I'm looking forward to push these concepts back to Spacelift. @lordofthejars if you could help on Spacelift API definitions, that would be awesome!

kpiwko commented 9 years ago

That Installation is actually mirrored by https://github.com/arquillian/arquillian-spacelift/blob/master/spacelift-api/src/main/java/org/arquillian/spacelift/installation/Installation.java in Spacelift itself.

lordofthejars commented 9 years ago

Yes I have seen this class but I think that there are only interfaces not implementation.

smiklosovic commented 9 years ago

We are not sure about how this all installation stuff should be done, it is only POC interface and it will likely change as the shape of current API is under the constant revision.

smiklosovic commented 9 years ago

If it helps temporarily, just implement easy caching on your own. I am doing the same thing when I am downloading Selendroid server for Droidium in runtime here (1). Download it to $HOME/.mongodbcache/file and check if it is present or not. If it is there, you download nothing, if it is not there, download it there and use it. The next time you look into .mongodbcache you will find it there so you can skip downloading completely.

By the way, you can see nice example of task chaining there, when you download that file with DownloadTool, process method returns you downloaded File which is put on UnzipTool's input (into its process method) automatically so you are just unzipping it instantly. Pretty slick, that is Spacelift in action ;)

(1) https://github.com/arquillian/arquillian-droidium/blob/master/droidium-native/arquillian-droidium-native/src/main/java/org/arquillian/droidium/native_/configuration/SelendroidDownloader.java#L66-L78

lordofthejars commented 9 years ago

Related with https://issues.jboss.org/browse/ARQ-1869