jpjones76 / SeisIO.jl

Julia language support for geophysical time series data
http://seisio.readthedocs.org
Other
47 stars 21 forks source link

Transition test file download from SVN to Base.download? #90

Open tclements opened 2 years ago

tclements commented 2 years ago

I am not able to test SeisIO on my work computer because I don't have permission to install svn. A solution to this would be to change the tests to use Base.download. Downloading using Base.download should work on all systems since Julia v1.6 -> https://julialang.org/blog/2021/03/julia-1.6-highlights/#downloads_networkingoptions. Here is a simple, albeit hackish, workaround for downloading the test files without svn:

using HTTP 
using JSON 
using SeisIO 

function get_sample_files()
    sampledir = dirname(pathof(SeisIO))*"/../test/SampleFiles"
    isdir(sampledir) && return nothing 

    # create directory stucture 
    mkpath(sampledir)

    # get test data file structure using Github API  
    r = HTTP.get("https://api.github.com/repos/jpjones76/SeisIO-TestData/git/trees/main?recursive=1",require_ssl_verification = false)
    j = JSON.parse(String(r.body))
    files = [j["tree"][ii]["path"] for ii in 1:length(j["tree"])]
    samplefiles = [file for file in files if occursin("SampleFiles/",file) && (occursin(".", file) || occursin("SampleFiles/UW/", file) || occursin("SampleFiles/Bottle/", file))]
    infiles = [joinpath("https://raw.githubusercontent.com/jpjones76/SeisIO-TestData/main/",file) for file in samplefiles]
    outfiles = [joinpath(dirname(sampledir),file) for file in samplefiles]

    # create directories 
    sampledirs = unique([dirname(file) for file in outfiles])
    mkpath.(sampledirs)

    # download 
    for ii in 1:length(infiles)
        download(infiles[ii],outfiles[ii])
    end
    return nothing 
end

This introduces a new JSON dependency but that is pretty lightweight. We could then change https://github.com/jpjones76/SeisIO.jl/blob/main/test/runtests.jl#L7-#L10 to:

cd(dirname(pathof(SeisIO))*"/../test")
if isdir("SampleFiles") == false
    get_sample_files()
end

The other option would be to use DataDeps.jl. Using DataDeps.jl would involve a little bit more work but I like that solution because test data would be stored once in ~/.julia/datadeps/ rather than in the folder for each version of SeisIO (~/.julia/packages/SeisIO/version/..).

I'm happy to submit a PR on this if it seems helpful.

tclements commented 2 years ago

I had to make a few edits to the get_sample_files above working for all the test files but can confirm that tests are passing on ubuntu 20.04 with Julia v1.7.0 and SeisIO v1.2.0 without svn.

jpjones76 commented 2 years ago

I'm transitioning this as a priority. svn is a real problem. I thought it was a standard package in most Linux distributions, and wowwww, no, I was very wrong, there. This, along with the update issue (#85 ), will be a bugfix patch as soon as I get GitHub Actions testing fully operational. Thank you for this recommendation!