Add Divand batch tool

[Marie59]

Something similar to divand interactive tool (but just a small part of it). The aim here is to create a climatology and have the workflow oceans variables more automatic.

TO DO :

• [ ] Add descrition on the kind of outputs and inputs netcdf

[Marie59]

I am also creating this PR because I need a bit of help .... For the planemo test I have the following error

Job in error state.. tool_id: divand_full_analysis, exit_code: 1, stderr: ERROR: LoadError: SystemError: opening file "/usr/local/share/julia/logs/manifest_usage.toml": Permission denied

Any ideas why ?

[bgruening]

Mh, looks like something tries to install something during runtime which is not supposed to happen.

google suggest adding JULIA_DEPOT_PATH https://stackoverflow.com/questions/68366538/change-where-julia-is-storing-logs-manifest-usage-toml

[Marie59]

Mh, looks like something tries to install something during runtime which is not supposed to happen.

google suggest adding JULIA_DEPOT_PATH https://stackoverflow.com/questions/68366538/change-where-julia-is-storing-logs-manifest-usage-toml

I tried to add something like this: `

/usr/local/share/julia

</environment_variables>`

And I tested multiple unsuccessful things... Would you have any other advices tha I could use to test new things ? (I am lacking Ideas at the moment)

[bgruening]

If you start the container locally, without Galaxy, does it work? If it does work, under which user are the processes started?

[Marie59]

hmmm it's not an interactive tool. I am not launching a container. And the script is working under planemo s. It's just when I launch planemo t --biocontainers that it fails

[bgruening]

But can you run this script inside the container, just without Galaxy?

Btw. it will also fail if you use planemo s --biocontainers I think.

[Marie59]

indeed it's not working with this option planemo s --biocontainers

uh I am not sure how I should do that to launch the container and run the script in it .... I am a bit lost there isn't the container created by galax with the right dependencies when I launch the tool ?

[Marie59]

Okay @bgruening so I tried the following:

1. Build a docker with a dockerfile containing this

   
   FROM julia

Create execution user/group

RUN \ echo "Creation group ${GROUPNAME}:${GROUPID}" && \ addgroup --gid 1001 juliagroup && \ echo "Creatin user ${USERNAME}:${USERID}" && \ adduser --uid 504427 --gid 1001 --home /home/marie marie

USER marie

RUN JULIA_PACKAGES="DIVAnd" \ julia --eval 'using Pkg; Pkg.add(split(ENV["JULIA_PACKAGES"]))'

2. Then I run it like this
 `docker run -it --rm -v "$PWD":/app -u 504427:1001 -w /app julia:test julia divandfull.jl test-data/data_from_Eutrophication_Med_profiles_2022_unrestricted.nc '19.0' '30.0' '32.0' '38.0' '19500101' '20171231' 'phosphate' '0' '100'
`
4. It's a success !
5.  One thing I notice when doing that like that I don't have anything in "/usr/local/share/" so my script is not writing anything there so this is related to something in Galaxy I think .... (as a reminder my error is `ERROR: LoadError: SystemError: opening file "/usr/local/share/julia/logs/manifest_usage.toml": Permission denied`)

Is this helpfull ?

[bgruening]

You don't need to creat a new container. Galaxy has done that for you already: mulled-v2-26583c1e86681aa6c5c517a2bf9c9c54aaedcde6:a25cf5282cc7c9ead800e7a856672df6e4631a1c

Try:

docker run -it --rm -v "$PWD":/app -w /app mulled-v2-26583c1e86681aa6c5c517a2bf9c9c54aaedcde6:a25cf5282cc7c9ead800e7a856672df6e4631a1c julia divandfull.jl test-data/data_from_Eutrophication_Med_profiles_2022_unrestricted.nc '19.0' '30.0' '32.0' '38.0' '19500101' '20171231' 'phosphate' '0' '100' 

[bgruening]

One thing I notice when doing that like that I don't have anything in "/usr/local/share/" so my script is not writing anything there so this is related to something in Galaxy I think .... (as a reminder my error is ERROR: LoadError: SystemError: opening file "/usr/local/share/julia/logs/manifest_usage.toml": Permission denied)

It can not be Galaxy. Why should Galaxy write into a julia/logs directory?

[bgruening]

(planemo) bag@bag:~/projects/code/tools-ecology/tools/ocean$ docker run --rm -i -t quay.io/local/mulled-v2-26583c1e86681aa6c5c517a2bf9c9c54aaedcde6:a25cf5282cc7c9ead800e7a856672df6e4631a1c-0 bash root@3e4838f19198:/# ls -l /usr/local/share/julia/logs/ total 8 -rw-r--r-- 1 root root 1441 Jul 23 19:54 artifact_usage.toml -rw-r--r-- 1 root root 282 Jul 23 19:54 manifest_usage.toml

[bgruening]

I think it is working now, but your tests are not valid I think and your Julia tool fails. See my last commits.

[Marie59]

Ah great ! Yeah it seems normal that my tests are not valid I still needed to really test them ;) Thank youuuuuu !!!

[bgruening]

ERROR: LoadError: MethodError: reducing over an empty collection is not allowed; consider supplying init to the reducer

What a cryptic error message.

[Marie59]

ERROR: LoadError: MethodError: reducing over an empty collection is not allowed; consider supplying init to the reducer

What a cryptic error message.

I agree I am struggling with it ... especiialy that it's working with planemo s nd not t ...

[bgruening]

especiialy that it's working with planemo s nd not t ...

that does not make sense. Are you running both with --biocontainers?

[Marie59]

especiialy that it's working with planemo s nd not t ...

that does not make sense. Are you running both with --biocontainers?

Yes I use : planemo s --biocontainers --galaxy_root ~/GxIT/galaxy/

and

planemo t --biocontainers --galaxy_root ~/GxIT/galaxy/ divandfull.xml

[Marie59]

@bgruening I am waiting for an answer of some julia expert see if he has an idea here ...

[Marie59]

Okay so, This is now working ! @bgruening do you think the tests are enough like that ? Should I try to reduce the size of the test files or can I keep them like that ?

[bgruening]

Is it possible to reduce the size? Then please do.

[Marie59]

I don't think I can do better than that .. I don't know how to reduce the last file without losing necessary info

[bgruening]

Ok, then please use location https://docs.galaxyproject.org/en/latest/dev/schema.html#id104

[Marie59]

OOOh ? Can I start my happy dance ?

[bgruening]