search

LSSTDESC / WLMassMap

DESC utility for weak lensing mass-mapping
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

Implement mass-mapping pipeline in descpipe framework #14

Closed EiffL closed 4 years ago

EiffL commented 6 years ago

The mass-mapping pipeline will use a number of different mapping algorithms, each one with their own dependencies. The descpipe (https://github.com/joezuntz/descpipe) framework provides a flexible way to handle this problem by packaging each step of the pipeline in a docker container.

I open this issue to work on the implementation of the mass-mapping pipeline with descpipe.

EiffL commented 6 years ago

@joezuntz I'm done implementing the mass-mapping pipeline with descpipe. Works very nicely, but I have one small issue at NERSC. Because this is a private project, I'm hosting the docker images on the private repository at NERSC ,which means that to load the shifter images I need to use something like:

shifter --image=registry.services.nersc.gov/flanusse/descpipe-wlmassmap-mock_observation:1.0

but descpipe automatically generates a docker based command:

shifter --image=docker:flanusse/descpipe-wlmassmap-mock_observation:1.0

Right now I'm manually replacing these lines in the generated bash script, but do you think there is a way to specify to descpipe which repository to use ?

joezuntz commented 6 years ago

I've added a few notes in the code. Good point about the shifter script. The whole script generation is a total hack and it would be useful to have a quick think about how it should look for local systems and NERSC.

EiffL commented 4 years ago

I'm closing this one as we now interface with ceci and TXPipe