Source Finding Container

[foxmouldy]

Software for a source finding container:

• [x] AEGEAN, https://github.com/PaulHancock/Aegean/wiki/Simple-usage

• [x] PYBDSF, http://www.astron.nl/citt/pybdsm/, also available via LOFAR.

• [x] ASTROPY, http://www.astropy.org

• [x] Python 2

• [x] Matplotlib, Numpy, Scipy, Pylab

• [x] Sourcery, https://github.com/radio-astro/sourcery

• [x] CASAPY, (via KERN-2)

• [x] Pyfits,https://pythonhosted.org/pyfits/

• [x] KARMA/KVIS, KERN2, http://www.ras.ucalgary.ca/karma/

• [ ] SOFIA, https://github.com/SoFiA-Admin/SoFiA.

• [x] DS9, http://ds9.si.edu/site/Home.html

[mattia-vaccari]

NOTE TO LIST ABOVE : The latest PyBDSF (formerly PyBDSM) is available at http://www.astron.nl/citt/pybdsf/ and/or https://github.com/lofar-astron/PyBDSF

Below I list the extra software required for multi-wavelength data fusion and machine learning work.

We may want to sit down and see whether our requirements are worth combining into a single VM.

• [x] java runtime environment ( for e.g. aladin / stilts / topcat )

• [x] python modules (for data inspection and machine learning - 2.7 and 3.* versions) : pandas / scikit-learn / astroML / astroML_addons / statsmodels / aplpy / seaborn / ginga / glueviz / hfpy / healpy / hips / pymoc / pyregion / astroquery / pyvo / https://github.com/JohannesBuchner/nway / https://github.com/kbarbary/sep / theano / tensorflow / keras / https://mxnet.incubator.apache.org

• [x] https://github.com/sciserver/SciScript-Python / https://github.com/peterwittek/somoclu / https://github.com/scikit-image/scikit-image / https://github.com/reiinakano/scikit-plot / https://github.com/cgevans/scikits-bootstrap / https://github.com/statsmodels/statsmodels

• [x] https://github.com/dfm/emcee / https://github.com/JohannesBuchner/PyMultiNest / https://github.com/pymc-devs/pymc3 / https://pypi.python.org/pypi/corner

• [ ] apache spark mlib https://spark.apache.org/mllib/ [Nice to have but not required at this stage]

• [x] image mosaicing & source extraction software : scamp, sextractor, swarp at https://www.astromatic.net

• [x] forced photometry software - http://thetractor.org

• [x] photometric redshift software, e.g. GPz - https://github.com/OxfordML/GPz

• [ ] https://github.com/IftachSadeh/ANNZ

• [ ] Hyperz & LePhare ?

• [x] https://github.com/gbrammer/eazy-photoz

• [x] SED fitting software, e.g. CIGALE : http://cigale.lam.fr

• [ ] MagPhys : http://www.iap.fr/magphys/ [nice to have]

• [ ] http://steatreb.altervista.org/alterpages/sed3fit.html [nice to have]

• [x] montage : https://github.com/Caltech-IPAC/Montage and montage-wrapper : http://www.astropy.org/montage-wrapper/

• [ ] https://github.com/GalSim-developers/GalSim

• [ ] LSST Science Pipelines - https://pipelines.lsst.io

• [x] PINK Source Classification Software (Proprietary - Ask Mattia!) - NB : runs on GPUs! PINK was installed in a separate container. However, we may need to look at the latest software version at some point!

• [x] Create a PINK container running on singularity 2.4 using the pink-2018 code base

• [ ] Incorporate PINK within the source finding container. This would be very nice to have but has been tried before (unsuccessfully).

• [x] astropy affiliated packages (http://www.astropy.org/affiliated/index.html) : astroquery / astroplan / ccdproc / ginga / glueviz / photutils / pydl / pyregion / regions / reproject / specutils

• [ ] https://users.obs.carnegiescience.edu/peng/work/galfit/galfit.html [nice to have]

• [x] BlobCat Source Finding Software : http://blobcat.sourceforge.net [nice to have]

[Shadowphax]

Mattia, when would you like to meet ? We can certainly do this. I would like to see everything combined into a single container instead of at a VM level. It would be best to decouple the applications from the VM to keep the VM as clean as possible.

[mattia-vaccari]

Hi Tim, happy to follow your advice about the Container / VM boundaries.

Perhaps we can get together (also with Brad) next week?

[Shadowphax]

Problem compiling SoFIA... need to come back this after compiling Mattia's list of software. Will consider troubleshooting via a separate container.

usr/local/lib/python2.7/dist-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]

warning "Using deprecated NumPy API, disable it by " \

^ In file included from /usr/include/c++/5/string:52:0 from /usr/include/c++/5/bits/locale_classes.h:40, from /usr/include/c++/5/bits/ios_base.h:41, from /usr/include/c++/5/ios:42, from src/linker/linker.cpp:242: /usr/include/c++/5/bits/basic_string.h:5346:2: error: #endif without #if

endif / C++11 && _GLIBCXX_USE_C99 ... /

^ /usr/include/c++/5/bits/basic_string.h:5466:2: error: #endif without #if

endif / _BASIC_STRING_H /

^ In file included from /usr/include/c++/5/bits/stl_algobase.h:59:0, from /usr/include/c++/5/vector:60, from src/linker/linker.cpp:241: /usr/include/c++/5/bits/basic_string.h:5342:1: error: expected declaration before '}' token _GLIBCXX_END_NAMESPACE_CXX11 ^ error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

[mattia-vaccari]

did we get around to install pink? (see also my e-mail to tim on July 31st).

i also wrote paolo serra about the sofia compile error and i've pointed him to this thread.

[Shadowphax]

@mattia-vaccari, i've started the install this morning and will provide you feedback soon.

[mattia-vaccari]

thanks tim, please let me know, particularly about pink.

On 10 Aug 2017, at 09:44, Timothy Carr notifications@github.com wrote:

@mattia-vaccari https://github.com/mattia-vaccari, i've started the install this morning and will provide you feedback soon.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AfricanResearchCloud/idia-containers/issues/4#issuecomment-321476925, or mute the thread https://github.com/notifications/unsubscribe-auth/AG5o3yebWGetzyvapK0hQhvu1J5AVT9nks5sWrTJgaJpZM4OcSn6.