search

nrnapolita / KIWI_LSST

0 stars 0 forks source link

CLUSTERS and Environment definition #3

Open nrnapolita opened 6 years ago

nrnapolita commented 6 years ago

Check the match between galaxies and the cluster/group catalogs from Mario

nrnapolita commented 6 years ago

Crescenzo send the latest version of the galaxy catalogs to match with (Mario asked)

nrnapolita commented 6 years ago

let's ask for the novel 9-band photo_z catalog at the next KiDS teleconf...

ctortora commented 6 years ago

@mariorpd13 You can find the catalog on calimero, the path is /home/galattici/ctortora/

The name of the catalog is: catalog_DR2_DR3_5M_234params_FULL_2017-10-10.fits It includes the 5M galaxies with ML redshift [+ masses, struc. parameters (take the second ones), subsamples of galaxies with spec. redshifts, etc. etc.]. The name of the readme file is: README_catalogs_DR2+DR3_2017-07-11

In general, comments on the catalog/questions are welcome.

nrnapolita commented 6 years ago

Mario has got the catalog, he's waiting for the latest version of the cluster catalog. Mario wanted to have a look in the last 9-band cat and check about the issue mentioned in the last meeting (i.e. the issue on the gap in the BPZ photo_z).

KiaraSpin commented 6 years ago

Group Meeting 18/12 AI CT: Check if Mario has the latest version of the catalog! AI MR (after Crescenzo's checking): Mario will write a script to match the catalog. Trivial but time consuming -> better to use THE RIGHT CATALOG!

KiaraSpin commented 6 years ago

Group Meeting 15/01/18 We are trying to match galaxy parameters with cluster parameters to assess whether we can say if a galaxy is in a cluster or not. But what exactly do we need? For the cluster catalog, it could happen that a galaxy has a non-null probability to be member of more than one cluster (one galaxy has probability to belong to 2,3, or more cluster). What is the important info to disentangle to which cluster the galaxy really belong?

It makes no sense to use all the parameters. It will be time consuming and it will create a huge catalog. We need to pick the "best parameters" to save time and be efficient. From Crescenzo: Structure parameters in r-band (second set of parameters). Best class parameters. Crescenzo will provide more details via email.
-> AI CT: Send further info to Mario.

KiaraSpin commented 6 years ago

GM 29/01 LOW PRIORITY To come back to this after we close the issue about catalogs and parameters checking.

KiaraSpin commented 6 years ago

GM 28/02 AI @mariorpd13:
Ask the GAMA people to ask that dataset (Konrad with cc Nicola) to use them within the KiDS collaboration.

KiaraSpin commented 6 years ago

GM 14/05 Mario is currently working on the cluster paper. Working on some "issues"

  1. What do you really mean by cluster member? In most of the cases you use the membership probability in a statistical way.
  2. If you have a galaxy belonging to more than one cluster, what do you do?
KiaraSpin commented 6 years ago

GM 21/05 @mariorpd13 Do you have updates? Nicola would like to coordinate with you about the talk(s) at the VLT Surveys meeting

mariorpd13 commented 6 years ago

I implemented what discussed last week: for each galaxy I assign a cluster membership if defined, if there is more than one "parent cluster" I assign the highest available probablity. The script is running, but it is not finished yet.

On 21/05/2018 14:54, KiaraSpin wrote:

GM 21/05 @mariorpd13 https://github.com/mariorpd13 Do you have updates? Nicola would like to coordinate with you about the talk(s) at the VLT Surveys meeting

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-390645534, or mute the thread https://github.com/notifications/unsubscribe-auth/AOR17TV0gfvxK64EggjJoOtFb19qGSn-ks5t0rjvgaJpZM4QlFid.

nrnapolita commented 6 years ago

On 21/05/18 17:27, mariorpd13 wrote:

I implemented what discussed last week: for each galaxy I assign a cluster membership if defined, if there is more than one "parent cluster" I assign the highest available probablity. The script is running, but it is not finished yet.

great!

NRN

On 21/05/2018 14:54, KiaraSpin wrote:

GM 21/05 @mariorpd13 https://github.com/mariorpd13 Do you have updates? Nicola would like to coordinate with you about the talk(s) at the VLT Surveys meeting

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub

https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-390645534,

or mute the thread

https://github.com/notifications/unsubscribe-auth/AOR17TV0gfvxK64EggjJoOtFb19qGSn-ks5t0rjvgaJpZM4QlFid.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-390688762, or mute the thread https://github.com/notifications/unsubscribe-auth/AXhvYRON2_NWTRQZ9rhFAtB_OR4Y_yNcks5t0tzsgaJpZM4QlFid.

-- Dr. Nicola R. Napolitano Staff Astronomer INAF - Astronomical Observatory of Capodimonte Salita Moiariello, 16, 80131, Napoli (Italy) phone: +39 081 5575509; fax: +39 081 5575457 email: napolita@na.astro.it — Skype: nnapolita web page: http://www.na.astro.it/~napolita

KiaraSpin commented 6 years ago

GM 18/07 Mario and "cluster people (Bellagamba et al.) are working on the second paper. BCG selection is going to go in paper III, with something about Crescenzo's catalog included. As soon as Mario has a final catalog (some cleaning still ongoing) he will share it with us. He has :

  1. "potential BCG" matched to Crescenzo's catalog
  2. membership probability assigned to each of Crescenzo's catalog (0<z<0.8) We need to understand how to use the probability (e.g. if you have a galaxy with a probability of 10% of being member of a cluster... do you call it "field" galaxy or do you associate it to cluster?) BUT things can change!!!

AI Mario: Share with the group the preliminary catalog. In this way the group can start working on routines and procedures to work with the catalog once it will be definitive.

mariorpd13 commented 6 years ago

I set up a google team drive with the catalog defining the cluster membership probabilities. Please send me the google emails (@inaf.it or @gmail.com) to define the team drive members

nrnapolita commented 6 years ago

I'm nrnapolita@gmail.com

Thanks a lot!

Nicola

On 19/07/18 18:43, mariorpd13 wrote:

I set up a google team drive with the catalog defining the cluster membership probabilities. Please send me the google emails (@inaf.it or @gmail.com) to define the team drive members

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-406341492, or mute the thread https://github.com/notifications/unsubscribe-auth/AXhvYT91EWqtOd5Levxry2Zz039PM61gks5uILdHgaJpZM4QlFid.

-- Dr. Nicola R. Napolitano Staff Astronomer INAF - Astronomical Observatory of Capodimonte Salita Moiariello, 16, 80131, Napoli (Italy) phone: +39 081 5575509; fax: +39 081 5575457 email: napolita@na.astro.it — Skype: nnapolita web page: http://www.na.astro.it/~napolita

mariorpd13 commented 6 years ago

P.S. if you have it, please send me the INAF email (@inaf.it). Up to now I added @nrnapolita and @KiaraSpin, I did not find it for @ctortora

FedorGetman commented 6 years ago

fedor.i.getman@gmail.com or fedor.getman@inaf.it

On Fri, Jul 20, 2018 at 12:12 PM, mariorpd13 notifications@github.com wrote:

P.S. if you have it, please send me the INAF email (@inaf.it). Up to now I added @nrnapolita https://github.com/nrnapolita and @KiaraSpin https://github.com/KiaraSpin, I did not find it for @ctortora https://github.com/ctortora

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-406556328, or mute the thread https://github.com/notifications/unsubscribe-auth/AgUA5rHYG-DjdqSsfm9p4WBEXjMb9Oizks5uIa0WgaJpZM4QlFid .

KiaraSpin commented 6 years ago

GM 10/09 We could join the "cluster catalog" with the velocity dispersion catalog that Peppe produced.

KiaraSpin commented 5 years ago

GM 15/10 No update for KIDS1000 (DR4) AI NICOLA check the structural parameters

mariorpd13 commented 5 years ago

MR 25/10 I made a catalog of galaxies with cluster membership probabilities WITHOUT structure parameter measurements. The catalog is in the google drive team, Nivya will start to work on it.

KiaraSpin commented 5 years ago

GM 03/12 Nivya is working on getting the structural parameters of the cluster galaxies. She will update us soon.

nrnapolita commented 5 years ago

Nivya has completed a first 2DPHOT run but found that there where missing galaxies in her final catalogs. Mario told he did not use flags for his cluster catalog and there are a lot of sources that might fall into masked regions I checked that, see figure below

mariorpd13 commented 5 years ago

Missing figure ... ?

    Mario

On 16/01/19 08:04, Nicola R. Napolitano wrote:

Nivya has completed a first 2DPHOT run but found that there where missing galaxies in her final catalogs. Mario told he did not use flags for his cluster catalog and there are a lot of sources that might fall into masked regions I checked that, see figure below

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-454674130, or mute the thread https://github.com/notifications/unsubscribe-auth/AOR17RtnqzQG0Yzi1QUfhBUleZbk6drxks5vDs73gaJpZM4QlFid.

nrnapolita commented 5 years ago

missing_galaxies_in_clusters

On the right image a smal section of Mario catalog overplotted over the KiDS image (129.0_-0.5, rband), on the left, in red is Mario sources and in blue the 2DPHOT output sources for which 2DPHOT can go and measure parameters.

Hence, the missing galaxies are in the masked regions and as Nivya is using masks when running 2DPHOT, for some reason the output is set in a way it automatically takes off all the flagged sources (e.g. imaflag_iso=0). Nivya is going to run 2DPHOT again with no masks. @MariaAngelaRaj might possibly help to run this quicker from Naples.

The whole process will produce a golden catalog (the one already obtained) with clean structural parameters as no flag applies to this galaxies. A lower quality catalog where there will be all galaxies, but the structural parameters might be affected by local poor background.

nrnapolita commented 5 years ago

please all download the file because the link will be removed in 1h

nrnapolita commented 5 years ago

@mariorpd13 @ctortora do we have masses of these galaxies? we also need all other info to match with. E.g colours, masses, redshift and their distance from the high probablility cluster. Or have a catalog with all these information which we can play

ctortora commented 5 years ago

Hi,

If they are in DR3 I guess yes. I will check.

I forgot to tell you that I cannot join the teleconf today, since I am travelling.

Cheers,

           C

Il 28/01/2019 09:38, Nicola R. Napolitano ha scritto:

@mariorpd13 https://github.com/mariorpd13 @ctortora https://github.com/ctortora do we have masses of these galaxies? we also need all other info to match with. E.g colours, masses, redshift and their distance from the high probablility cluster. Or have a catalog with all these information which we can play

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-458039644, or mute the thread https://github.com/notifications/unsubscribe-auth/AgUtQs-LYUDNgle9yNIlztfhllNF_LS_ks5vHrcBgaJpZM4QlFid.

--

Dr. Crescenzo Tortora

/Post-Doc/ INAF - Osservatorio Astrofisico di Arcetri Largo Enrico Fermi 5 I - 50125 Firenze (ITALY)

office phone number: +39 0552752242 e-mail: ctortora@arcetri.astro.it web-page: http://www.arcetri.inaf.it/~ctortora/ http://www.arcetri.inaf.it/%7Ectortora/

nrnapolita commented 5 years ago

On 28 Jan 2019, at 09:45, ctortora notifications@github.com wrote:

Hi,

If they are in DR3 I guess yes. I will check.

I forgot to tell you that I cannot join the teleconf today, since I am travelling.

ok, no problem

N

Cheers,

       C

Il 28/01/2019 09:38, Nicola R. Napolitano ha scritto:

@mariorpd13 https://github.com/mariorpd13 @ctortora https://github.com/ctortora do we have masses of these galaxies? we also need all other info to match with. E.g colours, masses, redshift and their distance from the high probablility cluster. Or have a catalog with all these information which we can play

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-458039644, or mute the thread https://github.com/notifications/unsubscribe-auth/AgUtQs-LYUDNgle9yNIlztfhllNF_LS_ks5vHrcBgaJpZM4QlFid.

--

Dr. Crescenzo Tortora

/Post-Doc/ INAF - Osservatorio Astrofisico di Arcetri Largo Enrico Fermi 5 I - 50125 Firenze (ITALY)

office phone number: +39 0552752242 e-mail: ctortora@arcetri.astro.it web-page: http://www.arcetri.inaf.it/~ctortora/ http://www.arcetri.inaf.it/%7Ectortora/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nrnapolita/KIWI_LSST/issues/3#issuecomment-458041697, or mute the thread https://github.com/notifications/unsubscribe-auth/AXhvYaAH_AwERJ3LNtGCqE2Ly-rgLgeMks5vHrjFgaJpZM4QlFid.

Prof. Nicola R. Napolitano School of Physics and Astronomy Sun Yat-sen University, Guangzhou, Zhuhai Campus 2 Daxue Road, Tangjia, Zhuhai, Guangdong 519082, P.R. China
Office 215, Building #18, Skype/WeChat: nrnapolita

本邮件及其附件含有发送给特定个人和用于特定目的的保密信息。如果您不是预期的收件人，请立即删除本邮件并通知发件人。严禁任何非预期的收件人使用、传播、分发或复制本邮件或其附件。 This email and its attachments may contain confidential information intended for a specific individual and purpose. If you are not the intended recipient, you should delete this email and notify the sender immediately. Any use, dissemination, distribution, or copying of this email or its attachments by persons other than the intended recipient(s), is strictly prohibited.

mariorpd13 commented 5 years ago

Hi, I have two questions:

  1. ~ 5000 galaxies are actually duplicated galaxies, probably from overlapping tiles: what is the best value to take ?

2.  I overplotted (https://user-images.githubusercontent.com/14972397/51840230-c9470d00-230b-11e9-8541-82e78719debe.png) the "full" members' catalog (~ 250k galaxies, in blue) and Nivya's catalog (~ 90k galaxies, in red). Precisely, red points are galaxies in both catalogs, blue points are galaxies in the input catalog, but not in Nivya's catalog. So, it seems that missing galaxies are not only due to flagging. What do you think ?

mariorpd13 commented 5 years ago

I updated the comment in the issues and added the plot

   Mario

nrnapolita commented 5 years ago

Thanks Mario. It seems to me that the mask problem is a minor effect, indeed. I propose that @MariaAngelaRaj starts the new 2DPHOT run with no masks and we will see what's going on. Maybe we can check on a small area whether the problel persists. @nivyaaroy please get in touch with Angela.

mariorpd13 commented 5 years ago

@nivyaaroy, is it possible that the problems we see in the results are still due to the issue that we fixed some time ago ? In the split_xxx script, there was a line delhead, that corrupted the fits file and produced random results. Can you check ? Thanks !