vosjo / speedyfit

Python package to fit single and binary photometric SEDs. Automatically pull archive photometry from the main surveys and use Bayesian inference to derive atmospheric parameters.
GNU General Public License v3.0
14 stars 4 forks source link

Install and download photometry #4

Closed AstroJLin closed 2 years ago

AstroJLin commented 3 years ago

Hi Joris,

I use speedyfit checkgrids to check whether everything went well after I install speedyfit. However, an error is given, which is not the problem of SPEEDYFIT_MODELS environmental variable. This error is as follow: Screenshot from 2021-08-28 16-59-32

In addition, I use command (speedyfit phot J183656.3+384701.2) to obtain the data of J183656.3+384701.2. Here, anther error is given, which is as follow: Screenshot from 2021-08-28 16-59-50

I don't know what caused those problem. Could you give me some suggestions?

Thanks!

vosjo commented 3 years ago

Hi,

Lets start with the fist issue. You set the SPEEDYFIT_MODELS directory to /home/jlin/speedyfit/modelgrids/ Is that the directory where you downloaded the atmosphere models? Could you check what the content is of that directory? The error you get means that the grid_description.yaml file is not present in that directory. Is it possible that when you downloaded the atmosphere models, you unzipped them in the wrong folder?

vosjo commented 3 years ago

The second issue you had is caused by the skymapper service. That service no longer supports http requests. I have updated the url for skymapper, and if you update to speedyfit v0.2.1, it should work. (This is independent of the SPEEDYFIT_MODEL directory problem that you have).

AstroJLin commented 3 years ago

Hi Joris,

Thank you very much for your reply.

The second issue has been solved after I updated the speedyfit. For the first issue, I find the grid_description.yaml file is not present in modelgrids.tar.gz, which only has eight files in the modelgrids.tar.gz.
Screenshot from 2021-08-30 18-01-32

Should this file be downloaded separately from another place?

vosjo commented 3 years ago

Hi,

It seems I forgot to include grid description file when I last updated the model grids archive. I will fix that. For now I have attached the grid_description.yaml file that you can use. Unzip the file, and add it to the SPEEDYFIT_MODELS directory.

grid_description.zip

AstroJLin commented 3 years ago

Hi Joris,

The speedyfit has been checked by speedyfit checkgrids and shows everything went well.

However, an error is given when I try to fit the script of the HIP_4618_setup_kurucz.yaml . This error is as follow: Screenshot from 2021-08-30 21-23-40

Could you explain this error?

Many thanks

vosjo commented 3 years ago

Hi,

I can indeed explain this error. It actually already says it in the output you get: passband GAIA3E.G missing from table ect... It means that the integrated photometry for the GAIA EDR3 photometric system is not available, and therefor it can't calculate anything, leading to the 'No models were accepted' error.

The reason it is not available is because I seem to have made a mistake when updating the archive file with the atmosphere models. You can download the correct files from here: https://drive.google.com/file/d/1yKHB3uB7QUyhhkwa1hNMuWazxjLkLAjN/view?usp=sharing I will also update the link in the documentation.

Hopefully this will solve all problems.

AstroJLin commented 3 years ago

Hi Joris,

Thank you very much for your help.

This is a great code. The code can run after I updated the archive file with the atmosphere models. However, some data in the photometry file is not included in the result. So I don't know what is happen?

In addition, is the data complete in the photometry file?

Many thanks

vosjo commented 3 years ago

Hi.

There are parameters in the setup file where you can set which photometry to include and which to exclude: 'photband_include' and 'photband_exclude'. By default some systems are excluded.

Just remove these parameters to include all photometry from the phot file in the fit.

AstroJLin commented 3 years ago

Hi Joris,

Thank you very much for your reply.

Here, you consider to exclude these data in the setup file, which mainly is due to these data is not good for fitting or these data is not very good compare with other data in the result?

In addition, the value of x^2 is very big when I try to fit those data with model (for example J152748 47+353657 3_sed_kurucz. So, I want to ask you how to make this value to become small by adjusting some parameters.

Thanks

vosjo commented 3 years ago

The example that is described in the documentation is just an example on how to use the code. It isn't meant to be a rigorous scientific analysis.

I can give you some pointers on what to look for when fitting SEDs. But this is going to depend a lot on what type of systems you will be working with and what you want to do with it.

In the example you have used, you have a high Chi2 value. From the figure you can see the main reasons of why that is. The SDSS observations are scattered, and the WISE W3 and W4 band also have a very bad fit. The likely reason for that is that SDSS is optimized for fainter targets, and is known to have bad results for a fairly bright star. (I am assuming that the figure you send is from HIP 4618). So in this case I would drop SDSS. The W3 and W4 bands are a different problem. For fainter stars they are often upper limits, but for HIP 4618 they are valid observations. You can check this in Vizier by looking at the snr measurements (http://vizier.u-strasbg.fr/viz-bin/VizieR-5?-ref=VIZ6131cb8334930&-out.add=.&-source=II/311/wise&WISE===J005919.53-582417.9&-out.orig=o) You can always choose to exclude those values from your analysis, but there isn't a clear reason for them being bad fits.

Chi2 values are often going to be high for SED fits, because photometric observations regularly underestimate their errors. So a high Chi2 does not necessarily mean a bad fit.

Another important thing to check when fitting, is that you haven't reached one of the parameter limits (teff, radius, ebv, ect.) that you set in the setup file. You can see that in the second figure that gets shown. If you hit a limit, you will need to change the parameter limits or if not possible us a different grid.

AstroSong commented 3 years ago

The example that is described in the documentation is just an example on how to use the code. It isn't meant to be a rigorous scientific analysis.

I can give you some pointers on what to look for when fitting SEDs. But this is going to depend a lot on what type of systems you will be working with and what you want to do with it.

In the example you have used, you have a high Chi2 value. From the figure you can see the main reasons of why that is. The SDSS observations are scattered, and the WISE W3 and W4 band also have a very bad fit. The likely reason for that is that SDSS is optimized for fainter targets, and is known to have bad results for a fairly bright star. (I am assuming that the figure you send is from HIP 4618). So in this case I would drop SDSS. The W3 and W4 bands are a different problem. For fainter stars they are often upper limits, but for HIP 4618 they are valid observations. You can check this in Vizier by looking at the snr measurements (http://vizier.u-strasbg.fr/viz-bin/VizieR-5?-ref=VIZ6131cb8334930&-out.add=.&-source=II/311/wise&WISE===J005919.53-582417.9&-out.orig=o) You can always choose to exclude those values from your analysis, but there isn't a clear reason for them being bad fits.

Chi2 values are often going to be high for SED fits, because photometric observations regularly underestimate their errors. So a high Chi2 does not necessarily mean a bad fit.

Another important thing to check when fitting, is that you haven't reached one of the parameter limits (teff, radius, ebv, ect.) that you set in the setup file. You can see that in the second figure that gets shown. If you hit a limit, you will need to change the parameter limits or if not possible us a different grid.

I think another reason for the high chi2 (except for the bad points, underestimated errors, or bad fittings) is that the fitting uses flux rather than mag, right?

vosjo commented 3 years ago

Indeed, the fit uses the fluxes.

On Fri, 3 Sep 2021, 10:01 Song Wang, @.***> wrote:

The example that is described in the documentation is just an example on how to use the code. It isn't meant to be a rigorous scientific analysis.

I can give you some pointers on what to look for when fitting SEDs. But this is going to depend a lot on what type of systems you will be working with and what you want to do with it.

In the example you have used, you have a high Chi2 value. From the figure you can see the main reasons of why that is. The SDSS observations are scattered, and the WISE W3 and W4 band also have a very bad fit. The likely reason for that is that SDSS is optimized for fainter targets, and is known to have bad results for a fairly bright star. (I am assuming that the figure you send is from HIP 4618). So in this case I would drop SDSS. The W3 and W4 bands are a different problem. For fainter stars they are often upper limits, but for HIP 4618 they are valid observations. You can check this in Vizier by looking at the snr measurements ( http://vizier.u-strasbg.fr/viz-bin/VizieR-5?-ref=VIZ6131cb8334930&-out.add=.&-source=II/311/wise&WISE===J005919.53-582417.9&-out.orig=o) You can always choose to exclude those values from your analysis, but there isn't a clear reason for them being bad fits.

Chi2 values are often going to be high for SED fits, because photometric observations regularly underestimate their errors. So a high Chi2 does not necessarily mean a bad fit.

Another important thing to check when fitting, is that you haven't reached one of the parameter limits (teff, radius, ebv, ect.) that you set in the setup file. You can see that in the second figure that gets shown. If you hit a limit, you will need to change the parameter limits or if not possible us a different grid.

I think another reason for the high chi2 (except for the bad points, underestimated errors, or bad fittings) is that the fitting uses flux rather than mag, right?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vosjo/speedyfit/issues/4#issuecomment-912341594, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASU63DWAUW5HJ7DCDL5CSDUAB6GBANCNFSM5C653DRA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

AstroJLin commented 3 years ago

Hi Joris,

Thank you very for your suggestions.

A high Chi2 value has been decreased after I drop SDSS and WISE w3/w4 band( J152748 47+353657 3_sed_kurucz ). For the WISE w4 band from a fairly bright star, it has a very low snr values (3.3), which should exclude this data with a clear reason. For the WISE w3 band, it has a very high snr values (34) so that I don't have enough reason to remove it . However, a Chi2 value would be increased when I use the WISE w3 band. Therefore, I don't allow to remove the WISE w3 band?

In addition, can we try to increase artificially their errors due to photo-metric observations regularly underestimate their errors, which may obtain a good fit.

vosjo commented 3 years ago

Hi,

That fit looks a lot better.

For the WISE W3 band. A snr of 34 should indicate a reliable measurement. What you can do is check the WISE images. Maybe there is so artifact on them. You can do that here: https://irsa.ipac.caltech.edu/applications/wise/

In addition, can we try to increase artificially their errors due to photo-metric observations regularly underestimate their errors, which may obtain a good fit.

I have spend some time thinking about this. But without re-reducing the actual photometric images, I would not know how to actually go about this.

AstroJLin commented 3 years ago

Hi Joris,

Should I how to determine that there is so artifact on them. This is the WISE W3 images (Screenshot from 2021-09-03 21-29-40.

Thanks

vosjo commented 3 years ago

Hi,

There is a clear measurement that is significantly stronger than the background, it isn't weirdly shaped, and there is no other nearby object visible that could have influenced the measurement. So I would say this is fine.

Depending on what system you are looking at, it is possible that there is dust around it. That can also explain a higher WISE W3 measurement. You can have a look at the vizier photometry viewer if there are other IR measurements available.

On Fri, 3 Sept 2021 at 15:36, AstroJLin @.***> wrote:

Hi Joris,

Should I how to determine that there is so artifact on them. This is the WISE W3 images ([image: Screenshot from 2021-09-03 21-29-40] https://user-images.githubusercontent.com/86318178/132013814-9a6edcb8-eb2f-42e1-89a5-ba28560f5b8d.png .

Thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vosjo/speedyfit/issues/4#issuecomment-912545587, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASU63EDQN2FYNSMRWR2UL3UADFMFANCNFSM5C653DRA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- m.v.g. Joris Vos

AstroJLin commented 3 years ago

Hi Joris,

Thank you very much your patience.

I have a look at the the vizier photometry viewer. There are not other IR measurements available. Can we have a reason to remove a higher WISE W3 measurement due to there may be additional contribution from dust or others.

In addition, the source have many data points (~100) within 3 arcsec in the vizier photometry viewer. However, I use the speedyfit to obtain photo-metric data. There are only 23 data points in the file. So I want to ask what is the reason?

vosjo commented 3 years ago

Hi,

No problem at all.

I have a look at the the vizier photometry viewer. There are not other IR

measurements available. Can we have a reason to remove a higher WISE W3 measurement due to there may be additional contribution from dust or others.

Sure, you can exclude wise W3. For most systems the W3 and W4 bands will not add much extra information for the star. But they can be useful to detect dust. So if there is no reason to indicate that it is a bad measurement, it is very possible that there is dust or gas in your system or around it.

In addition, the sour ce have many data points (~100) within 3 arcsec in

the vizier photometry viewer. However, I use the speedyfit to obtain photo-metric data. There are only 23 data points in the file. So I want to ask what is the reason

Speedyfit is capable to download any type of photometry that is available on vizier, as well as most catalogues that can be queried using tap. By default only the most popular are turned on, and only those have integrated photometry included in the atmosphere model files.

If you are interested in using more photometry sources. Let me know and I can provide instructions on how to set that up.

AstroJLin commented 3 years ago

Hi Joris,

Thank you very much your reply again.

Here, the most popular data have fitted pretty good, which means the physical parameters of the star have been obtained. In addition, sometimes many data points may affect the physical parameters of the star obtained by the model. Therefore, the most popular data have enough to address the physical parameters of the star. Do you agreement with my points?

vosjo commented 3 years ago

Hi,

In most cases the included photometry should be enough for a good fit. The reason to use these large surveys is that they have decent reduction pipelines and verification of the results. So they should have decent photometry. A lot of smaller catalogues are typically focused on some specific science case, and are not necessarily well suited for an SED fit. There are of course cases where extra photometry would be good. e.g. U-band photometry is very useful to constrain hotter stars. But if you have photometry from other sources or from your own observations, you can always add it manually to the photometry file.

btw, may I ask how you found out about speedyfit?

On Fri, 3 Sept 2021 at 18:14, AstroJLin @.***> wrote:

Hi Joris,

Thank you very much your reply again.

Here, the most popular data have fitted pretty good, which means the physical parameters of the star have been obtained. In addition, sometimes many data points may affect the physical parameters of the star obtained by the model. Therefore, the most popular data have enough to address the physical parameters of the star. Do you agreement with my points?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vosjo/speedyfit/issues/4#issuecomment-912655257, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASU63DAFKQEL46VNC5ODQTUADYADANCNFSM5C653DRA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- m.v.g. Joris Vos

AstroJLin commented 3 years ago

Hi Joris,

A friend told me about the speedyfit.

Thanks

AstroJLin commented 2 years ago

Hi Joris,

Thank you very much for your reply.

Here, you consider to exclude these data in the setup file, which mainly is due to these data is not good for fitting or these data is not very good compare with other data in the result?

In addition, the value of x^2 is very big when I try to fit those data with model (for example J152748 47+353657 3_sed_kurucz. So, I want to ask you how to make this value to become small by adjusting some parameters.

Thanks

Hi Joris,

Here, I want to know how to fix the Xlabel in the figure?

Thanks!