kirxkirx / vast

Variability Search Toolkit (VaST)
http://scan.sai.msu.ru/vast/
GNU General Public License v3.0
13 stars 4 forks source link

Usage of rotated reference frame #7

Open mrosseel opened 4 years ago

mrosseel commented 4 years ago

I take the reference frame chosen by VaST in 'vast_summary.log' and perform plate solving using astrometry.net.

Up till now, when I overlay this reference frame with the wcs transformed x/y coordinates on the ref frame (given by VaST), they match perfectly.

Now I have the case where the rotation of the reference frame chosen by VaST is 180.3 degrees as found in 'vast_image_details.log'.

==> there is no more a match between wcs transformed xy coordinates and the reference frame. Not when I plate-solve the original ref frame, nor when I rotate the reference frame by 180.3 degrees and plate solve that. Is there some translation I need to take into account or something else I misunderstood?

PS: we found our first 17 new variable stars, I'll send you the link by mail

kirxkirx commented 4 years ago

The rotation angle reported in 'vast_image_details.log' is the relative rotation of the current frame with respect to the reference frame (not the North direction - VaST knows nothing about it when matching images based on pixel coordinates of stars). The rotation angle reported in 'vast_image_details.log' for the reference frame is always 'rotation= 0.000'. If the reported rotation is even slightly different - this image is not what VaST used as the reference frame (same filename in a different directory?).

By default, VaST is using the first image it finds as the reference. This might be the first image specified on the command line, in the 'vast_list_of_input_images_with_time_corrections.txt' file or the first file in a directory (if a directory containing images was specified on the command line). This usually works fine as the practice shows that the good reference image foe VaST is the most typical image for the given data set rather than the best one (in terms of depth or seeing). The trick with specifying the directory rather than the list of files on the command line is that the files are written in the directory in quasi-random rather than in the alphabetical order (they are often displayed in the alphabetical order by 'ls' or file manager). So if you copy the same directory to another computer the file order in the directory (as it appears to VaST) might be different and VaST might choose a different reference image! The best way to avoid this is to explicitly specify the reference image as the first one on the command line, followed by the directory containing all the images.

VaST has also the option to try to automatically select a good reference image when started with the '--autoselectrefimage' option, but the selection algorithm is rudimentary and I was not sufficiently happy with the result to use this parameter in practice. Definitely there is a room for improvement here!

mrosseel commented 4 years ago

In most cases this value is indeed 0.0, but I encountered one case where it was 180 (see below). I can check that the rotation should be 0.0 but still would be interesting to know why it failed in this case.

(.venv) j@bt555:~/data/wwcra/vast/wwcra2014_aperture$ cat vast_summary.log
Images processed 7349
Images used for photometry 7332
Ref.  image: 2456845.67804 07.07.2014 04:16:08   ../data/wwcra/cleaned/2014/WWCrA#30V_000452143_FLAT.fit
First image: 2456817.58259 09.06.2014 01:58:41   ../data/wwcra/cleaned/2014/WWCrA#30V_000435001_FLAT.fit
Last  image: 2456921.53119 21.09.2014 00:44:40   ../data/wwcra/cleaned/2014/WWCrA#30V_000016123_FLAT.fit
JD time system (TT/UTC/UNKNOWN): UTC
Magnitude-Size filter: Disabled
For each source choose aperture with the smallest scatter: YES
Photometric errors rescaling: YES
Number of identified bad images: 30
Number of SysRem iterations: 0
Computation time: 24720 seconds
Number of parallel threads: 8
SExtractor parameter file: default.sex
Total objects detected (at least 2 times):  22869
Objects passed selection criteria:  12087
Measurements per detected object (mean, median, min, max):  298.6   62.0    2 1035
Measurements per selected object (mean, median, min, max):  557.8  563.0   40 1035
Average stars detected per image: 6528.05
Average stars matched: 6519.91 (99.8753 %)
Memory usage VmPeak:     1499496 kB
Software: VaST 1.0rc85 compiled with gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
VaST compiled on  Wed Oct  9 07:18:04 UTC 2019
SExtractor version 2.19.5 (2019-10-09)
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
Processing completed on  Tue Nov 12 20:36:53 UTC 2019
Host name: ns313855
(.venv) j@bt555:~/data/wwcra/vast/wwcra2014_aperture$ cat vast_image_details.log | grep WWCrA#30V_000452143_FLAT.fit
exp_start= 07.07.2014 04:16:08  exp=   30  JD= 2456845.67804  ap=  5.3  rotation= 180.381  *detected=  7798  *matched=  7795  status=OK     ../data/wwcra/cleaned/2014/WWCrA#30V_000452143_FLAT.fit
kirxkirx commented 4 years ago

This issue remains a total mystery for me. I've added tests searching for the reference frame to appear in vast_image_details.log with rotation that is not 0.0 or 180.0 degrees, but so far was not able to reproduce the situation.

mrosseel commented 2 years ago

hi, it happened again, let me know if there are any tests I can do to help debug this. Otherwise I'll just pick a non-rotated frame and use that as reference for another vast run.

bla@bla.net  /data/data/TXCar/vast/txcar-2021-2022  cat vast_summary.log
Images processed 2601
Images used for photometry 2558
Ref.  image: 2459667.82564 29.03.2022 07:48:40   /data/data/TXCar/2021-2022/TXCar_30V_001252384_FLAT.fit
First image: 2459253.74946 08.02.2021 05:58:50   /data/data/TXCar/2021-2022/TXCar_45V_000999873_FLAT.fit
Last  image: 2459700.70403 01.05.2022 04:53:33   /data/data/TXCar/2021-2022/TXCar_30V_001275648_FLAT.fit
Total objects detected (at least 2 times):  85805
Objects passed selection criteria:  0
Measurements per detected object (mean, median, min, max): 1560.9   29.0    0 17043
Measurements per selected object (mean, median, min, max):    0.0    0.0    0    0
Average stars detected per image: 7859.71
Average stars matched: 7856.79 (99.9628 %)
Memory usage VmPeak:     1466316 kB
Software: VaST 1.0rc86 compiled with gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
VaST compiled on  Wed Apr 27 14:49:55 CEST 2022
VaST build number  1.0rc85-348-g21d18a3
SExtractor version 2.19.5 (2022-04-27)
GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
Processing completed on  Fri May  6 19:49:35 CEST 2022
Host name: bla.bla.net
JD time system (TT/UTC/UNKNOWN): UTC
Magnitude-Size filter: Disabled
For each source choose aperture with the smallest scatter: NO
Photometric errors rescaling: NO
Number of identified bad images: 1373
Number of SysRem iterations: 0
Computation time: 111878 seconds
Number of parallel threads: 6
SExtractor parameter file: default.sex
Total objects detected (at least 2 times):  82638
Objects passed selection criteria:  0
Measurements per detected object (mean, median, min, max): 1499.8   28.0    2 14587
Measurements per selected object (mean, median, min, max):    0.0    0.0    2    2
Average stars detected per image: 7859.71
Average stars matched: 7856.79 (99.9628 %)
Memory usage VmPeak:     1466316 kB
Software: VaST 1.0rc86 compiled with gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
VaST compiled on  Wed Apr 27 14:49:55 CEST 2022
VaST build number  1.0rc85-348-g21d18a3
SExtractor version 2.19.5 (2022-04-27)
GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
Processing completed on  Sat May  7 00:26:37 CEST 2022
Host name: bla.bla.net
bla@bla  /data/data/TXCar/vast/txcar-2021-2022  cat vast_image_details.log | grep /data/data/TXCar/2021-2022/TXCar_30V_001252384_FLAT.fit
exp_start= 29.03.2022 07:48:40  exp=   30  JD= 2459667.82564  ap=  6.3  rotation= 179.407  *detected=  7068  *matched=  7067  status=OK     /data/data/TXCar/2021-2022/TXCar_30V_001252384_FLAT.fit
kirxkirx commented 2 years ago

Thanks for the update. The issue still a mystery! The tests I've added (named nonzero_ref_frame_rotation) in the main testing script util/examples/test_vast.sh that should catch a rotated reference image were not triggered once on any of the testing machines. If possible, can you please send me all vast*.log files from a run showing the reference image rotation? (My e-mail is kirx@kirx.net in case it would be more convenient way for sending the files.)