search

dstndstn / astrometry.net

Astrometry.net -- automatic recognition of astronomical images
http://astrometry.net
Other
662 stars 186 forks source link

backend: low cpu usage (10%) , and image not solving? #162

Open nevion opened 5 years ago

nevion commented 5 years ago

I'm using the local astronomy.net solver packaged by https://adgsoftware.com/ansvr/

I was trying to plate solve an image via prism and the solver doesn't seem to work.

ansvr_2019-06-16_153805.log

One of the things I find weird is backend.exe sticks around well after field-solver, and it consumes 10% cpu. In the log I see the poll loop for solution checks. I don't see any other processes hanging around.

Downsampling by factors of 2, 8, and 16 produce the issue still (8x speeds up nova solve to 3 seconds)

Image: https://drive.google.com/open?id=1L7__LAoRGw4KPxnJVfD4b4BqHks1Dd1o

Tried it on nova.astronomy.net with as far as I can tell same settings involved:

http://nova.astrometry.net/status/2749196

  B =            0           0 -8.1444e-08
                 0 -2.3422e-07
       -1.0538e-07
  AP =  -0.00042346  1.4996e-07  2.1743e-07
        1.0097e-06  1.8442e-08
        2.9736e-07
  BP =  -0.00042031  4.2524e-07  8.1582e-08
        4.5367e-07  2.3442e-07
        1.0553e-07
  sqrt(det(CD))=0.942186 [arcsec]
Found tag-along columns from field: FLUX BACKGROUND
Field 1: solved with index index-4108.fits.
Field 1: tried 109146 quads, matched 369626 codes.
Spent 88.456 s user, 1.6 s system, 90.056 s total, 90.1266 s wall time.
Writing 49 rows (of 4036 field and 51 index objects) to correspondence file.
cx<=dx constraints: 0
meanx constraints: 512384
RA,Dec constraints: 0
AB scale constraints: 0
Spent 90.1447 seconds on this field.

Full log

Log 2

2019-06-16 18:48:17,764 Starting Job processing for Job 3436227
2019-06-16 18:48:17,767 Creating directory /home/nova/nova/net/data/jobs/0343/03436227
2019-06-16 18:48:17,770 submission id 2749196
2019-06-16 18:48:17,775 running: augment-xylist --downsample 2 --tweak-order 2 --scale-units degwidth --scale-low 0.1 --wcs wcs.fits --out /home/nova/nova/net/data/jobs/0343/03436227/job.axy --image /home/nova/nova/net/data/files/uploaded/cb6/cb6c0947c7a0032aa8d5292c7b7ae89d5aed9a4b --rdls rdls.fits --scale-high 180.0 --corr corr.fits 
2019-06-16 18:48:25,505 created axy file /home/nova/nova/net/data/jobs/0343/03436227/job.axy
2019-06-16 18:48:25,505 command: cd /home/nova/nova/net/data/jobs/0343/03436227 && /home/nova/nova/net/solvescript.sh job-nova-3436227 job.axy >> /home/nova/nova/net/data/jobs/0343/03436227/log
2019-06-16 18:49:56,132 Solver completed successfully.
2019-06-16 18:49:56,132 Checking for WCS file /home/nova/nova/net/data/jobs/0343/03436227/wcs.fits
2019-06-16 18:49:56,132 WCS file exists
2019-06-16 18:49:56,136 Created TanWCS: <TanWCS: CRVAL (312.006093, 44.535882) CRPIX (2477.881989, 1268.529205) CD (0.000129, -0.000228; -0.000228 -0.000129) Image size (5496.000000, 3672.000000)>
2019-06-16 18:49:56,156 SkyLocation: <SkyLocation: nside(64) healpix(14307)>
2019-06-16 18:49:56,157 Created Calibration Calibration 2198482
2019-06-16 18:49:56,242 Finished job 3436227
dstndstn commented 5 years ago

How long have you waited for a solution? How much memory do you have in the machine? Is it possible that the solver is continuing to run, slowly?

Also: that's some ancient version you've got there; I renamed backend.exe to astrometry-engine.exe in 2013.

If you can run the solver directly (eg, in the "exec" line in your log file,

/usr/bin/solve-field -p -O -U none -B none -R none -M none -N none -C cancel --crpix-center -z 8 stars.fit

(if you can find where the GUI puts the stars.fit file.), you might be a more helpful error message, or a printed segfault, or something like that.

You might also want to report this at adgsoftware.com -- maybe they've seen it before. (I've never even seen that program before.)

nevion commented 5 years ago

I will see about updating the packaged binaries+reproducing after the fact, and maybe prod adgsoftware along the way

nevion commented 5 years ago

solve works on current git version without issue for the 4200 dataset, for 5000, it again goes back to very low cpu usage and low io. Not at all sure for what reasons. Is there anyway to help the software determine which dataset it should use or is it supposed to see all of these datasets in a single directory of fits index files at once? My effective focal length for that image was ~520 mm

I pinged ansrv owner to update his build, he says he's checking into it.

dstndstn commented 5 years ago

With solve-field, most useful is to give it --scale-low, --scale-high, --scale-units to tell it the size (angle on the sky) of the image. See the solve-field help for details. Also helpful is --ra, --dec, --radius if you know them. Those don't need to be very accurate; even within 10 degrees is quite helpful.

nevion commented 5 years ago

could you clarify on the dataset's question - should all of the fits files be put into a single directory , from multiple datasets (4200, 5000)? I was symliking all the individual fits files into a big "virtual" dataset directory but I'm not sure if that made sense - seems to if they're just for different scales.

dstndstn commented 5 years ago

The 4200 and 5000 series cover the same scales, so you should really only need to use one set.

You can describe which index files to load with a config file -- solve-field --backend-config [filename.cfg] with contents like

add_path /usr/local/astrometry/data autoindex

or explicitly list the ones to load (unfortunately, * patterns aren't supported, so this will be a long list!)

index index-4204-00 index index-4204-01 ...

cheers, --dustin

On Mon, Jun 24, 2019 at 9:10 AM Jason Newton notifications@github.com wrote:

could you clarify on the dataset's question - should all of the fits files be put into a single directory , from multiple datasets (4200, 5000)? I was symliking all the individual fits files into a big solve directory but I'm not sure if that made sense - seems to if they're just for different scales.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dstndstn/astrometry.net/issues/162?email_source=notifications&email_token=AAIEH7LPD5HET3SKEOUEDQ3P4DBS5A5CNFSM4HYSGJP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYM3XQI#issuecomment-505002945, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIEH7K632CLF3VXSFJJ56LP4DBS5ANCNFSM4HYSGJPQ .