Image annotations don't work (plotann)

tunarob commented 2 years ago

So I finally managed to run nova via docker-compose. It's all fine, I'm able to solve images fine.

The problem I'm having is being unable to show/get image annotations. Looks like plotannnot found?

That's what I can see in console when trying to get image (http://localhost:8000/user_images/3#annotated):

INFO Running: plotann.py --no-grid --toy -10 --scale 0.0847457627118644 /src/astrometry/net/data/jobs/0000/00000003/wcs.fits /tmp/tmplg9ks09q.ppm /tmp/tmple4hqjm_
nova_1  | INFO out: 
nova_1  | INFO err: /bin/sh: 1: plotann.py: not found

And that's what I see in API (http://localhost:8000/api/jobs/3/annotations/)

ModuleNotFoundError at /api/jobs/3/annotations/
No module named 'astrometry.plot.plotann'

Request Method: | GET
-- | --
http://localhost:8000/api/jobs/3/annotations/
4.0.4
ModuleNotFoundError
No module named 'astrometry.plot.plotann'
/usr/local/lib/python/astrometry/net/api.py, line 452, in get_anns
/usr/bin/python
3.8.10
['/src/astrometry/net',  '/usr/local/lib/python',  '/usr/lib/python38.zip',  '/usr/lib/python3.8',  '/usr/lib/python3.8/lib-dynload',  '/usr/local/lib/python3.8/dist-packages',  '/usr/lib/python3/dist-packages',  '/src/astrometry',  '/usr/local/lib/python/astrometry/net',  '/usr/local/lib/python/astrometry']

Any idea how to fix that?

dstndstn commented 2 years ago

I think I just fixed it in 63cb864

Basically, I wasn't installing plotann.py, because that part of the Makefile wasn't actually running -- oops.

You can either git pull to grab that fix, or:

copy plot/plotann.py and plot/init.py to $INSTALL_DIR/lib/python/astrometry/plot
symlink $INSTALL_DIR/lib/python/astrometry/plot/plotann.py to $INSTALL_DIR/bin

tunarob commented 2 years ago

Nice! Can see the error gone on both pages.

Screenshot 2022-04-12 at 20 27 41

Now the annotated image works. However, the API annotation part now getting another error:

Screenshot 2022-04-12 at 20 28 41

SystemError at /api/jobs/2/annotations/

&lt;class 'spherematch.KdTree'&gt; returned NULL without setting an error

Request Method: | GET
-- | --
http://localhost:8000/api/jobs/2/annotations/
4.0.4
SystemError
<class 'spherematch.KdTree'> returned NULL without setting an error
/usr/local/lib/python/astrometry/libkd/spherematch.py, line 392, in tree_open
/usr/bin/python
3.8.10
['/src/astrometry/net',  '/usr/local/lib/python',  '/usr/lib/python38.zip',  '/usr/lib/python3.8',  '/usr/lib/python3.8/lib-dynload',  '/usr/local/lib/python3.8/dist-packages',  '/usr/lib/python3/dist-packages',  '/src/astrometry',  '/usr/local/lib/python/astrometry/net',  '/usr/local/lib/python/astrometry']

Not sure if this is any help: https://stackoverflow.com/questions/53972542/how-to-fix-systemerror-built-in-function-name-returned-null-without-settin This Python level is not my thing so can't help much unfortunately.

tunarob commented 2 years ago

Any chance of fixing this issue in the near future? :) Key part of my new project so good to know if I can assume it's working fairly soon.

dstndstn commented 2 years ago

No, this is not something I can fix. You need to download those files and tell the code where to find them.

tunarob commented 2 years ago

Mmm ok. Any clue on how to do it? Would like to help and fix the issue so it also works for myself. But that’s something I’m usually away from and done have much idea about.

Also thinking why this doesn’t work in docker but working on prod?

dstndstn commented 2 years ago

I'm sure I wrote a better answer to this, but now I don't see it...

The failure here is that it's trying to read the Henry Draper catalog, which you can get from here http://data.astrometry.net/ and then tell the web service where to find it, here https://github.com/dstndstn/astrometry.net/blob/main/net/settings_common.py#L48

Same with Tycho-2 and Hipparcos.

dstndstn commented 2 years ago

Okay, so I also just pushed a couple of commits that don't try to annotate images using catalog files that don't exist, and also raises better exceptions.

tunarob commented 2 years ago

Nice one. Had a quick look o na new build in my docker compose.

Uploaded 1 image and can see the new errors you added, I think:

But can still see annotations in nova:

Then had a look at the annnotattion API:

Will give it a go with the missing files later but that's already a huge progress :) Thinking about putting my docker-compose solution to github as well, hopefully it makes running astrometry locally much easier for people.

Also found some django warnings regarding missing default_auto_field. Once done with the file I'll prepare a PR in fwe days that addresses it.

dstndstn commented 1 year ago

It's looking for a catalog to annotate the image with, but failing to find it. Specifically, the Henry Draper catalog (in kd-tree format), which you can get from http://data.astrometry.net/hd.fits and you can set the path it will be searched for in astrometry/net/settings.py : HENRY_DRAPER_CAT.

While you're there, may want to grab Tycho-2 and Hipparcos, see the bottom of the page here http://data.astrometry.net/

On Tue, Apr 12, 2022 at 2:31 PM Robert Węglarek @.***> wrote:

Nice! Can see the error gone on both pages.

[image: Screenshot 2022-04-12 at 20 27 41] https://user-images.githubusercontent.com/2023993/163029152-ccc6c310-926a-40f4-a497-262fc94c95e5.png

Now the annotated image works. However, the API annotation part now getting another error:

[image: Screenshot 2022-04-12 at 20 28 41] https://user-images.githubusercontent.com/2023993/163029320-0a5a9d18-8746-465f-be26-f4916ff1df5e.png

SystemError at /api/jobs/2/annotations/

<class 'spherematch.KdTree'> returned NULL without setting an error

Request Method: | GET -- | --http://localhost:8000/api/jobs/2/annotations/ 4.0.4 SystemError <class 'spherematch.KdTree'> returned NULL without setting an error /usr/local/lib/python/astrometry/libkd/spherematch.py, line 392, in tree_open /usr/bin/python 3.8.10 ['/src/astrometry/net', '/usr/local/lib/python', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages', '/src/astrometry', '/usr/local/lib/python/astrometry/net', '/usr/local/lib/python/astrometry']

Not sure if this is any help: https://stackoverflow.com/questions/53972542/how-to-fix-systemerror-built-in-function-name-returned-null-without-settin This Python level is not my thing so can't help much unfortunately.

— Reply to this email directly, view it on GitHub https://github.com/dstndstn/astrometry.net/issues/255#issuecomment-1097055918, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIEH7J4TXFGQYJCBFCHXKLVEW6RRANCNFSM5TGWFP2Q . You are receiving this because you commented.Message ID: @.***>

dstndstn / astrometry.net

Image annotations don't work (plotann) #255