ERROR: Image type not recognized: Could not determine file type (does the file exist?)

[mariavd]

I have installed Astrometry on a Ubuntu computer and it working fine for months, recognising stars and deep sky objects, and plotting them on a png file. Then during this summer I took a break from astro and just recently spend a night under the stars. To my surprise Astrometry gives an error when I give any photo but the demos to the solver. I found a thread in the Google group regarding this error and ran the recommended commands. I am sorry for the long list of output. I thought it might give some hints.

So, for example:

solve-field apod1.jpg --overwrite
Reading input file 1 of 1: "apod1.jpg"...
jpegtopnm: WRITING PPM FILE
Read file stdin: 800 x 526 pixels x 1 color(s); maxval 255
Using 8-bit output
Extracting sources...
simplexy: found 1776 sources.
Solving...

But on any other photo, including those that were solved and plotted before:

solve-field test.jpg --overwrite --verbose --no-delete-temp
Reading input file 1 of 1: "test.jpg"...
Base: "./test", basefile "test.jpg", basedir ".", suffix "jpg"
Checking if file "test.jpg" ext 0 is xylist or image: image
  (not xyls because: Failed to open FITS table test.jpg: Failed to open FITS file "test.jpg")
Running: /usr/bin/python -m astrometry.util.image2pnm --infile test.jpg --uncompressed-outfile /tmp/tmp.uncompressed.it9R0s --outfile /tmp/tmp.ppm.jyIWvL --ppm --mydir /usr/bin/solve-field
ERROR: Image type not recognized: Could not determine file type (does the file exist?): test.jpg
augment-xylist.c:589:backtick Failed to run command: /usr/bin/python -m astrometry.util.image2pnm --infile test.jpg --uncompressed-outfile /tmp/tmp.uncompressed.it9R0s --outfile /tmp/tmp.ppm.jyIWvL --ppm --mydir /usr/bin/solve-field
 ioutils.c:567:run_command_get_outputs Command failed: return value 255

Giving the absolute path does not help. Obviously, something went wrong with Python. For starters, I don't know why the system Python is being called when I have installed Anaconda. I re-compiled the code just to make sure that indeed at compilation time Anaconda is selected.

$ which python
/home/MYUSERNAME/anaconda2/bin/python

Jpeg2pnm is exported to PATH and when I give a name of some jpeg file, it seems to convert it to a binary because on standard output I can see long list of gibberish characters.

which jpegtopnm 
/usr/bin/jpegtopnm

Just as it was suggested in the Google group, I ran

$ file -b -N -L -k -r apod1.jpg 
JPEG image data, JFIF standard 1.02, resolution (DPI), density 300x300, segment length 16, Exif Standard: [TIFF image data, big-endian, direntries=8, description=Processed with MaxIm DL, orientation=upper-left, xresolution=134, yresolution=142, resolutionunit=2, software=Adobe Photoshop 7.0, datetime=2003:02:27 01:26:38], baseline, precision 8, 800x526, frames 3
- data

$ file -b -N -L -k -r test.jpg 
ERROR: JPEG image data, JFIF standard 1.01, resolution (DPI), density 240x240, segment length 16, Exif Standard: [TIFF image data, big-endian, direntries=8, description=Picture saved with settings embedded., orientation=upper-left, xresolution=148, yresolution=156, resolutionunit=2, software=Adobe Photoshop Lightroom 6.1.1 (Windows), datetime=2017:08:17 11:16:47] name use count (30) exceeded

I installed Astrometry in the default path but /usr/local/astrometry/bin/image2pnm.py does not exist there. Also:

python -c "import pyfits; print pyfits.__version__; print pyfits.__file__"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named pyfits

Do you have any ideas what I can do about it? It might be that something broke when I upgraded to Ubuntu 17. Thank you!

[dstndstn]

Hi,

This looks like a problem with the "file" program. A web search for the error message from "file", "name use count (30) exceeded", yields this old bug from 2015. This would have happened during your full OS update.

https://bugzilla.redhat.com/show_bug.cgi?id=1201630

so could you please run ''' file --version ''' and report which version you have.

[mariavd]

Hello,

So my file is newer than the buggy version but it seems that it is the same case.

$ file --version
file-5.29
magic file from /etc/magic:/usr/share/misc/magic

[dstndstn]

My web search did not turn up an ubuntu bug report about this... there isn't an update for that ubuntu package, is there?

[mariavd]

No, everything is up to date. Maybe I should submit a bug report.

[dstndstn]

I'm surprised that this 17.04 has been out for ~5 months and nobody has reported this, but yes, it looks like ubuntu shipped a known buggy 'file' to me.

[mariavd]

Thanks, I will close the issue now and get back to you in case the problem with Ubuntu gets resolved.

[mariavd]

I found the cause of the problem. It seems that the EXIF data is not being read properly. With an older version of file:

$ file --version
file-5.14                                                                             

$ file apod1.jpg
apod1.jpg: JPEG image data, JFIF standard 1.02                                        

$ file test.jpg 
test.jpg: JPEG image data, JFIF standard 1.01                                         

So the output from Lightroom is recognised as the older version of JFIF. My workaround is that I save the photo as JPEG2000 and choose the EXIF data to be removed, then save as JPEG again. Astrometry works just fine; objects are detected and plotted. I hope that they will fix file but I am glad that I can at least use the solver again. It is invaluable!

[dstndstn]

Thanks for the follow-up; apologies, I could have provided some hints for workarounds...

Another option that might work would be to take your JPEG image and go:

jpegtopnm test.jpg | pnmtojpeg > test2.jpg

and then run test2.jpg -- the PNM format doesn't support EXIF or anything fancy, so test2.jpg might work with your version of 'file'.

Thanks again.

On Mon, Sep 18, 2017 at 6:23 PM, mariavd notifications@github.com wrote:

I found the cause of the problem. It seems that the EXIF data is not being read properly. With an older version of file:

$ file --version file-5.14

$ file apod1.jpg apod1.jpg: JPEG image data, JFIF standard 1.02

$ file test.jpg test.jpg: JPEG image data, JFIF standard 1.01

So the output from Lightroom is recognised as the older version of JFIF. My workaround is that I save the photo as JPEG2000 and choose the EXIF data to be removed, then save as JPEG again. Astrometry works just fine; objects are detected and plotted. I hope that they will fix file but I am glad that I can use the solver. It is invaluable!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dstndstn/astrometry.net/issues/118#issuecomment-330373136, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBD_aXRd0gywqXaPHzml_0ZANrdppI9ks5sju1ZgaJpZM4PaUKt .

[mariavd]

Thank you! This line saves the trouble of removing the EXIF manually. Now file does not complain and the solver works fine.

$ file test2.jpg 
test2.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 2335x1831, frames 3