Deal with square brackets in filename

[jishnub]

This works in python but not julia:

In [1]: from astropy.io import fits

In [2]: import numpy as np

In [3]: fits.writeto("abc[12].fits", np.ones((2,2)))

In [4]: fits.getdata("abc[12].fits")
Out[4]: 
array([[1., 1.],
       [1., 1.]])

and

julia> using FITSIO

julia> FITSIO.fitsread("abc[12].fits")
ERROR: While processing file `abc[12].fits`: parse error in input file URL

[giordano]

Since the error message is from cfitsio, I guess this is an upstream issue

[jishnub]

Indeed, Cfitsio expects settings in square brackets when appended to filenames. I guess astropy has some inbuilt tricks to escape special characters. I wonder if it's worth replicating the astropy behavior (perhaps as an opt-in macro that inserts appropriate escape sequences)?

[giordano]

I guess astropy has some inbuilt tricks to escape special characters. I wonder if it's worth replicating the astropy behavior

How do they deal with the case where the user really wants to specify the settings?

I wonder if it's worth replicating the astropy behavior

Do you happen to know what they do internally?

(perhaps as an opt-in macro that inserts appropriate escape sequences)?

Why a macro? Wouldn't a function work just as fine?

[jishnub]

From what I understand after a quick look through the cfitsio documentation, settings go at the end of the filename. I suppose square brackets in the middle of the filename do not correspond to settings. This is definitely an upstream issue, and unfortunately I'm not sure what astropy does (not too familiar with their code).

I guess a function would work fine too, a macro isn't necessary.

[jishnub]

After looking through the astropy code, it appears that they are bypassing cfitsio altogether and instead reading the binary file using numpy. This explains the difference, but makes finding a quick fix more complicated.

[jishnub]

There appears to be a simple way around this -- define a soft link to the FITS file but this time without the square brackets.

$ ln -s 'abc[12].fits' abc.fits

julia> FITSIO.fitsread("abc.fits")
2×2 Array{Float64,2}:
 1.0  1.0
 1.0  1.0

This might be carried out in Julia too as

julia> f = tempname() * ".fits";

julia> symlink("$(pwd())/abc[12].fits", f)

julia> FITSIO.fitsread(f)
2×2 Array{Float64,2}:
 1.0  1.0
 1.0  1.0

so a wrapper might be defined as

julia> wrapper(filename) = (f = tempname(); symlink(joinpath(pwd(), filename), f); f)
wrapper (generic function with 1 method)

julia> FITSIO.fitsread(wrapper("abc[12].fits"))
2×2 Array{Float64,2}:
 1.0  1.0
 1.0  1.0

Closing this now, as the behavior of astropy is significantly different for us to not consider fixing this here.