pod folder names

[mbauer288]

I'm having an odd issue trying to test my podding using the SIDs from the sidecar file as the sids argument for starepandas.read_pods().

Using Panoply to view sidecar SID values

    DYAMONDv2_stare.STARE_index[0,0] = 2287820569163665660  => pystare.int2hex() '0x1fbff8afaed510fc'
    DYAMONDv2_stare.STARE_index[1,0] = 2287820569179220740  => pystare.int2hex() '0x1fbff8afafc26b04'

Using 'ncdump -v 'STARE_index' /Users/mbauer/tmp/data/POMD/discover/DYAMONDv2_stare.nc | less'

    2287820569163665748, ... 2287827571976060018, ...

Reading the same values in python using NetCDF4 (matches ncdump)

    STARE_index[0, 0] = 2287820569163665748  => pystare.int2hex() '0x1fbff8afaed51154'
    STARE_index[1, 0] = 2287827571976060018  => pystare.int2hex() '0x1fbfff0e26fa3c72'

Now the issue is none of these hex(SIDs) are directory names in my podding directory. Which are more like this:

    0x0000000000000004/
    0x0008000000000004/
    ...
    0x3ff0000000000004/
    0x3ff8000000000004/

Trying

    pystare.hex2int('0x0000000000000004') => 4
    pystare.hex2int('0x0008000000000004') => 2251799813685252

That is odd. I see that I must be doing something wrong with starepandas.write_pods(). Then again, one thing jumped out at me right away is the '4' value above because I see my podding script sets level to 4.

So I tried looking at a few SIDs from the sidecar file:

    sids_list     = [2287820569163665748, 2287820569179220756, 2287820569179367604, 2287820569153793364,
                     2287820569153358612, 2287820569150348116, 2287820569150501620, 2287820569133379092,
                     2287820569393028436, 2287820569428428436]
    sids_list_hex = ['0x1fbff8afaed51154', '0x1fbff8afafc26b14', '0x1fbff8afafc4a8b4', '0x1fbff8afae3e6d54',
                     '0x1fbff8afae37cb14', '0x1fbff8afae09db54', '0x1fbff8afae0c32f4', '0x1fbff8afad06ee14',
                     '0x1fbff8afbc80dd54', '0x1fbff8afbe9d0694']

and then coercing them to level=4

    sids_new = pystare.spatial_coerce_resolution(sids_list, 4)
    sids_new_list_hex = [pystare.int2hex(_) for _ in sids_new]

    sids_new          = [2287820569163665732, 2287820569179220740, 2287820569179367588, 2287820569153793348, 
                         2287820569153358596, 2287820569150348100, 2287820569150501604, 2287820569133379076, 
                         2287820569393028420, 2287820569428428420]
    sids_new_list_hex = ['0x1fbff8afaed51144', '0x1fbff8afafc26b04', '0x1fbff8afafc4a8a4', '0x1fbff8afae3e6d44', 
                         '0x1fbff8afae37cb04', '0x1fbff8afae09db44', '0x1fbff8afae0c32e4', '0x1fbff8afad06ee04', 
                         '0x1fbff8afbc80dd44', '0x1fbff8afbe9d0684']

Okay this changes the values but, none of sids_new_list_hex are podding subfolders either.

I let starepandas.write_pods() make the podding subfolder names so I'm unsure what I am doing incorrectly. I can see that I only have 2048 podding subfolders, whereas STARE_index(i=1800, j=3600) so clearly not a 1-to-1 mapping of sidecar SIDs and podding subfolders.

BTW, my podding code (outside of that injected into STAREPandas) is in a github repo STAREpodder, which is private, but I believe bayesics/SpatialTemporal members can at least read it.

[NiklasPhabian]

Remember that STARE is left-justified. I.e. the level is not implicitly given by the length of the SID, but explicitly by the 5 least significant bits (resolution/neighborhood bits). When you spatial_coerce_resolution() all you do is modify the last 5 bits. I.e. the location is still specified all the way down to the leaf resolution, but the resolution bits tell you to ignore them.

e.g. these two point to the same trixel

0x0000001111110004
0x0000000000000004

In order to have two SIDs denoting the same trixel to be identical (and thus understood to be spatially coincident by a non-STARE-enabled system such as POSIX), what you need to do is to clear (i.e. set to 0) the location bits all the way down to the resolution.

You can do this with spatial_clear_to_resolution()

STAREPandas has to_stare_level() wrapping spatial_coerce_resolution and spatial_clear_to_resolution. You can see we use this in write_pods_spatial()

Also, check out this repo https://github.com/SpatioTemporal/STAREPods_py

[mbauer288]

That worked. Thank you.