Analysis returning only nulls

[ethervoid]

Analysis Only Returns Nulls

I ran an analysis that looked at Zillow Rental Value Index Single Family (2016-8 timespan) and Non-US Citizen Population (2011-2015 timespan) on the Wal-Mart Dataset. Only NULL values where returned.

I tried this for a couple of different maps, datasets, and analysis and was never able to get proper values to appear in the new column.

Reported by @kevin-reilly

[ethervoid]

@rafatower pointed that if the source are polygons could be problems to get an output:

When reporting issues it is quite important to specify what input data you use, at least if it is points or polygons (even better: provide a .carto file), because that affects the DO ability to return meaningful data.

From the "Data Observatory API Architecture Guide":

Unfortunately we may have some trouble reflecting that in the UI.

[ethervoid]

This could be an issue because of the above-mentioned dataset, walmart, only have points so it seems like something is not right.

I've executed the obs_getdata query and I'm getting this messages:

NOTICE:  00000: cdb_dataservices_server._obs_getdata(5): [obs_2017_05_04_f20ee1131c] REMOTE NOTICE: Cannot perform calculation over polygon for us.zillow.SingleFamilyResidenceRental_Zri//us.census.tiger.zcta5/2016-08

Seems like it's thinking we're using polygons when the dataset is made of points. I'll keep digging

[ethervoid]

I've figured out what the problem is. The geomvals generated with NULLs are interfering and making the OBS_GetData function think that is dealing with Polygons.

These are the geomvals with NULL values:

 {"(,1058)","(,1645)","(0101000020E61000009510ACAA97DB55C0274EEE7728D44340,977)","(,1017)","(0101000020E610000016A1D80A9A4E55C0BA9F53909F054140,699)","(0101000020E6100000D95E0B7A6FDD54C0F9669B1BD31F4140,3)","(0101000020E6100000EF92382BA26854C0FB0626378A5A4140,605)","(0101000020E610000064E597C1189556C03140A209144F4340,3005)","(0101000020E6100000E23E726BD2F55DC0F58079C894674240,1769)","(0101000020E610000055A69883A0E057C0D787F546ADD44040,145)","(0101000020E6100000DB317557761A57C0267156444D904140,82)","(0101000020E6100000E7C2482F6A9C54C0B62DCA6C90034440,1550)","(,2568)","(0101000020E61000005D70067FBFEC55C0CE1C925A285D4340,527)","(,1212)","(0101000020E61000002272FA7ABEFA54C0732B84D558D84340,1420)","(0101000020E61000005DA8FC6B798957C09B8F6B43C52A4240,1)","(0101000020E61000005853591476DD57C0BB287AE063AC3D40,2505)","(0101000020E6100000766EDA8CD33D57C01A1538D9061A4240,2)","(0101000020E610000079B29B19FD2B58C01EFCC401F4873E40,544)","(0101000020E6100000D9791B9B1D055EC0213EB0E3BF7C4240,1539)","(,1586)","(0101000020E61000005BB4006DABD057C0F73DEAAF57764340,464)","(0101000020E6100000790261A758C352C0FD6A0E10CCF74340,3099)","(0101000020E61000006342CC2555D451C0F05014E813C14440,2623)","(0101000020E6100000BA8784EFFD9B54C0FCE25295B61A4340,2553)","(0101000020E6100000E90B21E7FD1154C05E0F26C5C7A34340,1501)","(0101000020E6100000E318C91EA13D52C0917EFB3A70CE4540,2348)","(,2590)","(,2043)"}

and these are the geomvals without NULL values:

{"(0101000020E61000009510ACAA97DB55C0274EEE7728D44340,977)","(0101000020E610000016A1D80A9A4E55C0BA9F53909F054140,699)","(0101000020E6100000D95E0B7A6FDD54C0F9669B1BD31F4140,3)","(0101000020E6100000EF92382BA26854C0FB0626378A5A4140,605)","(0101000020E610000064E597C1189556C03140A209144F4340,3005)","(0101000020E6100000E23E726BD2F55DC0F58079C894674240,1769)","(0101000020E610000055A69883A0E057C0D787F546ADD44040,145)","(0101000020E6100000DB317557761A57C0267156444D904140,82)","(0101000020E6100000E7C2482F6A9C54C0B62DCA6C90034440,1550)","(0101000020E61000005D70067FBFEC55C0CE1C925A285D4340,527)","(0101000020E61000002272FA7ABEFA54C0732B84D558D84340,1420)","(0101000020E61000005DA8FC6B798957C09B8F6B43C52A4240,1)","(0101000020E61000005853591476DD57C0BB287AE063AC3D40,2505)","(0101000020E6100000766EDA8CD33D57C01A1538D9061A4240,2)","(0101000020E610000079B29B19FD2B58C01EFCC401F4873E40,544)","(0101000020E6100000D9791B9B1D055EC0213EB0E3BF7C4240,1539)","(0101000020E61000005BB4006DABD057C0F73DEAAF57764340,464)","(0101000020E6100000790261A758C352C0FD6A0E10CCF74340,3099)","(0101000020E61000006342CC2555D451C0F05014E813C14440,2623)","(0101000020E6100000BA8784EFFD9B54C0FCE25295B61A4340,2553)","(0101000020E6100000E90B21E7FD1154C05E0F26C5C7A34340,1501)","(0101000020E6100000E318C91EA13D52C0917EFB3A70CE4540,2348)"}

So the way to deal with this in to remove NULL values here and here

Doing the query filtering the NULL values I was able to get some results:

 singlefamilyresidencerental_zri_2016_08
-----------------------------------------
                                    1076
                                     934
                                     954
                                    1041
                                    1213
                                    1301
                                     890
                                     741
                                     941
                                     955
                                    1187
                                    1173
                                    1604
                                    1489
                                    2490
                                    2329
                                    1103
                                    1914
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
                                  [NULL]
(30 rows)

[ethervoid]

PR that fix this problem cartodb/camshaft#312