[E & A] Analysis fails for Alaska

[j08lue]

We added analysis region presets in and in that context discovered that analysis of Alaska as well as all-US (when it includes Alaska) fails.

• 872

Unsure whether this is because of details in that geometry or the antimeridian crossing.

Acceptance criteria

• [ ] Analysis works for Alaska

[j08lue]

In the UI, the steps to reproduce this issue are:

1. Go to the Exploration & Analysis interface in this preview: https://deploy-preview-319--ghg-demo.netlify.app/exploration?datasets=%5B%5D
2. Select Alaska from the drop-down of regions to analyze
3. Hit "Run Analysis"
4. Observe the above error message

What is going on behind the scenes is a two-step process:

1. A STAC /search with the geometry and time range as payload to get the items
2. One request for each item against TiTiler /statistics

In this case, it is the STAC https://ghg.center/api/stac/search call that throws 403 Forbidden, for example with this payload: stac-search-payload-alaska.json.

[j08lue]

We are sending the geometry through an antimeridian fixing function before sending it to STAC search:

https://github.com/NASA-IMPACT/veda-ui/blob/4c98710497230432c2707053dd6303e314cbbe8b/app/scripts/utils/antimeridian.ts#L164

https://github.com/NASA-IMPACT/veda-ui/blob/4c98710497230432c2707053dd6303e314cbbe8b/app/scripts/components/analysis/utils.ts#L109

• not sure how well that works.

The data comes from Census.gov’s 1:20million cartographic state boundaries and we store it in our app here: https://github.com/NASA-IMPACT/veda-ui/blob/main/static/public/geo-data/states/Alaska.geojson

You can see how we pull the data and conver it to GeoJSON here: https://github.com/NASA-IMPACT/veda-ui/blob/main/docs/development/geo-data-preparation.ipynb

[j08lue]

Pretty sure it has to do with the MultiPolygon complexity, actually: When I in the UI combine different rings of the Alaska polygon, even staying away from the antimeridian, at some point the failure starts happening.

In the case below, I have the same number of rings in the MultiPolygon, but once I select a more complex one than the other time. The issue happens with any additional complex ring I add, not this particular one:

This would be the minimum failing payload to https://ghg.center/api/stac/search: failing-payload-alaska.json.zip

[hanbyul-here]

To add to @j08lue 's detective work 🕵️ I tried running an analysis with extreme simplification per polygon (like below). It still got 403 response from STAC. I wonder the problem lies on some kinds of combinations of complexity of polygons + the number of polygons?

Another thing to note is that AK fails with most of datasets on GHG (afa I tested), but it works with no2 datasets on UI

[anayeaye]

Just for data, I tried this in dev.openveda.cloud/api/stac and the search is OK (I downloaded the 173 KB stac-search-payload-alaska.json linked above). Sharing below before I lose it in my terminal history. Caveat: I have no idea if swapping for the veda collection lis-tws-anomaly is a fair comparison at all.

curl -X 'POST' \
  'https://dev.openveda.cloud/api/stac/search' \
  -H 'accept: application/geo+json' \
  -H 'Content-Type: application/json' \
  -d '{"filter-lang":"cql2-json","limit":10000,"fields":{"include":["assets.npp.href","properties.start_datetime","properties.datetime"],"exclude":["collection","links"]},"filter":{"op":"and","args":[{"op":"t_intersects","args":[{"property":"datetime"},{"interval":["2017-10-01T00:00:00.000Z","2018-01-01T23:59:59.999Z"]}]},{"op":"s_intersects","args":[{"property":"geometry"},{"type":"MultiPolygon","coordinates":[[[[-179.17426,51.27906],[-179.12686,51.21986],[-178.90888,51.34059],[-178.92687,51.38365],[-178.99559,51.41449],[-179.17426,51.27906]]],[[[-178.89596,51.77923],[-178.79241,51.74608],[-178.73336,51.78396],[-178.77966,51.85156],[-178.87684,51.83793],[-178.89596,51.77923]]],[[[-178.88935,51.57036],[-178.82596,51.54709],[-178.73459,51.54233],[-178.58479,51.56387],[-178.55148,51.61018],[-178.67815,51.62602],[-178.88935,51.57036]]],[[[-178.22413,51.86489],[-178.08607,51.80806],[-177.99527,51.78154],[-177.98163,51.71563],[-178.0548,51.70478],[-178.11786,51.67784],[-178.04566,51.63007],[-177.90919,51.59668],[-177.86796,51.67938],[-177.75502,51.77284],[-177.64928,51.80186],[-177.61531,51.85509],[-177.75743,51.84705],[-177.88742,51.8509],[-177.9521,51.91536],[-178.09063,51.91941],[-178.19709,51.90547],[-178.22413,51.86489]]],[[[-177.7078,51.70328],[-177.65139,51.65361],[-177.34801,51.69652],[-177.27512,51.68052],[-177.18994,51.69723],[-177.10519,51.71934],[-177.09866,51.82966],[-177.04509,51.89861],[-177.09927,51.93613],[-177.18127,51.94318],[-177.19954,51.91025],[-177.22818,51.80379],[-177.31315,51.77823],[-177.49213,51.77031],[-177.67012,51.74339],[-177.7078,51.70328]]],[[[-176.98738,51.60688],[-176.93892,51.59099],[-176.809,51.61624],[-176.71543,51.62043],[-176.65601,51.65831],[-176.46705,51.72668],[-176.28992,51.74169],[-175.99847,51.80155],[-175.96304,51.84626],[-175.78912,51.91933],[-175.63974,51.93365],[-175.42486,51.97234],[-175.45047,52.01275],[-175.66428,51.99387],[-175.88752,51.99515],[-176.00759,52.06624],[-176.05599,52.10948],[-176.14951,52.11758],[-176.21119,52.06471],[-176.16864,51.94803],[-176.17387,51.88246],[-176.57638,51.84228],[-176.54912,51.95557],[-176.57998,52.00325],[-176.69877,51.96446],[-176.77402,51.9659],[-176.81043,51.9271],[-176.76248,51.86789],[-176.78189,51.83238],[-176.91709,51.79702],[-176.95013,51.68673],[-176.98738,51.60688]]],[[[-175.32332,52.0075],[-175.01481,52.00701],[-174.89231,52.0197],[-174.73659,52.00732],[-174.55628,52.03674],[-174.4087,52.01282],[-174.38266,52.08167],[-174.27828,52.0895],[-173.8207,52.04332],[-173.69532,52.05533],[-173.51305,52.02532],[-173.16956,52.04386],[-172.98023,52.06406],[-172.94781,52.10738],[-173.17441,52.12629],[-173.37523,52.10824],[-173.52993,52.15937],[-173.65441,52.1462],[-174.02264,52.13372],[-174.047,52.23627],[-173.98521,52.31761],[-174.06825,52.39034],[-174.18535,52.4178],[-174.32982,52.37356],[-174.45598,52.3137],[-174.46296,52.21304],[-174.55467,52.16041],[-174.71521,52.12738],[-174.88724,52.12861],[-175.03122,52.09212],[-175.30156,52.05561],[-175.32332,52.0075]]],[[[-173.11557,60.65897],[-173.0638,60.50259],[-172.896,60.45059],[-172.63027,60.33492],[-172.25426,60.29738],[-172.23886,60.33664],[-172.38095,60.38276],[-172.54591,60.41223],[-172.84734,60.51674],[-172.91264,60.60413],[-173.07464,60.70466],[-173.11557,60.65897]]],[[[-172.64,52.24477],[-172.52809,52.25434],[-172.41442,52.27674],[-172.30144,52.32995],[-172.32644,52.36647],[-172.44818,52.39144],[-172.54512,52.35787],[-172.61228,52.30684],[-172.64,52.24477]]],[[[-171.84998,63.48504],[-171.76011,63.38163],[-171.43332,63.30758],[-171.22633,63.39511],[-171.06766,63.42458],[-170.89617,63.41775],[-170.66354,63.37611],[-170.43066,63.31428],[-170.30363,63.23869],[-170.26303,63.17915],[-170.18648,63.18162],[-170.04962,63.16338],[-169.88123,63.10585],[-169.78847,63.04302],[-169.75725,62.96009],[-169.63831,62.93753],[-169.56802,62.97688],[-169.57696,63.02703],[-169.53498,63.07436],[-169.43675,63.11358],[-169.23052,63.17295],[-169.07503,63.17769],[-168.93915,63.13765],[-168.84165,63.15384],[-168.75154,63.21796],[-168.68514,63.29643],[-168.93738,63.33379],[-169.46273,63.36046],[-169.56656,63.38873],[-169.65647,63.42993],[-169.85708,63.44198],[-170.00794,63.47543],[-170.09583,63.6127],[-170.17641,63.62549],[-170.26748,63.67582],[-170.34485,63.69423],[-170.48819,63.69672],[-170.60628,63.67273],[-170.85903,63.5875],[-170.95082,63.57013],[-171.30933,63.62109],[-171.55286,63.66625],[-171.61318,63.78507],[-171.7434,63.78297],[-171.80282,63.71639],[-171.79188,63.62063],[-171.83683,63.56488],[-171.84998,63.48504]]],[[[-171.31266,52.4935],[-171.30417,52.44995],[-171.22673,52.43427],[-171.19601,52.50011],[-171.25677,52.52858],[-171.31266,52.4935]]],[[[-170.84194,52.55817],[-170.78849,52.54024],[-170.68591,52.58123],[-170.58496,52.58719],[-170.53214,52.67997],[-170.67154,52.69808],[-170.81794,52.63628],[-170.84194,52.55817]]],[[[-170.42187,57.1612],[-170.28632,57.12817],[-170.13388,57.18133],[-170.144,57.2428],[-170.30309,57.23803],[-170.42005,57.21292],[-170.42187,57.1612]]],[[[-170.20789,52.7089],[-170.07773,52.72042],[-169.9515,52.78862],[-169.81855,52.79158],[-169.70387,52.77712],[-169.66651,52.86435],[-169.66238,52.95175],[-169.68003,53.03508],[-169.74746,53.0932],[-169.8202,53.06679],[-169.76274,52.97805],[-169.85757,52.90853],[-170.02634,52.94491],[-170.09222,52.91939],[-170.17068,52.78492],[-170.20789,52.7089]]],[[[-169.81831,56.63361],[-169.68582,56.53972],[-169.58262,56.53694],[-169.45379,56.58379],[-169.47432,56.62518],[-169.61369,56.62276],[-169.81831,56.63361]]],[[[-169.28652,52.78475],[-169.26176,52.7549],[-169.00504,52.82999],[-168.85102,52.90804],[-168.75553,52.90751],[-168.68847,52.9664],[-168.61396,53.00878],[-168.4571,53.05584],[-168.29623,53.22724],[-168.09201,53.28827],[-167.84233,53.38649],[-167.85684,53.42861],[-167.80812,53.47386],[-167.78916,53.51933],[-167.91467,53.52272],[-168.02701,53.56276],[-168.23832,53.5219],[-168.34213,53.47599],[-168.39535,53.39778],[-168.44508,53.26533],[-168.58189,53.28652],[-168.76333,53.18281],[-168.8049,53.12002],[-168.78524,53.04504],[-168.86106,53.01638],[-168.95946,52.93674],[-169.28652,52.78475]]],[[[-168.12893,65.65574],[-166.34719,65.27634],[-166.91192,65.12596],[-166.41393,64.65123],[-165.00196,64.43392],[-160.79336,64.61932],[-161.5049,64.42307],[-160.76562,63.82871],[-161.07357,63.5617],[-162.70756,63.57761],[-162.35227,63.45407],[-163.054,63.05833],[-164.44237,63.20267],[-166.21179,61.60837],[-164.99167,60.69884],[-165.42035,60.55069],[-164.13339,59.84561],[-162.37224,60.16701],[-161.70253,59.49091],[-162.04858,59.25418],[-161.82411,58.73455],[-162.17172,58.64844],[-160.32292,58.95395],[-158.88093,58.39067],[-158.61968,58.91105],[-158.14031,58.61502],[-156.99355,58.8368],[-157.54104,58.3773],[-157.67889,57.56389],[-158.65635,56.81001],[-160.38592,56.27971],[-160.50843,55.86938],[-161.80783,55.89195],[-163.34377,54.97444],[-164.57626,54.89534],[-164.74398,54.39422],[-161.55036,55.06573],[-161.86334,55.26699],[-161.58705,55.62006],[-161.25398,55.3559],[-158.43147,55.99445],[-158.1178,56.23074],[-158.37195,56.46733],[-154.22246,58.13257],[-153.26741,58.86722],[-154.12268,59.28762],[-152.03560881493112,60.44099409412744],[-152.0641,60.41714],[-152.07999,60.34119],[-151.95626,60.36784],[-151.83919,60.48586],[-151.9217706731372,60.50390424525599],[-150.536,61.26972],[-149.71717,61.0113],[-151.28181,60.49603],[-151.86947,59.76916],[-151.20546,59.63028],[-151.85812,59.14423],[-145.98855,60.38743],[-139.58579,59.64277],[-139.86131,59.54668],[-136.70125,58.21942],[-136.37970542420462,58.21685979743296],[-136.36554,58.14885],[-136.44629,58.11334],[-136.53871,58.09348],[-136.56322,58.03505],[-136.57329,57.92684],[-136.48426,57.89646],[-136.45883,57.8539],[-136.37199,57.83223],[-136.30468,57.77105],[-136.25082,57.68483],[-136.23817,57.62599],[-136.16306,57.55886],[-136.08807,57.55529],[-136.04755,57.51376],[-135.94377,57.45878],[-135.89213,57.40805],[-135.83772,57.28207],[-135.87052,57.22164],[-135.83225,57.17065],[-135.75358,57.16717],[-135.75501,57.12397],[-135.84461,57.08357],[-135.85602,56.99564],[-135.8256,56.98903],[-135.63688,57.00987],[-135.60455,57.04583],[-135.5715,57.1057],[-135.45791,57.07017],[-135.35345,57.02091],[-135.44234,56.94236],[-135.47682,56.89123],[-135.55072,56.84123],[-135.46718,56.77141],[-135.36224,56.75874],[-135.21583,56.66534],[-135.17583,56.67788],[-135.12339,56.60282],[-134.91591,56.36056],[-134.83941,56.3094],[-134.81017,56.24499],[-134.76353,56.21036],[-134.67403,56.16693],[-134.65383,56.19839],[-134.63467,56.26583],[-134.64177,56.44548],[-134.66978,56.52413],[-134.62694,56.55387],[-134.61596,56.63729],[-134.62967,56.7096],[-134.69574,56.90079],[-134.73822,56.97574],[-134.85495,57.26477],[-134.82558,57.37214],[-134.82489,57.50007],[-134.93992,57.76361],[-135.00495,57.88434],[-134.9264,57.92192],[-134.95084,58.03699],[-135.1089,58.08827],[-135.2758,58.09702],[-135.6448021009046,58.211008349402206],[-134.88050053285494,58.20492282663551],[-134.8643,58.18049],[-134.78377,58.08229],[-134.70902,57.7805],[-134.69543,57.68534],[-134.55554,57.40743],[-134.64017,57.23985],[-134.63456,57.10986],[-134.56569,57.02374],[-134.49772,57.03119],[-134.38605,57.08739],[-134.37836,57.11502],[-134.30272,57.13656],[-134.19363,57.18488],[-134.10012,57.26629],[-133.9835,57.30284],[-133.87567,57.26761],[-133.84089,57.27107],[-133.7864,57.31153],[-133.86693,57.36787],[-133.87158,57.48416],[-133.8176,57.56835],[-133.80829,57.6096],[-133.89685,57.68552],[-133.90487,57.80741],[-133.99995,57.91481],[-134.13823,58.0471],[-134.18398,58.0773],[-134.17435,58.12528],[-134.25622,58.14479],[-134.32987,58.13499],[-134.46176,58.15929],[-134.60891,58.17164],[-134.69996,58.16149],[-134.72064499639438,58.20365002471127],[-134.14668,58.19908],[-133.46693,57.15936],[-133.20225817604185,57.00431389421799],[-133.33427,57.00244],[-133.88796,57.09774],[-134.00886,57.07458],[-134.04922,57.0292],[-133.92145,56.96151],[-133.86904,56.84594],[-133.94294,56.80555],[-134.00663,56.85159],[-134.04775,56.923],[-134.1471,56.95697],[-134.19375,56.9336],[-134.27046,56.93558],[-134.39298,56.86428],[-134.41853,56.82233],[-134.37627,56.66861],[-134.30112,56.62032],[-134.32013,56.55448],[-134.24194,56.55553],[-134.19797,56.53103],[-134.25192,56.44455],[-134.24313,56.39578],[-134.29468,56.33589],[-134.25975,56.13444],[-134.20218,56.03518],[-134.2918,55.92622],[-134.37496,55.92849],[-134.34465,55.84631],[-134.31176,55.81229],[-134.20825,55.87671],[-134.11806,55.91464],[-134.0998,55.98414],[-134.12187,56.0299],[-134.08745,56.09494],[-133.96052,56.09136],[-133.92773,56.14595],[-133.94199,56.1801],[-133.88114,56.2232],[-133.87662,56.27588],[-133.83455,56.3198],[-133.82163,56.3916],[-133.65547,56.44228],[-133.30057,56.46234],[-133.18349,56.45424],[-133.04173,56.51836],[-132.93304,56.5222],[-132.80684,56.50549],[-132.67115,56.54307],[-132.61133,56.59991],[-132.61926,56.66078],[-132.74321,56.71372],[-132.77945852366625,56.756635699212275],[-132.45079,56.5641],[-133.07006,56.33095],[-132.06741,55.87508],[-131.59457,54.93113],[-131.09381,55.19134],[-130.68619,54.71691],[-129.98235,55.30208],[-130.10276,56.1167],[-131.83513,56.60185],[-133.37991,58.42791],[-135.23115,59.69718],[-137.52642,58.90683],[-139.20035,60.0907],[-139.08667,60.35765],[-141.00184,60.30611],[-141.00267,69.64561],[-143.28188,70.15105],[-151.17519,70.37556],[-155.06076,71.14542],[-155.73184,70.83116],[-155.5877,71.17256],[-156.56865,71.35256],[-161.87927,70.32927],[-163.57403,69.12408],[-166.22419,68.87318],[-166.32846,68.44226],[-166.83897,68.33719],[-160.99396,66.23444],[-163.69539,66.05955],[-163.60396,66.55809],[-164.40072,66.58111],[-168.12893,65.65574]],[[-132.92878771402212,56.84411350133721],[-132.82715527739933,56.78457670106945],[-132.90321,56.80361],[-132.92878771402212,56.84411350133721]]],[[[-167.85151,53.30867],[-167.74775,53.27356],[-167.62217,53.25036],[-167.53925,53.27786],[-167.48821,53.26912],[-167.41771,53.32986],[-167.30813,53.33433],[-167.29183,53.3641],[-167.16635,53.41279],[-167.07539,53.42498],[-166.74916,53.44094],[-166.58101,53.53045],[-166.44491,53.64065],[-166.32026,53.67428],[-166.24406,53.71071],[-166.13866,53.73108],[-166.09753,53.82693],[-166.11304,53.85372],[-166.19875,53.8361],[-166.33677,53.78709],[-166.4049,53.80935],[-166.32,53.86953],[-166.25093,53.87685],[-166.21069,53.91592],[-166.07528,53.96957],[-166.17237,53.99812],[-166.26452,53.97755],[-166.35712,54.00234],[-166.50839,53.92395],[-166.58739,53.95983],[-166.64463,54.0145],[-166.74259,54.0155],[-166.87949,53.98872],[-167.03125,53.9452],[-167.14099,53.86677],[-167.14197,53.82693],[-167.00578,53.75545],[-167.07182,53.66556],[-167.10784,53.63306],[-167.16164,53.60591],[-167.13569,53.55123],[-167.27883,53.47857],[-167.36979,53.45065],[-167.45737,53.44279],[-167.59122,53.39335],[-167.69448,53.38803],[-167.79093,53.33552],[-167.85151,53.30867]]],[[[-167.43062,60.19701],[-167.34351,60.12629],[-167.33329,60.06663],[-167.06578,59.98796],[-166.93957,59.96548],[-166.79519,59.91319],[-166.66553,59.87817],[-166.51282,59.84643],[-166.36213,59.8386],[-166.27217,59.81146],[-166.19041,59.75021],[-166.08428,59.77606],[-166.08499,59.83965],[-165.98232,59.87164],[-165.85682,59.86973],[-165.76983,59.9005],[-165.70661,59.88356],[-165.5823,59.90824],[-165.53263,59.95343],[-165.70922,60.066],[-165.66668,60.1243],[-165.72348,60.16382],[-165.68355,60.19834],[-165.72253,60.23647],[-165.67987,60.29244],[-165.88309,60.34337],[-166.03734,60.31946],[-166.1033,60.36732],[-166.14976,60.4367],[-166.24103,60.38882],[-166.37187,60.35516],[-166.49036,60.38947],[-166.61657,60.31922],[-166.71354,60.32734],[-166.83582,60.26875],[-166.81313,60.24977],[-166.8426,60.21047],[-167.11284,60.23148],[-167.3176,60.23154],[-167.43062,60.19701]]],[[[-166.11224,54.12253],[-166.04644,54.04419],[-165.90165,54.06287],[-165.87513,54.03642],[-165.78443,54.06944],[-165.54922,54.1122],[-165.38372,54.19673],[-165.47845,54.29533],[-165.62555,54.29896],[-165.86819,54.21488],[-165.95975,54.22098],[-166.06155,54.18509],[-166.11224,54.12253]]],[[[-165.60222,54.04527],[-165.55577,54.02355],[-165.28766,54.03835],[-165.08828,54.07249],[-164.95614,54.06099],[-164.81617,54.15875],[-164.7637,54.22315],[-164.82418,54.22553],[-165.00791,54.13493],[-165.28077,54.11563],[-165.60222,54.04527]]],[[[-162.86174,54.42477],[-162.76025,54.37219],[-162.60861,54.36915],[-162.46695,54.34269],[-162.38875,54.36762],[-162.34484,54.40134],[-162.84436,54.51043],[-162.86174,54.42477]]],[[[-161.69713,55.24915],[-161.69155,55.19848],[-161.56021,55.20705],[-161.52343,55.27166],[-161.69713,55.24915]]],[[[-161.45129,55.17803],[-161.34415,55.1585],[-161.32987,55.21942],[-161.426,55.21656],[-161.45129,55.17803]]],[[[-161.07849,58.63558],[-161.07563,58.54992],[-160.96142,58.55372],[-160.88051,58.58133],[-160.67931,58.78023],[-160.70063,58.81737],[-161.05659,58.7022],[-161.07849,58.63558]]],[[[-160.85662,55.31849],[-160.82138,55.11785],[-160.73494,55.15131],[-160.65558,55.16026],[-160.52523,55.12987],[-160.48651,55.18195],[-160.52762,55.25637],[-160.46826,55.28893],[-160.34122,55.2518],[-160.30655,55.30328],[-160.15404,55.37752],[-160.13703,55.45071],[-160.26057,55.46367],[-160.51751,55.37938],[-160.68744,55.4022],[-160.80893,55.37012],[-160.85662,55.31849]]],[[[-160.25275,54.91325],[-160.22697,54.86408],[-160.02704,55.02091],[-159.81362,55.02747],[-159.75278,55.06614],[-159.63523,55.03729],[-159.50443,55.02732],[-159.44798,54.94137],[-159.30968,54.86581],[-159.27235,54.8642],[-159.20323,54.91484],[-159.48877,55.18881],[-159.5211,55.25339],[-159.67031,55.18234],[-159.81642,55.17805],[-159.84386,55.24937],[-159.87059,55.28489],[-160.02526,55.20391],[-160.1371,55.17157],[-160.18726,55.11838],[-160.19206,55.03816],[-160.25275,54.91325]]],[[[-157.32606,56.52517],[-157.16878,56.53021],[-157.04717,56.51993],[-156.97555,56.54045],[-157.09115,56.58113],[-157.17203,56.59804],[-157.2887,56.56604],[-157.32606,56.52517]]],[[[-156.73529,56.02264],[-156.68181,55.99434],[-156.6182,56.0178],[-156.61476,56.06518],[-156.683,56.09881],[-156.73363,56.07759],[-156.73529,56.02264]]],[[[-155.75,55.82185],[-155.71859,55.77236],[-155.591,55.76173],[-155.56631,55.78949],[-155.53059,55.91221],[-155.60537,55.92883],[-155.75,55.82185]]],[[[-154.84041,56.42032],[-154.74289,56.40168],[-154.62428,56.47518],[-154.52951,56.50266],[-154.02093,56.48203],[-153.88768,56.53364],[-153.87876,56.56593],[-154.02504,56.57252],[-154.09583,56.61779],[-154.51408,56.60406],[-154.70614,56.52127],[-154.84041,56.42032]]],[[[-154.79384,57.28886],[-154.59498,57.25716],[-154.52343,57.12911],[-154.52854,57.00189],[-154.31289,56.91867],[-154.30571,56.84687],[-154.12902,56.74217],[-154.15315,56.6817],[-154.01704,56.68931],[-153.97178,56.74486],[-153.83964,56.8221],[-153.54149,56.88788],[-153.54343,56.99525],[-153.58083,57.04905],[-153.48652,57.08592],[-153.40426,57.08051],[-153.34202,56.98283],[-153.06433,57.10379],[-152.90054,57.13208],[-152.88032,57.1648],[-152.94933,57.18735],[-152.94346,57.25696],[-152.6957,57.28132],[-152.63044,57.32267],[-152.57053,57.44891],[-152.47488,57.4342],[-152.32369,57.34266],[-152.2536,57.38402],[-152.32368,57.46786],[-152.15968,57.59361],[-152.16162,57.62329],[-152.31397,57.63642],[-152.38681,57.66792],[-152.44018,57.72664],[-152.29876,57.74593],[-152.21225,57.79143],[-152.35115,57.83477],[-152.3241,57.9166],[-152.42257,57.94866],[-152.43261,57.97603],[-152.52628,57.91327],[-152.63538,57.91861],[-152.75344,57.83445],[-152.80481,57.89918],[-152.72252,57.98736],[-152.76667,58.02989],[-152.52904,58.09378],[-152.48267,58.12981],[-152.1122,58.14856],[-152.03412,58.18374],[-151.86232,58.16827],[-151.7957,58.2111],[-151.81711,58.26344],[-152.12926,58.39641],[-152.22783,58.37642],[-152.30171,58.4287],[-152.35609,58.42347],[-152.34486,58.39163],[-152.38734,58.3595],[-152.49848,58.37235],[-152.51248,58.42735],[-152.33721,58.5891],[-152.35471,58.63828],[-152.56017,58.61968],[-152.61613,58.60185],[-152.66622,58.54409],[-152.64031,58.46987],[-152.73384,58.46066],[-152.78778,58.41131],[-152.88311,58.40044],[-152.92559,58.33969],[-153.04432,58.30634],[-153.2028,58.20808],[-153.22371,58.16212],[-153.31613,58.14039],[-153.41978,58.05964],[-153.29901,57.98563],[-153.38642,57.93653],[-153.4846,57.9765],[-153.5332,57.94112],[-153.51202,57.90916],[-153.6488,57.8801],[-153.72118,57.89062],[-153.93522,57.81305],[-153.93028,57.69679],[-153.99457,57.65691],[-154.19696,57.66464],[-154.52206,57.57779],[-154.77966,57.36634],[-154.79384,57.28886]]],[[[-153.59741,59.38683],[-153.54669,59.33135],[-153.51529,59.32088],[-153.3869,59.33075],[-153.34777,59.37799],[-153.41249,59.41511],[-153.489,59.41523],[-153.59741,59.38683]]],[[[-133.93502,55.92069],[-133.92025,55.8603],[-133.70047,55.83742],[-133.70115,55.78516],[-133.64332,55.72904],[-133.71666,55.66022],[-133.72855,55.59313],[-133.78905,55.45789],[-133.6979,55.45476],[-133.63094,55.41611],[-133.63301,55.3613],[-133.69017,55.30441],[-133.65836,55.23267],[-133.59676,55.21823],[-133.5866,55.3088],[-133.46822,55.28168],[-133.47194,55.24753],[-133.4045,55.21499],[-133.34126,55.2057],[-133.28198,55.21712],[-133.23249,55.19883],[-133.21509,55.13688],[-133.2397,55.09242],[-133.16479,54.97691],[-132.99059,54.82099],[-132.87721,54.75377],[-132.86635,54.70039],[-132.75302,54.67324],[-132.67432,54.67465],[-132.63903,54.75325],[-132.50937,54.78126],[-132.40353,54.7846],[-132.36639,54.7512],[-132.30794,54.71871],[-132.22822,54.72517],[-132.16518,54.69405],[-132.02975,54.70119],[-131.95791,54.79124],[-131.98332,54.89781],[-131.98459,55.02798],[-132.02751,55.10468],[-131.9774,55.18095],[-132.03712,55.27514],[-132.1264,55.28842],[-132.25806,55.41614],[-132.14294,55.45794],[-132.30112,55.55096],[-132.3825,55.66534],[-132.46253,55.67385],[-132.4707,55.78216],[-132.61846,55.91148],[-132.83759,56.02433],[-132.89643,56.09974],[-132.83386,56.1039],[-132.88758,56.17294],[-132.96692,56.22428],[-133.07823,56.2468],[-133.15823,56.31477],[-133.19701,56.33302],[-133.41837,56.33213],[-133.58212,56.35251],[-133.65642,56.32691],[-133.65689,56.28123],[-133.6721,56.22273],[-133.64382,56.12774],[-133.65924,56.08382],[-133.81636,55.96403],[-133.93502,55.92069]]],[[[172.45891,52.95456],[172.47286,52.89024],[172.58508,52.92133],[172.66995,52.91302],[172.75424,52.8775],[172.76337,52.82366],[172.80939,52.7893],[172.90363,52.76167],[172.9826,52.79109],[173.09624,52.78679],[173.1669,52.79524],[173.20495,52.84892],[173.28442,52.82794],[173.42382,52.82881],[173.42537,52.86835],[173.12199,52.99036],[172.79288,53.00858],[172.64327,53.00499],[172.45891,52.95456]]],[[[173.31995,52.41207],[173.48638,52.36862],[173.54378,52.39267],[173.6513,52.35638],[173.7257,52.35659],[173.7483,52.39235],[173.70226,52.43481],[173.7728,52.50991],[173.63806,52.52422],[173.55574,52.47948],[173.43903,52.47054],[173.31995,52.41207]]],[[[173.81904,52.75981],[173.97512,52.70747],[174.15815,52.70607],[174.14012,52.75074],[173.864,52.79249],[173.81904,52.75981]]],[[[177.17879,51.87923],[177.2622,51.8619],[177.31177,51.82598],[177.33423,51.86678],[177.40954,51.93083],[177.60101,51.92226],[177.61155,51.95084],[177.57207,52.00182],[177.67595,52.09217],[177.64866,52.13091],[177.58127,52.14493],[177.52101,52.06307],[177.46054,51.99976],[177.36736,51.96838],[177.31083,51.93328],[177.21309,51.92037],[177.17879,51.87923]]],[[[178.07921,52.01897],[178.12094,51.97702],[178.2009,51.99116],[178.20131,52.03151],[178.15477,52.06143],[178.09367,52.05515],[178.07921,52.01897]]],[[[178.20444,51.8309],[178.27096,51.76519],[178.37407,51.74786],[178.37801,51.79263],[178.3297,51.8368],[178.20444,51.8309]]],[[[178.43246,51.96554],[178.50249,51.89965],[178.5394,51.90325],[178.5916,51.95266],[178.55261,51.97398],[178.46339,51.98786],[178.43246,51.96554]]],[[[178.60049,51.65526],[178.60487,51.61602],[178.7724,51.55413],[178.86925,51.55699],[178.93022,51.5301],[179.03153,51.44989],[179.25327,51.33725],[179.48042,51.36387],[179.41824,51.4162],[179.29579,51.41924],[178.92533,51.62391],[178.6606,51.68307],[178.60049,51.65526]]],[[[179.48132,51.97531],[179.48463,51.92128],[179.54352,51.89093],[179.64948,51.87368],[179.74301,51.91176],[179.77392,51.9707],[179.63685,52.02572],[179.58286,52.01685],[179.48132,51.97531]]]]}]},{"op":"in","args":[{"property":"collection"},["lis-tws-anomaly"]]}]}}' | jq .context

 out > 
  {
  "limit": 10000,
  "matched": 93,
  "returned": 93
}

[j08lue]

Here is a notebook for testing different combinations of failing / working GHG Center API stages and complex/simple geometries: https://gist.github.com/j08lue/4fc288bbe534da2a7dc0b9c2f15240a0

Results:

Only current GHG Center production STAC is failing for the complex Alaska geometry.

[bitner]

Have you checked the validity of the polygons that are being used? (ie making sure there are no self-intersections etc - in PostGIS, you can do this with ST_IsValid(geom)

[j08lue]

✅

[anayeaye]

See note in PR 928

We identified an AWS WAF size restriction associated with the GHG cloudfront that was causing everything > 16 KB to be rejected with 'exceeds size limit'. This can be adjusted up to a maximum 64 KB for more wiggle room. The geojson in this PR is small enough that it works without increasing that size limit on the GHG application firewall.

[j08lue]

This can be adjusted up to a maximum 64 KB for more wiggle room. The geojson in this PR is small enough that it works without increasing that size limit on the GHG application firewall.

Thanks for checking, @anayeaye!

I think we should crank up the maximum size to 64 KB either way. Users may provide more complex geometries.

And can we fail more descriptively than a 403 when the payload exceeds that size? Something the client (VEDA UI) can catch and tell the user what to do? This is stuff for the wish list, though...

[anayeaye]

@j08lue: @amarouane-ABDELHAK is already working on a solution to update the existing WAF payload size restrictions. As for error handling I will do some googling to see what might be available, perhaps there is a way to set the response code in the WAF policy? These cutoffs at the cloud front never make it to to the veda backend APIs.

[amarouane-ABDELHAK]

The issue is coming from the WAF blocking oversized requests, considering them as potential attacks. I've temporarily resolved this by overriding the rule to permit such requests. However, we need a more permanent solution. One approach is to compress the JSON data (using gzip) before transmission and decompress it on the server side.

Here is the new WAF rule configuration with the override

[amarouane-ABDELHAK]

What makes me nervous about allowing oversized requests is the attacks to a .gov domain

[j08lue]

This is fixed for now, but we need follow up:

• https://github.com/US-GHG-Center/ghgc-architecture/issues/215