search

privacy-tech-lab / privacy-pioneer

Privacy browser extension for analyzing web traffic of visited websites
https://www.privacytechlab.org/
Other
26 stars 1 forks source link

Analyze HTTP(s) request logs #14

Closed rgoldstein01 closed 3 years ago

rgoldstein01 commented 3 years ago

Our selenium tool now is successfully creating network logs for the HTTP requests. Now, we must go through these logs to pull out important information we would like to give to the developer. Previously, we have discussed the importance of looking for personal data like emails, passwords, usernames. We also would like to focus on third parties and the data they are receiving.

cc @SebastianZimmeck

SebastianZimmeck commented 3 years ago

Here are a few ideas to get started analyzing the HTTP requests.

  1. The following sites asked me for location permission when I visited them. Maybe, their requests contain keywords such as "location", "latitude", "lat", "gps", "geo" that may be a good proxy for identifying that they are storing location data on their servers. (The other part of the analysis could be indeed the detection of the permission in the browser per @davebaraka's work. In combination with the HTTP request this could be strong evidence that location is being sent off to a server.)

Similarly, xfinity asked me for motion sensors; maybe also interesting to look at.

  1. Another approach is the other way around. Create a fresh account, say, on Yahoo, surf a bit around, and capture all requests during this whole time. Then examine all the captured requests and see whether there is certain data in there that you entered or the site got on its own. Repeat with other sites and see whether keywords can be used or whether there is a pattern to the data in the requests that can be used to identify it. Some random sites to do this type of analysis:

  1. Beyond keywords the format of data in HTTP requests could be revealing. For example, credit card numbers always have 16 digits and follow the Luhn algorithm. Identifying these type of numbers would indicate that a credit card number is sent (@rgoldstein01, make sure you are not posting any credit card numbers or other sensitive information here when if you decide to test this on a bank site; only do this test if you have no doubts). Latitude/longitude data also follows a distinct format.

  2. Identify who is receiving information (first party vs third party is important). Is that possible from the server URLs? One problem I could see here is that there are content delivery networks (e.g., Cloudflare making it hard to know the first or third party behind a request).

All of these would work best with a fresh browser, no extensions installed/all extensions disabled, and default settings that are not blocking HTTP requests.

rgoldstein01 commented 3 years ago

OK so first I just wanted to document some patterns I found with data collection. Here's fedex:

{"pageref": "fedex.com", "startedDateTime": "2020-11-30T16:18:24.595-05:00", "request": {"method": "POST", "url": "https://www.fedex.com/akam/11/pixel_5d5afff2", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.fedex.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/x-www-form-urlencoded"}, {"name": "Content-Length", "value": "2799"}, {"name": "Origin", "value": "https://www.fedex.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://www.fedex.com/global/choose-location.html"}, {"name": "Cookie", "value": "siteDC=edc; xacc=US; Rbt=f0; ak_bmsc=2AAB8E76CCB7EE5D5643A5C3D4E314E51726711EF97B00009F61C55FD24BF122~plFAUwD5CKqdn/KCBKBDZNmwgB0mW8PS4/7eHMVQNOv0o5BvBzVGEBJ7jWL8Cuh/2qucHdkdgU4X5PsVufhj2GglnBB3v6g/0zfRTKj7gQsIPNZ/AHxo9naVpU6qZ1YhC1felzn0vr4VcbCzgkNJV/MHXemgvBlTmQ2TKhJHOvhglM2I7ZtKtU4EN7jDulRYgVB/pDUS8KZZJOe4oqJsDV1MXHI49/HPZ0LjNjqwrprFo=; bm_sz=6433DAAD6B2693CAA791D98F2749BBCF~YAAQHnEmF7k7pIl1AQAARVUFGwkg7Kah5HmmD2JuvNFz7yvmb/s4C/y14EQwRGZPXkl2gQxWKzLcWId1NYRhe/HUxrY8WTTakiB8asiWwR20dHR3a9jpMi80c4ypLxc4GrQ3wo36MwL+EwdXQoA9/2vSZwoKCz9g6Dy/VGUemcWOfsW3OhL2D1uqABQm9Q==; _abck=9D6AE8E989E6FD529A6D3B02C6A7307E~0~YAAQHnEmF8E7pIl1AQAAUFsFGwRFfTs839U+tnTv3fAc2dZ/Naw9knBKJjXAi2efx+rjwzw5DG2+TyvpPYSurCqsuBgjm3Aav6aq9nOc5zGMthero83JDKJKEtpiP4ZCEb9PPnZIzWs8InPJPQu8epHsTB7jOcNCSverAIJGA5NVIlMZP4fGXtrtpAuwLSz0DlaFAeZ+FgvWy5F0XmiT7P7PHoNvKALGvxx8oZQeXK33S/Xq61ZpLztB8B/W3R1OgHjvpoJxyYfYFJzziTlG+KL0sth5B5bWZ0kYEPly+FbVvPF6lzbwV1UF21yR5H1B9/4WHSp7jMUjAf5RgLmOTV+fw7ZZ~-1~-1~-1; fdx_cbid=31211212281606771103016320340101; fdx_locale=en_US; isTablet=false; isMobile=false; isWireless=false; level=test; aemserver=Prod-c0016058.prod.cloud.fedex.com; bm_mi=994D064632720E3964E8A7F7C743CDF9~zkpWL3ZBG/BqQRmFVQOMr/G9naYUk9CFNzQcsMQ3xT18OXG9Ovsm7UyhiENiKExgw/SFzt5dM2cm/xIu8Ef6fTt9xsV86/gcVkS8uP4S+ZY7lA7yqWKomHJVJXrMGCa9RwWvvt6Bg6z8dg+PQYQuQId7kPILYpML/QKrpG1/VLt/G4DT3wRMxoD8YF14oUgX5opLAAB33nruGRyTk50azKyIJmRB0UOJMhuQCDjkRlDMUGXtBhdGvV1v3NLE5YKHlzF8h+1IOp9bCyFmAIlaImHmWcbYoMN/UPRoRVQs5EBrqGHkevO0gO1u+qONi8wj; bm_sv=8ED49CF2C4029014952021ED31D2891C~z7GKfio/Yss+VK2zut5oLYF9tdZYqrVovBwUWd3OrrEuDmav393Mix4uY1DGVtfq2IXqn1N3c23X+X2mLb02/DVsE8Ah7RdhKyIn+8xOfqU1pjNKmQEOKN93p07U6LwwPiDAKm9aeM3XK3weayCZlRGIaJtOyIp+zCfiYnZrCCk=; AMCV_1E22171B520E93BF0A490D44%40AdobeOrg=359503849%7CMCIDTS%7C18597%7CvVersion%7C5.0.1; mbox=session#fb769ec4f59d4fd1b06f62d7b367c0d8#1606772965; at_check=true; s_pers=%20s_dfa%3Dfedexglbl%252Cfedexus%7C1606772904239%3B; s_sess=%20setLink%3D%3B"}], "queryString": [], "postData": {"mimeType": "application/x-www-form-urlencoded", "params": [{"name": "ap", "value": "true", "comment": ""}, {"name": "bt", "value": "0", "comment": ""}, {"name": "fonts", "value": "4,14,15,16,21,22,23,24,43,47,48,49,50,51", "comment": ""}, {"name": "fh", "value": "4e48896ac5550f63acb8438a67b8b7041fd7b9eb", "comment": ""}, {"name": "timing", "value": "{\"1\":115,\"2\":407,\"3\":552,\"4\":670,\"profile\":{\"bp\":0,\"sr\":90,\"dp\":0,\"lt\":2,\"ps\":0,\"cv\":14,\"fp\":0,\"sp\":1,\"br\":0,\"ieps\":0,\"av\":0,\"z1\":5,\"jsv\":1,\"nav\":0,\"nap\":0,\"crc\":0,\"z2\":3,\"z3\":1,\"z4\":1,\"fonts\":72},\"main\":400,\"compute\":115,\"send\":742}", "comment": ""}, {"name": "bp", "value": "", "comment": ""}, {"name": "sr", "value": "{\"inner\":[1280,758],\"outer\":[1280,832],\"screen\":[4,4],\"pageOffset\":[0,0],\"avail\":[1440,900],\"size\":[1440,900],\"client\":[1280,4768],\"colorDepth\":24,\"pixelDepth\":24}", "comment": ""}, {"name": "dp", "value": "{\"XDomainRequest\":0,\"createPopup\":0,\"removeEventListener\":1,\"globalStorage\":0,\"openDatabase\":0,\"indexedDB\":1,\"attachEvent\":0,\"ActiveXObject\":0,\"dispatchEvent\":1,\"addBehavior\":0,\"addEventListener\":1,\"detachEvent\":0,\"fireEvent\":0,\"MutationObserver\":1,\"HTMLMenuItemElement\":1,\"Int8Array\":1,\"postMessage\":1,\"querySelector\":1,\"getElementsByClassName\":1,\"images\":1,\"compatMode\":\"CSS1Compat\",\"documentMode\":0,\"all\":1,\"now\":1,\"contextMenu\":null}", "comment": ""}, {"name": "lt", "value": "1606771103940-5", "comment": ""}, {"name": "ps", "value": "true,true", "comment": ""}, {"name": "cv", "value": "fb1df2e3f14da83d955799ae20c68696a2efb3b4", "comment": ""}, {"name": "fp", "value": "false", "comment": ""}, {"name": "sp", "value": "false", "comment": ""}, {"name": "br", "value": "Firefox", "comment": ""}, {"name": "ieps", "value": "false", "comment": ""}, {"name": "av", "value": "false", "comment": ""}, {"name": "z", "value": "{\"a\":1566243968,\"b\":1,\"c\":0}", "comment": ""}, {"name": "zh", "value": "", "comment": ""}, {"name": "jsv", "value": "1.5", "comment": ""}, {"name": "nav", "value": "{\"userAgent\":\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0\",\"appName\":\"Netscape\",\"appCodeName\":\"Mozilla\",\"appVersion\":\"5.0 (Macintosh)\",\"appMinorVersion\":0,\"product\":\"Gecko\",\"productSub\":\"20100101\",\"vendor\":\"\",\"vendorSub\":\"\",\"buildID\":\"20181001000000\",\"platform\":\"MacIntel\",\"oscpu\":\"Intel Mac OS X 10.15\",\"hardwareConcurrency\":2,\"language\":\"en-US\",\"languages\":[\"en-US\",\"en\"],\"systemLanguage\":0,\"userLanguage\":0,\"doNotTrack\":\"unspecified\",\"msDoNotTrack\":0,\"cookieEnabled\":true,\"geolocation\":1,\"vibrate\":1,\"maxTouchPoints\":0,\"webdriver\":true,\"plugins\":[]}", "comment": ""}, {"name": "crc", "value": "{\"window.chrome\":\"-not-existent\"}", "comment": ""}, {"name": "t", "value": "16f9db8127b564a9edb2bae2b097a2e1ac3ff2c4", "comment": ""}, {"name": "u", "value": "3b9fcf18117a5290bfae2ff069f51a44", "comment": ""}, {"name": "nap", "value": "11133333331333333333", "comment": ""}, {"name": "fc", "value": "true", "comment": ""}], "comment": ""}, "headersSize": 2400, "bodySize": 2799, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Type", "value": "text/html"}, {"name": "Content-Length", "value": "0"}, {"name": "Date", "value": "Mon, 30 Nov 2020 21:18:24 GMT"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Set-Cookie", "value": "Rbt=f0; path=/"}, {"name": "Set-Cookie", "value": "ak_bmsc=2AAB8E76CCB7EE5D5643A5C3D4E314E51726711EF97B00009F61C55FD24BF122~plKaPJNxPs0dZeP030mxq2Ww6B3fyfrxVSyxZg2ciczK8M4PsiG0AyL7mkU08DFtVWecvCuK+RAH4bycBPi+JKKl8LFLTD1cnc5OEUGT/jSqHiXE/4h728i0Sc0C4YNf+TKCsuzOIzJrQzEmooZZFMR3cB2AQUpT8JIIeauSDzqFAX6/ppPRAoq3ZrFXHw+KB2eW+UbkTH04TRuTQQnJlAiaoTm1NgkZP7IJDuCa39UyDI6Pge+EDgsjxXi28F7X7ghkwBPNlCKJ4zsb3OYkCmoOqf4p4F1+T7rcyFEMVuf70=; expires=Mon, 30 Nov 2020 23:18:23 GMT; max-age=7199; path=/; domain=.fedex.com; HttpOnly"}], "content": {"size": 0, "mimeType": "text/html", "text": "", "comment": ""}, "redirectURL": "", "headersSize": 633, "bodySize": 0, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": -1, "dns": -1, "connect": -1, "send": 0, "wait": 35, "ssl": -1}, "serverIPAddress": "23.38.113.34", "comment": "", "time": 35}

So this has a few things that stand out. First off, we have geolocation = 1, so we can presume geolocation is ON. This is under postData --> params --> 19 --> value. Not sure if this is norm but this is where this was located. The harder part was finding the actual location itself, where that was stored

I believe I found it in this request under postData as well:

{"pageref": "fedex.com", "startedDateTime": "2020-11-30T16:18:24.464-05:00", "request": {"method": "POST", "url": "https://www.fedex.com/assets/2967da73ui262087b4967bee0c68b8", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.fedex.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "text/plain;charset=UTF-8"}, {"name": "Content-Length", "value": "1623"}, {"name": "Origin", "value": "https://www.fedex.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://www.fedex.com/global/choose-location.html"}, {"name": "Cookie", "value": "siteDC=edc; xacc=US; Rbt=f0; ak_bmsc=2AAB8E76CCB7EE5D5643A5C3D4E314E51726711EF97B00009F61C55FD24BF122~plFAUwD5CKqdn/KCBKBDZNmwgB0mW8PS4/7eHMVQNOv0o5BvBzVGEBJ7jWL8Cuh/2qucHdkdgU4X5PsVufhj2GglnBB3v6g/0zfRTKj7gQsIPNZ/AHxo9naVpU6qZ1YhC1felzn0vr4VcbCzgkNJV/MHXemgvBlTmQ2TKhJHOvhglM2I7ZtKtU4EN7jDulRYgVB/pDUS8KZZJOe4oqJsDV1MXHI49/HPZ0LjNjqwrprFo=; bm_sz=6433DAAD6B2693CAA791D98F2749BBCF~YAAQHnEmF7k7pIl1AQAARVUFGwkg7Kah5HmmD2JuvNFz7yvmb/s4C/y14EQwRGZPXkl2gQxWKzLcWId1NYRhe/HUxrY8WTTakiB8asiWwR20dHR3a9jpMi80c4ypLxc4GrQ3wo36MwL+EwdXQoA9/2vSZwoKCz9g6Dy/VGUemcWOfsW3OhL2D1uqABQm9Q==; _abck=9D6AE8E989E6FD529A6D3B02C6A7307E~-1~YAAQHnEmF787pIl1AQAAyFgFGwSGdNq6vVe3eqEXF7zqJANa9WdtxEpBIKvphjlbdig4jEAYEgXCsGOBEdy20AgOTOCId3/pXZx8YJ4NpLtvCLL22/jajhW3qDRJchGZHpT2B30xWM8PzWzhOEKu2oD/FVJYJC16DL8AOR2zsTuZ5Pkwydy4JHe9se3M0qkc18sp1Fn9vyJUj+eB1p4/Z9Gf3KBuBdd1yoO/4rmrFFt1040vB0Eqxu4POOsVnm6lRy8vx555Sch4ecpvCxH2/4vi69f99komIkTzv7mNLwkfALBhW19lBQ982BaUexNsmoUcR4FyfQ==~-1~-1~-1; fdx_cbid=31211212281606771103016320340101; fdx_locale=en_US; isTablet=false; isMobile=false; isWireless=false; level=test; aemserver=Prod-c0016058.prod.cloud.fedex.com; bm_mi=994D064632720E3964E8A7F7C743CDF9~zkpWL3ZBG/BqQRmFVQOMr/G9naYUk9CFNzQcsMQ3xT18OXG9Ovsm7UyhiENiKExgw/SFzt5dM2cm/xIu8Ef6fTt9xsV86/gcVkS8uP4S+ZY7lA7yqWKomHJVJXrMGCa9RwWvvt6Bg6z8dg+PQYQuQId7kPILYpML/QKrpG1/VLt/G4DT3wRMxoD8YF14oUgX5opLAAB33nruGRyTk50azKyIJmRB0UOJMhuQCDjkRlDMUGXtBhdGvV1v3NLE5YKHlzF8h+1IOp9bCyFmAIlaImHmWcbYoMN/UPRoRVQs5EBrqGHkevO0gO1u+qONi8wj; bm_sv=8ED49CF2C4029014952021ED31D2891C~z7GKfio/Yss+VK2zut5oLYF9tdZYqrVovBwUWd3OrrEuDmav393Mix4uY1DGVtfq2IXqn1N3c23X+X2mLb02/DVsE8Ah7RdhKyIn+8xOfqU1pjNKmQEOKN93p07U6LwwPiDAKm9aeM3XK3weayCZlRGIaJtOyIp+zCfiYnZrCCk=; AMCV_1E22171B520E93BF0A490D44%40AdobeOrg=359503849%7CMCIDTS%7C18597%7CvVersion%7C5.0.1; mbox=session#fb769ec4f59d4fd1b06f62d7b367c0d8#1606772965; at_check=true; s_pers=%20s_dfa%3Dfedexglbl%252Cfedexus%7C1606772904239%3B; s_sess=%20setLink%3D%3B"}], "queryString": [], "postData": {"mimeType": "text/plain;charset=UTF-8", "text": "{\"sensor_data\":\"7a74G7m23Vrp0o5c9115301.66-1,2,-94,-100,Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0,uaend,11059,20100101,en-US,Gecko,0,0,0,0,395342,1103842,1440,900,1440,900,1280,758,1280,,cpen:0,i1:0,dm:0,cwen:0,non:1,opc:0,fc:1,sc:0,wrc:1,isc:78,vib:1,bat:0,x11:0,x12:1,6006,0.726477817363,803385551921,0,loc:-1,2,-94,-101,do_en,dm_en,t_dis-1,2,-94,-105,0,-1,0,1,1386,447,0;1,0,0,1,1649,331,0;0,-1,0,0,1498,-1,0;-1,2,-94,-102,0,-1,0,1,1386,447,0;1,0,0,1,1649,331,0;0,-1,0,0,1498,-1,0;-1,2,-94,-108,-1,2,-94,-110,-1,2,-94,-117,-1,2,-94,-111,-1,2,-94,-109,-1,2,-94,-114,-1,2,-94,-103,3,132;-1,2,-94,-112,https://www.fedex.com/global/choose-location.html-1,2,-94,-115,1,32,32,0,0,0,0,531,0,1606771103842,7,17188,0,0,2864,0,0,532,0,0,9D6AE8E989E6FD529A6D3B02C6A7307E~-1~YAAQHnEmF787pIl1AQAAyFgFGwSGdNq6vVe3eqEXF7zqJANa9WdtxEpBIKvphjlbdig4jEAYEgXCsGOBEdy20AgOTOCId3/pXZx8YJ4NpLtvCLL22/jajhW3qDRJchGZHpT2B30xWM8PzWzhOEKu2oD/FVJYJC16DL8AOR2zsTuZ5Pkwydy4JHe9se3M0qkc18sp1Fn9vyJUj+eB1p4/Z9Gf3KBuBdd1yoO/4rmrFFt1040vB0Eqxu4POOsVnm6lRy8vx555Sch4ecpvCxH2/4vi69f99komIkTzv7mNLwkfALBhW19lBQ982BaUexNsmoUcR4FyfQ==~-1~-1~-1,30697,899,-1549651816,26067385,PiZtE,77905,87-1,2,-94,-106,9,1-1,2,-94,-119,0,0,0,0,0,200,0,0,0,200,200,0,0,200,-1,2,-94,-122,0,0,1,0,1,0,0-1,2,-94,-123,-1,2,-94,-124,-1,2,-94,-126,-1,2,-94,-127,11133333331333333333-1,2,-94,-70,50988738;1681675197;dis;;true;true;true;300;true;24;24;true;false;unspecified-1,2,-94,-80,6331-1,2,-94,-116,149018253-1,2,-94,-118,85269-1,2,-94,-129,d6350cc6832ca216bbeb88243f8742dbb972e8f7f09960559265cf71b2842f75,2,0,,,,0-1,2,-94,-121,;18;7;0\"}", "comment": ""}, "headersSize": 2391, "bodySize": 1623, "comment": ""}, "response": {"status": 201, "statusText": "Created", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Length", "value": "17"}, {"name": "Date", "value": "Mon, 30 Nov 2020 21:18:24 GMT"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Set-Cookie", "value": "Rbt=f0; path=/"}, {"name": "Content-Type", "value": "application/json"}, {"name": "Vary", "value": "Origin"}, {"name": "Access-Control-Allow-Credentials", "value": "true"}, {"name": "Access-Control-Allow-Origin", "value": "https://www.fedex.com"}, {"name": "Access-Control-Allow-Headers", "value": "Content-Type"}, {"name": "Set-Cookie", "value": "_abck=9D6AE8E989E6FD529A6D3B02C6A7307E~0~YAAQHnEmF8E7pIl1AQAAUFsFGwRFfTs839U+tnTv3fAc2dZ/Naw9knBKJjXAi2efx+rjwzw5DG2+TyvpPYSurCqsuBgjm3Aav6aq9nOc5zGMthero83JDKJKEtpiP4ZCEb9PPnZIzWs8InPJPQu8epHsTB7jOcNCSverAIJGA5NVIlMZP4fGXtrtpAuwLSz0DlaFAeZ+FgvWy5F0XmiT7P7PHoNvKALGvxx8oZQeXK33S/Xq61ZpLztB8B/W3R1OgHjvpoJxyYfYFJzziTlG+KL0sth5B5bWZ0kYEPly+FbVvPF6lzbwV1UF21yR5H1B9/4WHSp7jMUjAf5RgLmOTV+fw7ZZ~-1~-1~-1; Domain=.fedex.com; Path=/; Expires=Tue, 30 Nov 2021 21:18:24 GMT; Max-Age=31536000; Secure"}], "content": {"size": 17, "mimeType": "application/json", "text": "{\"success\":true}\n", "comment": ""}, "redirectURL": "", "headersSize": 821, "bodySize": 17, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": -1, "dns": -1, "connect": -1, "send": 0, "wait": 123, "ssl": -1}, "serverIPAddress": "23.38.113.34", "comment": "", "time": 123}

But again, because I do not know the actual lat and longitudinal values on my computer whatever it was giving i can't know for certain it is there, as there is no place where it is just straight up lat and long labeled. But, I still think we can confidently say that this is using location. Perhaps, we can find a way in selenium to get the real location that the browser would give up, and then try and match that against HTTP requests. Regardless, I think it is clear I can keep my searches in the "postData" section.

Next I tried homedepot:

Here I see explicit mentions of SOMERVILLE (where I currently am located) as well as my zip.

Here is another request

"pageref": "homedepot.com", "startedDateTime": "2020-11-30T16:35:16.855-05:00", "request": {"method": "POST", "url": "https://www.homedepot.com/thdrecommends/v1/deals?key=aGAQFG4j6QtVTSWqujfFYWeIU6BR5Mee&type=json", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.homedepot.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "application/json, text/javascript, */*; q=0.01"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/json; charset=utf-8"}, {"name": "X-Requested-With", "value": "XMLHttpRequest"}, {"name": "Content-Length", "value": "212"}, {"name": "Origin", "value": "https://www.homedepot.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://www.homedepot.com/"}, {"name": "Cookie", "value": "akaau=1606772412~id=c77e33714d6eb4a8d0b0e1bf470d0b6b; ak_bmsc=9F2A0004B520DDD7EA34F3FD593123BB1724278F6D2200009065C55F5BFE696D~plUcpMvu+heW307DK8pcadf2BRzYx2g5wPWavUe+bUKvG5dBrjVTsNCvYt0kDjv6gI026YcM9RrE2biAfoHUln1m5MohYlKq7rhFOy/kQGVvbSk7yiSFl1osku1EoNuY81OZgIl77jFsj0O1E8ggLL1EPs83mtlq4HtiFd6IajnvPwWYnZ+vY8yab3MjOQzy/pi5Gdw1UNZKca3ob5KuQPFXB90w8ExWU+AMh/5JSJvlqsHTsDpZ4RmGwYpl3pllph6xGbnQWh7i+2HBNBQMXaB1wmMMHdlyzPvpXfjSyhCuI=; HD_DC=origin; bm_sz=992F410E360AD20293AA018C155D9447~YAAQjyckF+sEQhd2AQAA/LsUGwkEXAN3W4Hto5ZZNY8Z98Rq9iPzX8AZEHBPDOislt2aQ6pxRadvntsi43FSM0YFWuxKuSI5GgPGiQkg87DirfaRy4w6Piltv6XP9sMpfMuzDjrl0nC8AsubUBJvdjU20OjonOPlotlvYpOBGy3UGsDUSzR89N+5UgsDVIj5zx3r; _abck=1B93E72672708CF6F1BEDEFF5D70039F~-1~YAAQnCckF+m5YRd2AQAAe8MUGwT2YeZV3OSV5CyKRZI6DjsAUETb8Tg5C+qFdpUpCRO6jelALLA2CItdJDt2T08NlIVuBG9MqzkzLgJe4nXaaWHDiM4WSXsz5fSiCWfeMGceVVnGrTV78IVdGTDytFx0e7/Sz3lHdKstbsetG4EdquFPBNpOM57y/2eyKXP6G8alKZ/ceXwOtKFZmk+/qB5IC+dnebBCQCQxnIaok4cuY4OaY9hdBAPvASP0ONNZVnMACqFExaZE0wY+2Rnt0MuZzME51b6v7OtsjcxXsoRPM5arXdrnDQKpGthtbhF8vTTO3F9w7bEuUDA=~-1~-1~-1; THD_NR=1; AMCV_F6421253512D2C100A490D45%40AdobeOrg=1585540135%7CMCIDTS%7C18597%7CMCMID%7C73574787607895023910473568945350315652%7CMCAAMLH-1607376916%7C7%7CMCAAMB-1607376916%7C6G1ynYcLPuiQxYZrsz_pkqfLG9yMXBpb2zX5dvJdYQJzPXImdj0y%7CMCOPTOUT-1606779316s%7CNONE%7CMCCIDH%7C-1295772520%7CvVersion%7C4.4.0; check=true; mbox=session#22783eb262384c618237ed46c08a55e9#1606773976|PC#22783eb262384c618237ed46c08a55e9.34_0#1670016916; THD_PERSIST=C4%3D2667%2BSomerville%20-%20Somerville%2C%20MA%2B%3A%3BC4_EXP%3D1638308114%3A%3BC24%3D02145%3A%3BC24_EXP%3D1638308114%3A%3BC39%3D1%3B8%3A00-20%3A00%3B2%3B6%3A00-21%3A00%3B3%3B6%3A00-21%3A00%3B4%3B6%3A00-21%3A00%3B5%3B6%3A00-21%3A00%3B6%3B6%3A00-21%3A00%3B7%3B6%3A00-21%3A00%3A%3BC39_EXP%3D1606775714; THD_SESSION=; THD_CACHE_NAV_SESSION=; THD_CACHE_NAV_PERSIST=; IN_STORE_API_SESSION=TRUE; bm_sv=BD5F457EBEF5A1D104F37CDF415694DF~KWb3JRapolwUmNbOsi4IdpsIuQtqKH5273NLMG2p3+RTtsdY1YKoy45qI/QQTaaXjAjkNP8iij9hcubcUcLrnNjxxD9+3M4ow0J/7DNv8ZbShDYty8stvmOX+RyjGR+F1SvX6F6bk1+iIaIFEhTsQ1DpNSW7jDO3BX44y4RFz58=; thda.s=88ef4f03-4bc2-7423-4503-f8cf3f40b965; thda.u=cc449b9e-32c2-b9ae-a940-68087a0836be; AMCVS_F6421253512D2C100A490D45%40AdobeOrg=1; WORKFLOW=GEO_LOCATION; THD_FORCE_LOC=1; THD_INTERNAL=0; THD_LOCALIZER=%7B%22WORKFLOW%22%3A%22GEO_LOCATION%22%2C%22THD_FORCE_LOC%22%3A%221%22%2C%22THD_INTERNAL%22%3A%220%22%2C%22THD_STRFINDERZIP%22%3A%2202145%22%2C%22THD_LOCSTORE%22%3A%222667%2BSomerville%20-%20Somerville%2C%20MA%2B%22%2C%22THD_STORE_HOURS%22%3A%221%3B8%3A00-20%3A00%3B2%3B6%3A00-21%3A00%3B3%3B6%3A00-21%3A00%3B4%3B6%3A00-21%3A00%3B5%3B6%3A00-21%3A00%3B6%3B6%3A00-21%3A00%3B7%3B6%3A00-21%3A00%22%2C%22THD_STORE_HOURS_EXPIRY%22%3A1606775714%7D; DELIVERY_ZIP=02145; DELIVERY_ZIP_TYPE=DEFAULT; _px=cjNM9NbMvKmdQNhtQrz44ueZGAk9t6jqwjnBgqXc9McAOj/6vdyOMXRexqtjYjtRzq5oxJMVQbz8lEgCkvApfA==:1000:tioscPPmO0iE57fJT6yJTx2V3sRt6WLECUKww85GIEbgYzYkVVXMi+QXW3dzIpRXTR39DQaM2K7t/4SWJBVN47Q+SYgKTEYdCPIYbZUM9RmLMn3ggohEhb/ocIsHOMlFzjRj/jefeDFhG70Ev2X4xZjE8FjVMHVU8ABrAYIFoL0srVZ+pgMbM4GiLVQcTzOtXr4YX6AlV2/83gldBCtODlWPIqx+bRUZQD70AKf0QAvnkxtLon7vjOHxi1pM5pIGp5nhF7PsdI9B1gT5kVfdNQ==; _pxvid=eee16ecb-3353-11eb-9f7c-0242ac120016; _pxff_cc=U2FtZVNpdGU9TGF4Ow==; _pxde=c0f181ffdedd7e63a682e9b332e1461a896375600c9cb801fe8991e4346c47d3:eyJ0aW1lc3RhbXAiOjE2MDY3NzIxMTY3NDMsImluY19pZCI6WyI0ZGNmZWY4YTNjZTNiNDc0ZDllNWYxYWNkNzQ3ZTM5OCIsIjU4NDE4YWU3ZTY3NzQ3NmFlNzliYWExNjFlY2M1MjJiIl19; _px_4946459675_cs=eyJpZCI6ImVmNjQ5MmEwLTMzNTMtMTFlYi04ZjY4LWZiMWJlZmYwZDEwNyIsInN0b3JhZ2UiOnt9LCJleHBpcmF0aW9uIjoxNjA2NzczOTE1OTY1fQ==; _px_f394gi7Fvmc43dfg_user_id=ZWY2NTA3ZDAtMzM1My0xMWViLThmNjgtZmIxYmVmZjBkMTA3; s_sess=%20stsh%3D%3B%20s_pv_pName%3Dhomepage%3B%20s_pv_pType%3Dhomepage%3B%20s_pv_cmpgn%3D%3B%20s_pv_pVer%3Dhome%253Aversion%253Agen2%3B%20s_cc%3Dtrue%3B; forterToken=65950c61cd094ca0a1fa50e7ca17925d_1606772115677___9ck; s_pers=%20s_nr365%3D1606772116115-New%7C1638308116115%3B%20s_dslv%3D1606772116116%7C1701380116116%3B; RES_TRACKINGID=67695725361127760; ResonanceSegment=; RES_SESSIONID=58807535361127760; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%2285377d54-402f-441b-b913-8cf3e2b87938%22; _meta_mediaMath_iframe_counter=0; _meta_bing_beaconFired=true; _meta_facebookPixel_beaconFired=true; _meta_merkle_securedVisitBeaconFired=true; _meta_merkle_rkdmsBeaconFired=true; _meta_movableInk_mi_u=85377d54-402f-441b-b913-8cf3e2b87938; _meta_metarouter_timezone_offset=300; _gcl_au=1.1.1724200439.1606772116; thda.m=73574787607895023910473568945350315652"}], "queryString": [{"name": "key", "value": "aGAQFG4j6QtVTSWqujfFYWeIU6BR5Mee"}, {"name": "type", "value": "json"}], "postData": {"mimeType": "application/json; charset=utf-8", "text": "{\"appid\":\"desktop\",\"level\":\"L1\",\"showproducts\":true,\"minproduct\":1,\"maxproduct\":15,\"exitems\":[\"305712398\",\"305573330\"],\"predefinedcategories\":{\"N-5yc1vZ2fkp6kt\":\"Model2\"},\"storeid\":\"2667\",\"defaultbackfill\":false}", "comment": ""}, "headersSize": 5100, "bodySize": 212, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Server", "value": "Apache-Coyote/1.1"}, {"name": "X-TM-ZONE", "value": "us-east4-b"}, {"name": "Access-Control-Allow-Origin", "value": "https://www.homedepot.com"}, {"name": "Vary", "value": "Origin"}, {"name": "Content-Type", "value": "application/json;charset=UTF-8"}, {"name": "Content-Length", "value": "9679"}, {"name": "Expires", "value": "Mon, 30 Nov 2020 21:35:17 GMT"}, {"name": "Cache-Control", "value": "max-age=0, no-cache, no-store"}, {"name": "Pragma", "value": "no-cache"}, {"name": "Date", "value": "Mon, 30 Nov 2020 21:35:17 GMT"}, {"name": "Connection", "value": "keep-alive"}, {"name": "X-Proto", "value": "secure"}], "content": {"size": 9679, "mimeType": "application/json;charset=UTF-8", "text": "{\"categories\":[{\"catName\":\"Holiday Top Sellers\",\"catImage\":\"https://images.homedepot-static.com/productImages/5ff42d5e-cc58-4087-8e72-49ed183d2816/svn/home-accents-holiday-pre-lit-christmas-trees-nrv2-300-75-64_<SIZE>.jpg\",\"catURL\":\"/b/Featured-Products-Holiday-Top-Sellers/N-5yc1vZ2fkp6kt\",\"products\":[{\"productId\":312951587,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"5ff42d5e-cc58-4087-8e72-49ed183d2816\",\"imageURL\":\"https://images.homedepot-static.com/productImages/5ff42d5e-cc58-4087-8e72-49ed183d2816/svn/home-accents-holiday-pre-lit-christmas-trees-nrv2-300-75-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/Home-Accents-Holiday-7-5-ft-North-Valley-Spruce-Artificial-Christmas-Tree-with-550-White-Mini-Lights-NRV2-300-75/312951587\",\"brand\":\"Home Accents Holiday\",\"productName\":\"7.5 ft. North Valley Spruce Artificial Christmas Tree with 550 White Mini Lights\",\"originalPrice\":199.00,\"price\":159.20,\"dollarOff\":39.80,\"percentageOff\":20.00,\"uom\":\"each\",\"rating\":3.2,\"reviews\":5},{\"productId\":311411352,\"itemType\":\"MAJOR_APPLIANCE\",\"imageGuid\":\"ea551d3b-b691-49ac-8d7a-6eb68ee21676\",\"imageURL\":\"https://images.homedepot-static.com/productImages/ea551d3b-b691-49ac-8d7a-6eb68ee21676/svn/fingerprint-resistant-stainless-steel-ge-french-door-refrigerators-pvd28bynfs-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/GE-Profile-27-9-cu-ft-Smart-4-Door-French-Door-Refrigerator-with-Door-in-Door-in-Fingerprint-Resistant-Stainless-Steel-PVD28BYNFS/311411352\",\"brand\":\"GE\",\"productName\":\"Profile 27.9 cu. ft. Smart 4-Door French Door Refrigerator with Door in Door in Fingerprint Resistant Stainless Steel\",\"originalPrice\":3799.00,\"price\":2798.00,\"dollarOff\":1001.00,\"percentageOff\":26.00,\"uom\":\"each\",\"rating\":4.6561,\"reviews\":1451},{\"productId\":305708068,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"8515b9f5-ce58-4d94-8698-3f0f5ee32da0\",\"imageURL\":\"https://images.homedepot-static.com/productImages/8515b9f5-ce58-4d94-8698-3f0f5ee32da0/svn/black-magic-chef-countertop-microwaves-hmm770b-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/Magic-Chef-0-7-cu-ft-Countertop-Microwave-in-Black-with-Gray-Cavity-HMM770B/305708068\",\"brand\":\"Magic Chef\",\"productName\":\"0.7 cu. ft. Countertop Microwave in Black with Gray Cavity\",\"originalPrice\":54.98,\"price\":44.97,\"dollarOff\":10.01,\"percentageOff\":18.00,\"uom\":\"each\",\"rating\":4.2914,\"reviews\":803},{\"productId\":310535782,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"88504495-0b65-4c16-8f22-b0b6993aa11c\",\"imageURL\":\"https://images.homedepot-static.com/productImages/88504495-0b65-4c16-8f22-b0b6993aa11c/svn/black-aria-air-fryers-aafo-880-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/ARIA-Aria-10-Qt-Black-AirFryer-with-Recipe-Book-AAFO-880/310535782\",\"brand\":\"ARIA\",\"productName\":\"Aria 10 Qt. Black AirFryer with Recipe Book\",\"originalPrice\":149.99,\"price\":96.99,\"dollarOff\":53.00,\"percentageOff\":35.00,\"uom\":\"each\",\"rating\":4.3071,\"reviews\":495},{\"productId\":305970984,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"90ad2407-f948-4fb0-8c00-5fb815383d1a\",\"imageURL\":\"https://images.homedepot-static.com/productImages/90ad2407-f948-4fb0-8c00-5fb815383d1a/svn/irobot-robot-vacuums-r675020-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/iRobot-Roomba-675-Wi-Fi-Connected-Robot-Vacuum-Cleaner-R675020/305970984\",\"brand\":\"iRobot\",\"productName\":\"Roomba 675 Wi-Fi Connected Robot Vacuum Cleaner\",\"originalPrice\":274.99,\"price\":179.99,\"dollarOff\":95.00,\"percentageOff\":35.00,\"uom\":\"each\",\"rating\":4.1736,\"reviews\":766},{\"productId\":206391087,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"22e839b4-f9dd-44e2-9209-3555534cc38c\",\"imageURL\":\"https://images.homedepot-static.com/productImages/22e839b4-f9dd-44e2-9209-3555534cc38c/svn/stainless-steel-google-programmable-thermostats-t3007es-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/Google-Nest-Learning-Thermostat-3rd-Gen-in-Stainless-Steel-T3007ES/206391087\",\"brand\":\"Google\",\"productName\":\"Nest Learning Thermostat 3rd Gen in Stainless Steel\",\"originalPrice\":249.00,\"price\":199.00,\"dollarOff\":50.00,\"percentageOff\":20.00,\"uom\":\"each\",\"rating\":4.8116,\"reviews\":11727},{\"productId\":312887279,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"00c90eb0-a66d-46e2-82bf-84109c63a3af\",\"imageURL\":\"https://images.homedepot-static.com/productImages/00c90eb0-a66d-46e2-82bf-84109c63a3af/svn/home-accents-holiday-christmas-inflatables-117578-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/Home-Accents-Holiday-11-ft-Giant-Sized-LED-Inflatable-Penguin-117578/312887279\",\"brand\":\"Home Accents Holiday\",\"productName\":\"11 ft. Giant-Sized LED Inflatable Penguin\",\"originalPrice\":79.98,\"price\":69.98,\"dollarOff\":10.00,\"percentageOff\":13.00,\"uom\":\"each\",\"rating\":4.8,\"reviews\":5},{\"productId\":308067489,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"a426acda-fce2-47c5-83b9-9fd131b3df9d\",\"imageURL\":\"https://images.homedepot-static.com/productImages/a426acda-fce2-47c5-83b9-9fd131b3df9d/svn/dewalt-power-tool-combo-kits-dck278c2-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/DEWALT-ATOMIC-20-Volt-MAX-Cordless-Brushless-Compact-Drill-Impact-Combo-Kit-2-Tool-with-2-1-3Ah-Batteries-Charger-DCK278C2/308067489\",\"brand\":\"DEWALT\",\"productName\":\"ATOMIC 20-Volt MAX Cordless Brushless Compact Drill/Impact Combo Kit (2-Tool) with (2) 1.3Ah Batteries & Charger\",\"originalPrice\":229.00,\"price\":149.00,\"dollarOff\":80.00,\"percentageOff\":35.00,\"uom\":\"each\",\"rating\":4.7141,\"reviews\":1602},{\"productId\":309659455,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"85f7aa68-f72a-4eac-8265-49491f5a5d90\",\"imageURL\":\"https://images.homedepot-static.com/productImages/85f7aa68-f72a-4eac-8265-49491f5a5d90/svn/ryobi-power-tool-combo-kits-p1819-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/RYOBI-18-Volt-ONE-Lithium-Ion-Cordless-6-Tool-Combo-Kit-with-2-Batteries-Charger-and-Bag-P1819/309659455\",\"brand\":\"RYOBI\",\"productName\":\"18-Volt ONE+ Lithium-Ion Cordless 6-Tool Combo Kit with (2) Batteries, Charger, and Bag\",\"originalPrice\":299.00,\"price\":199.00,\"dollarOff\":100.00,\"percentageOff\":33.00,\"uom\":\"each\",\"rating\":4.6848,\"reviews\":4514},{\"productId\":309415135,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"9a19f62c-0e35-48ee-90a9-6a07ce34d41b\",\"imageURL\":\"https://images.homedepot-static.com/productImages/9a19f62c-0e35-48ee-90a9-6a07ce34d41b/svn/ridgid-portable-table-saws-r4514-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/RIDGID-10-in-Pro-Jobsite-Table-Saw-with-Stand-R4514/309415135\",\"brand\":\"RIDGID\",\"productName\":\"10 in. Pro Jobsite Table Saw with Stand\",\"originalPrice\":549.00,\"price\":349.00,\"dollarOff\":200.00,\"percentageOff\":36.00,\"uom\":\"each\",\"rating\":4.3046,\"reviews\":673},{\"productId\":306703572,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"0fab0c9e-bcd6-467d-9b78-dc314844dbea\",\"imageURL\":\"https://images.homedepot-static.com/productImages/0fab0c9e-bcd6-467d-9b78-dc314844dbea/svn/ryobi-power-tool-batteries-p145-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/RYOBI-18-Volt-ONE-4-0-Ah-Lithium-Ion-Battery-2-Pack-P145/306703572\",\"brand\":\"RYOBI\",\"productName\":\"18-Volt ONE+ 4.0 Ah Lithium-Ion Battery (2-Pack)\",\"originalPrice\":178.00,\"price\":79.00,\"dollarOff\":99.00,\"percentageOff\":56.00,\"uom\":\"each\",\"rating\":4.6044,\"reviews\":1135},{\"productId\":310729430,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"8ce8b4fe-3980-4c77-beec-0a9ae07ec255\",\"imageURL\":\"https://images.homedepot-static.com/productImages/8ce8b4fe-3980-4c77-beec-0a9ae07ec255/svn/milwaukee-circular-saws-2732-20-48-11-1880-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/Milwaukee-M18-FUEL-18-Volt-Lithium-Ion-Cordless-7-1-4-in-Circular-Saw-W-HIGH-OUTPUT-XC-8-0Ah-Battery-2732-20-48-11-1880/310729430\",\"brand\":\"Milwaukee\",\"productName\":\"M18 FUEL 18-Volt Lithium-Ion Cordless 7-1/4 in. Circular Saw W/ HIGH OUTPUT XC 8.0Ah Battery\",\"originalPrice\":419.00,\"price\":249.00,\"dollarOff\":170.00,\"percentageOff\":41.00,\"uom\":\"each\",\"rating\":4.7024,\"reviews\":336},{\"productId\":312926888,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"38b7eeb3-9f0d-49d9-aa58-160696298d81\",\"imageURL\":\"https://images.homedepot-static.com/productImages/38b7eeb3-9f0d-49d9-aa58-160696298d81/svn/home-accents-holiday-pre-lit-christmas-trees-tg76p2900d00-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/Home-Accents-Holiday-7-5-ft-Manchester-White-Spruce-LED-Pre-Lit-Artificial-Christmas-Tree-with-500-SureBright-Color-Changing-Lights-TG76P2900D00/312926888\",\"brand\":\"Home Accents Holiday\",\"productName\":\"7.5 ft Manchester White Spruce LED Pre-Lit Artificial Christmas Tree with 500 SureBright Color Changing Lights\",\"originalPrice\":199.00,\"price\":149.00,\"dollarOff\":50.00,\"percentageOff\":25.00,\"uom\":\"each\",\"rating\":3.7931,\"reviews\":29},{\"productId\":308067442,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"c6327a22-e931-482b-9d3e-aeab30caa4be\",\"imageURL\":\"https://images.homedepot-static.com/productImages/c6327a22-e931-482b-9d3e-aeab30caa4be/svn/dewalt-power-drills-dcd708c2-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/DEWALT-ATOMIC-20-Volt-MAX-Cordless-Brushless-Compact-1-2-in-Drill-Driver-with-2-20-Volt-1-3Ah-Batteries-Charger-Bag-DCD708C2/308067442\",\"brand\":\"DEWALT\",\"productName\":\"ATOMIC 20-Volt MAX Cordless Brushless Compact 1/2 in. Drill/Driver with (2) 20-Volt 1.3Ah Batteries, Charger & Bag\",\"originalPrice\":159.00,\"price\":99.00,\"dollarOff\":60.00,\"percentageOff\":38.00,\"uom\":\"each\",\"rating\":4.7721,\"reviews\":2628},{\"productId\":312823074,\"itemType\":\"MERCHANDISE\",\"imageGuid\":\"86c34da0-835b-4a4d-bba1-5b7f5539d83e\",\"imageURL\":\"https://images.homedepot-static.com/productImages/86c34da0-835b-4a4d-bba1-5b7f5539d83e/svn/home-decorators-collection-pre-lit-christmas-trees-r79375plpyglww-64_<SIZE>.jpg\",\"canonicalURL\":\"/p/Home-Decorators-Collection-7-5-ft-Mayfield-Balsam-Fir-LED-Pre-Lit-Artificial-Christmas-Tree-with-4000-Warm-White-Lights-R79375PLPYGLWW/312823074\",\"brand\":\"Home Decorators Collection\",\"productName\":\"7.5 ft. Mayfield Balsam Fir LED Pre-Lit Artificial Christmas Tree with 4000 Warm White Lights\",\"originalPrice\":599.00,\"price\":499.00,\"dollarOff\":100.00,\"percentageOff\":17.00,\"uom\":\"each\",\"rating\":4.5357,\"reviews\":28}],\"nvalue\":\"N-5yc1vZ2fkp6kt\"}]}", "comment": ""}, "redirectURL": "", "headersSize": 391, "bodySize": 9679, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": -1, "dns": -1, "connect": -1, "send": 0, "wait": 248, "ssl": -1}, "serverIPAddress": "104.77.250.51", "comment": "", "time": 248}
{"pageref": "homedepot.com", "startedDateTime": "2020-11-30T16:35:16.908-05:00", "request": {"method": "POST", "url": "https://www.homedepot.com/recommendations/v1?type=json&key=aGAQFG4j6QtVTSWqujfFYWeIU6BR5Mee", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.homedepot.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "application/json, text/javascript, */*; q=0.01"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/json"}, {"name": "X-Requested-With", "value": "XMLHttpRequest"}, {"name": "Content-Length", "value": "176"}, {"name": "Origin", "value": "https://www.homedepot.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://www.homedepot.com/"}, {"name": "Cookie", "value": "akaau=1606772412~id=c77e33714d6eb4a8d0b0e1bf470d0b6b; ak_bmsc=9F2A0004B520DDD7EA34F3FD593123BB1724278F6D2200009065C55F5BFE696D~plUcpMvu+heW307DK8pcadf2BRzYx2g5wPWavUe+bUKvG5dBrjVTsNCvYt0kDjv6gI026YcM9RrE2biAfoHUln1m5MohYlKq7rhFOy/kQGVvbSk7yiSFl1osku1EoNuY81OZgIl77jFsj0O1E8ggLL1EPs83mtlq4HtiFd6IajnvPwWYnZ+vY8yab3MjOQzy/pi5Gdw1UNZKca3ob5KuQPFXB90w8ExWU+AMh/5JSJvlqsHTsDpZ4RmGwYpl3pllph6xGbnQWh7i+2HBNBQMXaB1wmMMHdlyzPvpXfjSyhCuI=; HD_DC=origin; bm_sz=992F410E360AD20293AA018C155D9447~YAAQjyckF+sEQhd2AQAA/LsUGwkEXAN3W4Hto5ZZNY8Z98Rq9iPzX8AZEHBPDOislt2aQ6pxRadvntsi43FSM0YFWuxKuSI5GgPGiQkg87DirfaRy4w6Piltv6XP9sMpfMuzDjrl0nC8AsubUBJvdjU20OjonOPlotlvYpOBGy3UGsDUSzR89N+5UgsDVIj5zx3r; _abck=1B93E72672708CF6F1BEDEFF5D70039F~-1~YAAQnCckF+m5YRd2AQAAe8MUGwT2YeZV3OSV5CyKRZI6DjsAUETb8Tg5C+qFdpUpCRO6jelALLA2CItdJDt2T08NlIVuBG9MqzkzLgJe4nXaaWHDiM4WSXsz5fSiCWfeMGceVVnGrTV78IVdGTDytFx0e7/Sz3lHdKstbsetG4EdquFPBNpOM57y/2eyKXP6G8alKZ/ceXwOtKFZmk+/qB5IC+dnebBCQCQxnIaok4cuY4OaY9hdBAPvASP0ONNZVnMACqFExaZE0wY+2Rnt0MuZzME51b6v7OtsjcxXsoRPM5arXdrnDQKpGthtbhF8vTTO3F9w7bEuUDA=~-1~-1~-1; THD_NR=1; AMCV_F6421253512D2C100A490D45%40AdobeOrg=1585540135%7CMCIDTS%7C18597%7CMCMID%7C73574787607895023910473568945350315652%7CMCAAMLH-1607376914%7C7%7CMCAAMB-1607376914%7C6G1ynYcLPuiQxYZrsz_pkqfLG9yMXBpb2zX5dvJdYQJzPXImdj0y%7CMCOPTOUT-1606779314s%7CNONE%7CvVersion%7C4.4.0; check=true; mbox=session#22783eb262384c618237ed46c08a55e9#1606773976|PC#22783eb262384c618237ed46c08a55e9.34_0#1670016916; THD_PERSIST=C4%3D2667%2BSomerville%20-%20Somerville%2C%20MA%2B%3A%3BC4_EXP%3D1638308114%3A%3BC24%3D02145%3A%3BC24_EXP%3D1638308114%3A%3BC39%3D1%3B8%3A00-20%3A00%3B2%3B6%3A00-21%3A00%3B3%3B6%3A00-21%3A00%3B4%3B6%3A00-21%3A00%3B5%3B6%3A00-21%3A00%3B6%3B6%3A00-21%3A00%3B7%3B6%3A00-21%3A00%3A%3BC39_EXP%3D1606775714; THD_SESSION=; THD_CACHE_NAV_SESSION=; THD_CACHE_NAV_PERSIST=; IN_STORE_API_SESSION=TRUE; bm_sv=BD5F457EBEF5A1D104F37CDF415694DF~KWb3JRapolwUmNbOsi4IdpsIuQtqKH5273NLMG2p3+RTtsdY1YKoy45qI/QQTaaXjAjkNP8iij9hcubcUcLrnNjxxD9+3M4ow0J/7DNv8ZbShDYty8stvmOX+RyjGR+F1SvX6F6bk1+iIaIFEhTsQ1DpNSW7jDO3BX44y4RFz58=; thda.s=88ef4f03-4bc2-7423-4503-f8cf3f40b965; thda.u=cc449b9e-32c2-b9ae-a940-68087a0836be; AMCVS_F6421253512D2C100A490D45%40AdobeOrg=1; WORKFLOW=GEO_LOCATION; THD_FORCE_LOC=1; THD_INTERNAL=0; THD_LOCALIZER=%7B%22WORKFLOW%22%3A%22GEO_LOCATION%22%2C%22THD_FORCE_LOC%22%3A%221%22%2C%22THD_INTERNAL%22%3A%220%22%2C%22THD_STRFINDERZIP%22%3A%2202145%22%2C%22THD_LOCSTORE%22%3A%222667%2BSomerville%20-%20Somerville%2C%20MA%2B%22%2C%22THD_STORE_HOURS%22%3A%221%3B8%3A00-20%3A00%3B2%3B6%3A00-21%3A00%3B3%3B6%3A00-21%3A00%3B4%3B6%3A00-21%3A00%3B5%3B6%3A00-21%3A00%3B6%3B6%3A00-21%3A00%3B7%3B6%3A00-21%3A00%22%2C%22THD_STORE_HOURS_EXPIRY%22%3A1606775714%7D; DELIVERY_ZIP=02145; DELIVERY_ZIP_TYPE=DEFAULT; _px=Edx0ZsXSEHeKOxHIzbdqmbqZR4XHC2vA6LDlCSEunkKmtPsQHI8brnhlkMhMM2iDRwFRTbiit1Rhpl2Iv2kt3w==:1000:8ieBLPzlQfe5nSYwdNoe0PtBjDZGWobDZX4NyhmGYHnThsrEOgiCI8uxJ461Gqjm9X8JP19lumm+1mqG/bIm2jnzqhn0+Szu1K5tLqnqwor7HjIytVeaERLjr7P8T1IyF/qw3ubBz42bsbnTm8nbfUVvCUbu5ddTHiya8jmSAOlDR+1Weqe3/meOb7JaC+FoZnC4vwH/qDQ0lHEXk1tDu2YWV/8TGdC9ShljsWkJkRHUGhy5OGryaBv2+guxOUvNWm8S45iUde4nXpdZpZfo9A==; _pxvid=eee16ecb-3353-11eb-9f7c-0242ac120016; _pxff_cc=U2FtZVNpdGU9TGF4Ow==; _pxde=6a8c107cd8b7b15b6bfe52e3718ca51371136d7d0567b7e7d34f75b0ac6287a6:eyJ0aW1lc3RhbXAiOjE2MDY3NzIxMTU4MzksImluY19pZCI6WyIxOTliNDQxN2Q1MGM0MzU4YzJlYjE3MThkYjY2ZmVlYyIsImJiMDYwODcxMmE4NTEzOTU5YjkwZjIwYzFlNGMwODg5Il19; _px_4946459675_cs=eyJpZCI6ImVmNjQ5MmEwLTMzNTMtMTFlYi04ZjY4LWZiMWJlZmYwZDEwNyIsInN0b3JhZ2UiOnt9LCJleHBpcmF0aW9uIjoxNjA2NzczOTE1OTY1fQ==; _px_f394gi7Fvmc43dfg_user_id=ZWY2NTA3ZDAtMzM1My0xMWViLThmNjgtZmIxYmVmZjBkMTA3; s_sess=%20stsh%3D%3B%20s_pv_pName%3Dhomepage%3B%20s_pv_pType%3Dhomepage%3B%20s_pv_cmpgn%3D%3B%20s_pv_pVer%3Dhome%253Aversion%253Agen2%3B; forterToken=65950c61cd094ca0a1fa50e7ca17925d_1606772115677___9ck; s_pers=%20s_nr365%3D1606772116115-New%7C1638308116115%3B%20s_dslv%3D1606772116116%7C1701380116116%3B; RES_TRACKINGID=67695725361127760; ResonanceSegment=; RES_SESSIONID=58807535361127760"}], "queryString": [{"name": "type", "value": "json"}, {"name": "key", "value": "aGAQFG4j6QtVTSWqujfFYWeIU6BR5Mee"}], "postData": {"mimeType": "application/json", "text": "{\"maxProducts\":16,\"certonaSchema\":\"homepagehorizontal1_rr,con1_home_rr\",\"appId\":\"homedepot01\",\"sessionId\":\"58807535361127760\",\"trackingId\":\"67695725361127760\",\"storeId\":\"2667\"}", "comment": ""}, "headersSize": 4583, "bodySize": 176, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "X-TM-ZONE", "value": "us-east4-b"}, {"name": "Access-Control-Allow-Origin", "value": "https://www.homedepot.com"}, {"name": "Vary", "value": "Origin"}, {"name": "Content-Type", "value": "application/json;charset=UTF-8"}, {"name": "Content-Length", "value": "3732"}, {"name": "Expires", "value": "Mon, 30 Nov 2020 21:35:17 GMT"}, {"name": "Cache-Control", "value": "max-age=0, no-cache, no-store"}, {"name": "Pragma", "value": "no-cache"}, {"name": "Date", "value": "Mon, 30 Nov 2020 21:35:17 GMT"}, {"name": "Connection", "value": "keep-alive"}, {"name": "X-Proto", "value": "secure"}], "content": {"size": 3732, "mimeType": "application/json;charset=UTF-8", "text": "{\"schemas\":[{\"schemaId\":\"homepagehorizontal1_rr\",\"title\":\"You May Also Like\"},{\"schemaId\":\"con1_home_rr\",\"title\":\"Top Selling Categories \",\"contents\":[{\"contentId\":{\"id\":\"f33670f0-11ab-44dd-acfa-89fca67512e3\",\"category\":\"pipe & fittings\",\"categoryLevel\":\"2\",\"imagePath\":\"https://images.homedepot-static.com/productImages/1973e761-4e97-42da-8438-279921640d68/svn/copper-mueller-streamline-copper-pipe-lh04010-64_1000.jpg\",\"title\":\"Pipe & Fittings\",\"linkUrl\":\"/b/Plumbing-Pipe-Fittings/N-5yc1vZbqpf\",\"active\":true}},{\"contentId\":{\"id\":\"8182cf4c-b975-444d-b2e0-ec97becac576\",\"category\":\"garden center\",\"categoryLevel\":\"2\",\"imagePath\":\"https://contentgrid.homedepot-static.com/hdus/en_US/DTCCOMNEW/fetch/Personalized_Homepage/GardenCenter.jpg\",\"title\":\"Garden Center\",\"linkUrl\":\"/b/Outdoors-Garden-Center/N-5yc1vZbx6k\",\"active\":true}},{\"contentId\":{\"id\":\"5b6a71e1-0f67-4f01-99cb-79262c98d06b\",\"category\":\"power tool accessories\",\"categoryLevel\":\"2\",\"imagePath\":\"https://contentgrid.homedepot-static.com/hdus/en_US/DTCCOMNEW/fetch/Personalized_Homepage/saw-blade-drill-bits.jpg\",\"title\":\"Power Tools & Accessories\",\"linkUrl\":\"/b/Tools-Power-Tools/N-5yc1vZc298\",\"active\":true}},{\"contentId\":{\"id\":\"213419e7-7a2c-468a-a4b9-c717c3f7bc4b\",\"category\":\"electrical boxes, conduit & fittings\",\"categoryLevel\":\"2\",\"imagePath\":\"https://contentgrid.homedepot-static.com/hdus/en_US/DTCCOMNEW/fetch/Personalized_Homepage/electrical-boxes-conduit.jpg\",\"title\":\"Electrical Boxes, Conduit & Fittings\",\"linkUrl\":\"/b/Electrical-Electrical-Boxes-Conduit-Fittings/N-5yc1vZbm4m\",\"active\":true}},{\"contentId\":{\"id\":\"36bf60c2-05a8-4266-b4d1-f2ecffcb3286\",\"category\":\"plumbing parts & repair\",\"categoryLevel\":\"2\",\"imagePath\":\"https://contentgrid.homedepot-static.com/hdus/en_US/DTCCOMNEW/fetch/Personalized_Homepage/5-4-Plumbing-Parts-and-Repair.jpg\",\"title\":\"Plumbing Parts & Repair\",\"linkUrl\":\"/b/Plumbing-Plumbing-Parts/N-5yc1vZbqkp\",\"active\":true}},{\"contentId\":{\"id\":\"2f7ade2f-4a7a-4668-9ea9-801881532e18\",\"category\":\"cleaning supplies\",\"categoryLevel\":\"2\",\"imagePath\":\"https://images.homedepot-static.com/productImages/4e4edc54-54f0-43ed-9846-ce9747d82ed2/svn/simple-green-all-purpose-cleaners-2730103613005-64_1000.jpg\",\"title\":\"Cleaning Supplies\",\"linkUrl\":\"/b/Cleaning-Cleaning-Supplies/N-5yc1vZcb33\",\"active\":true}},{\"contentId\":{\"id\":\"ed4c0c15-5bdf-448e-af99-f1651ea1f633\",\"category\":\"cleaning tools & supplies\",\"categoryLevel\":\"2\",\"imagePath\":\"https://contentgrid.homedepot-static.com/hdus/en_US/DTCCOMNEW/fetch/Personalized_Homepage/5-4-Cleaning-Supplies.jpg\",\"title\":\"Cleaning Supplies\",\"linkUrl\":\"/b/Cleaning-Cleaning-Tools/N-5yc1vZcb51\",\"active\":true}},{\"contentId\":{\"id\":\"a136db41-a4ae-4722-b8a5-a76734012b9d\",\"category\":\"door hardware\",\"categoryLevel\":\"2\",\"imagePath\":\"https://images.homedepot-static.com/productImages/aabdddf5-a084-41df-a897-b0810d856635/svn/andersen-deadbolt-locks-hndltrezn-64_1000.jpg\",\"title\":\"Door Hardware\",\"linkUrl\":\"/b/Hardware-Door-Hardware/N-5yc1vZc213\",\"active\":true}},{\"contentId\":{\"id\":\"dd891601-41fd-45ca-8c26-008141cab844\",\"category\":\"light bulbs\",\"categoryLevel\":\"2\",\"imagePath\":\"https://contentgrid.homedepot-static.com/hdus/en_US/DTCCOMNEW/fetch/Personalized_Homepage/VisNav-L2-Lrg-light-bulbs.jpg\",\"title\":\"Light Bulbs\",\"linkUrl\":\"/b/Lighting-Light-Bulbs/N-5yc1vZbmbu\",\"active\":true}},{\"contentId\":{\"id\":\"64a62a21-568b-4c04-a802-9b43d4af8451\",\"category\":\"paint colors\",\"categoryLevel\":\"2\",\"imagePath\":\"https://images.homedepot-static.com/productImages/a5c7e32c-0d21-4f87-9c5b-6a7abc96182a/svn/balsam-behr-marquee-paint-colors-245301-64_1000.jpg\",\"title\":\"Paint Colors\",\"linkUrl\":\"/b/Paint-Paint-Colors/N-5yc1vZcaw8\",\"active\":true}}]}],\"trackingId\":\"67695725361127760\",\"pageId\":\"res20113013586712356929999\"}", "comment": ""}, "redirectURL": "", "headersSize": 364, "bodySize": 3732, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": 0, "dns": 0, "connect": 500, "send": 0, "wait": 226, "ssl": 203}, "serverIPAddress": "104.77.250.51", "comment": "", "time": 727}

Here all the important info is stored in the 11th header in the request. In this very long string we see "Somerville", ""GEO_LOCATION", "DELIVERY_ZIP", "02145" (my zip).

Last we have another reference to geolocation = 1:

{"pageref": "homedepot.com", "startedDateTime": "2020-11-30T16:35:13.906-05:00", "request": {"method": "POST", "url": "https://www.homedepot.com/akam/11/pixel_11ea67a9", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.homedepot.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/x-www-form-urlencoded"}, {"name": "Content-Length", "value": "2822"}, {"name": "Origin", "value": "https://www.homedepot.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://www.homedepot.com/"}, {"name": "Cookie", "value": "akaau=1606772412~id=c77e33714d6eb4a8d0b0e1bf470d0b6b; ak_bmsc=9F2A0004B520DDD7EA34F3FD593123BB1724278F6D2200009065C55F5BFE696D~pltxgF2u/8FPoJxbnBVg5naimWQYAie4F8HG1DXePIkEFdrFMAGYgNVKcCwSAaYt/8WgENzzKUvSeydLC9FCncXE36hmKw+Cqu1bh9KGxyamzHo8Z2mHs9YWBGmXzIpd+YQUDS/+HuB7Iqp3CmuJg3WcXDa7f/b6s1sSipnqLdfpST/fUWuKbuAXdmle9JhXhGkhfUP40u5UcZiRp7Wto0QEO2XGVxscuw1rClW6CfdyiEXF3aj8mB3LpIjeEb9Q4KfNy0mPLNG07tKEMDe818OKnInRWdonTWb9eVcP2Lv804XsA/wf754IKlk0m8b5/RPQO07645q5pL4jkDGtXaZg==; HD_DC=origin; bm_sz=992F410E360AD20293AA018C155D9447~YAAQjyckF+sEQhd2AQAA/LsUGwkEXAN3W4Hto5ZZNY8Z98Rq9iPzX8AZEHBPDOislt2aQ6pxRadvntsi43FSM0YFWuxKuSI5GgPGiQkg87DirfaRy4w6Piltv6XP9sMpfMuzDjrl0nC8AsubUBJvdjU20OjonOPlotlvYpOBGy3UGsDUSzR89N+5UgsDVIj5zx3r; _abck=1B93E72672708CF6F1BEDEFF5D70039F~0~YAAQ3SckF8/NNBp2AQAAvMEUGwTNFvuRFgDV960b3rZgFeyKjb4Oeorw6etbf5qRasLfB4zRakPO7cxHKX/ccBktXDBYrMo7mrtgoi8w7ZY4OfrS1nfChcgxEY5L8rGGUvw54VW83BrPdBKBa3T5HVScXajISqdtZrwi8/6QH57fjWP3KPNqkjwX8tbrn6GsJhJ4ZLWL72TsUF9r652I8qVGB6jb+i26MYidcXwkZBaAz0O33KwlTbbsU1o5CVs63hwI0vuhsodgYwjN9ALS/ml8SvbdWIJgjjVubd7AbSB9shkwfwZKPYmD5vpn3NDCCPUivjeRO1SNdyIRqT5OAdMMBWZOlSVlSw==~-1~-1~-1; THD_NR=1; AMCV_F6421253512D2C100A490D45%40AdobeOrg=1585540135%7CMCIDTS%7C18597%7CvVersion%7C4.4.0; check=true; mbox=session#22783eb262384c618237ed46c08a55e9#1606773974; THD_PERSIST=; THD_SESSION=; THD_CACHE_NAV_SESSION=; THD_CACHE_NAV_PERSIST=; IN_STORE_API_SESSION=TRUE; bm_sv=A205D19E7766C346FA5E43BDF9BD254A~xOnuUYIMyrroynhL8xNOme0/UWVbuyABqsK9xBWBrbr/hUt6Z22Vi9v8BDslkRRiYIwardKJjb7LCvf1r38p1eZ4p7PUzKxjUXrxIyKGMOwe1ARyrcb9FN4F8s6pGfRq9rbAR+BPB6W5IfqqYYBmOJZ/aX8OW1rtgjzlzP8ZfTc="}], "queryString": [], "postData": {"mimeType": "application/x-www-form-urlencoded", "params": [{"name": "ap", "value": "true", "comment": ""}, {"name": "bt", "value": "0", "comment": ""}, {"name": "fonts", "value": "4,14,15,16,21,22,23,43,47,48,49,50,51", "comment": ""}, {"name": "fh", "value": "cdf3f65934ab41fd05bcf4701c9ac7ab665ba37c", "comment": ""}, {"name": "timing", "value": "{\"1\":25,\"2\":222,\"3\":324,\"4\":507,\"5\":607,\"profile\":{\"bp\":0,\"sr\":2,\"dp\":1,\"lt\":0,\"ps\":1,\"cv\":12,\"fp\":0,\"sp\":0,\"br\":0,\"ieps\":0,\"av\":0,\"z1\":7,\"jsv\":0,\"nav\":0,\"nap\":2,\"crc\":0,\"z2\":0,\"z3\":1,\"z4\":1,\"z5\":0,\"fonts\":71},\"main\":1051,\"compute\":25,\"send\":679}", "comment": ""}, {"name": "bp", "value": "", "comment": ""}, {"name": "sr", "value": "{\"inner\":[1280,758],\"outer\":[1280,832],\"screen\":[4,4],\"pageOffset\":[0,0],\"avail\":[1440,900],\"size\":[1440,900],\"client\":[1280,2213],\"colorDepth\":24,\"pixelDepth\":24}", "comment": ""}, {"name": "dp", "value": "{\"XDomainRequest\":0,\"createPopup\":0,\"removeEventListener\":1,\"globalStorage\":0,\"openDatabase\":0,\"indexedDB\":1,\"attachEvent\":0,\"ActiveXObject\":0,\"dispatchEvent\":1,\"addBehavior\":0,\"addEventListener\":1,\"detachEvent\":0,\"fireEvent\":0,\"MutationObserver\":1,\"HTMLMenuItemElement\":1,\"Int8Array\":1,\"postMessage\":1,\"querySelector\":1,\"getElementsByClassName\":1,\"images\":1,\"compatMode\":\"CSS1Compat\",\"documentMode\":0,\"all\":1,\"now\":1,\"contextMenu\":null}", "comment": ""}, {"name": "lt", "value": "1606772113226-5", "comment": ""}, {"name": "ps", "value": "true,true", "comment": ""}, {"name": "cv", "value": "fb1df2e3f14da83d955799ae20c68696a2efb3b4", "comment": ""}, {"name": "fp", "value": "false", "comment": ""}, {"name": "sp", "value": "false", "comment": ""}, {"name": "br", "value": "Firefox", "comment": ""}, {"name": "ieps", "value": "false", "comment": ""}, {"name": "av", "value": "false", "comment": ""}, {"name": "z", "value": "{\"a\":300573794,\"b\":1,\"c\":0}", "comment": ""}, {"name": "zh", "value": "", "comment": ""}, {"name": "jsv", "value": "1.5", "comment": ""}, {"name": "nav", "value": "{\"userAgent\":\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0\",\"appName\":\"Netscape\",\"appCodeName\":\"Mozilla\",\"appVersion\":\"5.0 (Macintosh)\",\"appMinorVersion\":0,\"product\":\"Gecko\",\"productSub\":\"20100101\",\"vendor\":\"\",\"vendorSub\":\"\",\"buildID\":\"20181001000000\",\"platform\":\"MacIntel\",\"oscpu\":\"Intel Mac OS X 10.15\",\"hardwareConcurrency\":2,\"language\":\"en-US\",\"languages\":[\"en-US\",\"en\"],\"systemLanguage\":0,\"userLanguage\":0,\"doNotTrack\":\"unspecified\",\"msDoNotTrack\":0,\"cookieEnabled\":true,\"geolocation\":1,\"vibrate\":1,\"maxTouchPoints\":0,\"webdriver\":true,\"plugins\":[]}", "comment": ""}, {"name": "crc", "value": "{\"window.chrome\":\"-not-existent\"}", "comment": ""}, {"name": "t", "value": "1a38aa102071fded192104ad5f72c63a6d387062", "comment": ""}, {"name": "u", "value": "65b98d4e4bfe5e0f87e533ae4307ea34", "comment": ""}, {"name": "nap", "value": "11133333331333333333", "comment": ""}, {"name": "fc", "value": "true", "comment": ""}], "comment": ""}, "headersSize": 2040, "bodySize": 2822, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Type", "value": "text/html"}, {"name": "Content-Length", "value": "0"}, {"name": "Expires", "value": "Mon, 30 Nov 2020 21:35:13 GMT"}, {"name": "Cache-Control", "value": "max-age=0, no-cache, no-store"}, {"name": "Pragma", "value": "no-cache"}, {"name": "Date", "value": "Mon, 30 Nov 2020 21:35:13 GMT"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Set-Cookie", "value": "ak_bmsc=9F2A0004B520DDD7EA34F3FD593123BB1724278F6D2200009065C55F5BFE696D~plUcpMvu+heW307DK8pcadf2BRzYx2g5wPWavUe+bUKvG5dBrjVTsNCvYt0kDjv6gI026YcM9RrE2biAfoHUln1m5MohYlKq7rhFOy/kQGVvbSk7yiSFl1osku1EoNuY81OZgIl77jFsj0O1E8ggLL1EPs83mtlq4HtiFd6IajnvPwWYnZ+vY8yab3MjOQzy/pi5Gdw1UNZKca3ob5KuQPFXB90w8ExWU+AMh/5JSJvlqsHTsDpZ4RmGwYpl3pllph6xGbnQWh7i+2HBNBQMXaB1wmMMHdlyzPvpXfjSyhCuI=; expires=Mon, 30 Nov 2020 23:35:12 GMT; max-age=7199; path=/; domain=.homedepot.com; HttpOnly"}, {"name": "X-Proto", "value": "secure"}], "content": {"size": 0, "mimeType": "text/html", "text": "", "comment": ""}, "redirectURL": "", "headersSize": 730, "bodySize": 0, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": -1, "dns": -1, "connect": -1, "send": 0, "wait": 22, "ssl": -1}, "serverIPAddress": "104.77.250.51", "comment": "", "time": 22}

This time this geolocation tag was in postData --> params --> 19. But we are now 2/2 with websites using location having SOMEWHERE in one of the requests saying geolocation = 1

Last for the locations I tried bikemaps.org.

{"pageref": "bikemaps.org", "startedDateTime": "2020-11-30T16:50:21.579-05:00", "request": {"method": "POST", "url": "https://www.googleapis.com/geolocation/v1/geolocate?key=AIzaSyB2h2OuRcUgy5N-5hsZqiPW6sH3n_rptiQ", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.googleapis.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/json; charset=UTF-8"}, {"name": "Origin", "value": "null"}, {"name": "Content-Length", "value": "582"}, {"name": "Connection", "value": "keep-alive"}], "queryString": [{"name": "key", "value": "AIzaSyB2h2OuRcUgy5N-5hsZqiPW6sH3n_rptiQ"}], "postData": {"mimeType": "application/json; charset=UTF-8", "text": "{\"wifiAccessPoints\":[{\"macAddress\":\"d8-07-b6-af-76-0b\",\"signalStrength\":-44},{\"macAddress\":\"20-f3-75-cf-61-c2\",\"signalStrength\":-46},{\"macAddress\":\"20-f3-75-cf-61-c3\",\"signalStrength\":-47},{\"macAddress\":\"f8-bc-0e-79-59-06\",\"signalStrength\":-47},{\"macAddress\":\"d8-07-b6-af-76-0a\",\"signalStrength\":-59},{\"macAddress\":\"b0-95-75-fb-00-da\",\"signalStrength\":-65},{\"macAddress\":\"f8-bc-0e-79-59-07\",\"signalStrength\":-67},{\"macAddress\":\"b0-95-75-fb-00-db\",\"signalStrength\":-72},{\"macAddress\":\"58-d9-d5-3a-50-61\",\"signalStrength\":-78},{\"macAddress\":\"b0-be-76-08-72-b2\",\"signalStrength\":-87}]}", "comment": ""}, "headersSize": 399, "bodySize": 582, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Type", "value": "application/json; charset=UTF-8"}, {"name": "Vary", "value": "Origin"}, {"name": "Vary", "value": "X-Origin"}, {"name": "Vary", "value": "Referer"}, {"name": "Content-Encoding", "value": "gzip"}, {"name": "Date", "value": "Mon, 30 Nov 2020 21:50:21 GMT"}, {"name": "Server", "value": "scaffolding on HTTPServer2"}, {"name": "Cache-Control", "value": "private"}, {"name": "X-XSS-Protection", "value": "0"}, {"name": "X-Frame-Options", "value": "SAMEORIGIN"}, {"name": "X-Content-Type-Options", "value": "nosniff"}, {"name": "Access-Control-Allow-Origin", "value": "null"}, {"name": "Access-Control-Expose-Headers", "value": "vary,vary,vary,content-encoding,date,server,content-length"}, {"name": "Alt-Svc", "value": "h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\""}, {"name": "Transfer-Encoding", "value": "chunked"}], "content": {"size": 88, "mimeType": "application/json; charset=UTF-8", "text": "{\n  \"location\": {\n    \"lat\": 42.3769931,\n    \"lng\": -71.1024005\n  },\n  \"accuracy\": 34\n}\n", "comment": ""}, "redirectURL": "", "headersSize": 649, "bodySize": 92, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": 0, "dns": 26, "connect": 188, "send": 0, "wait": 53, "ssl": 166}, "serverIPAddress": "172.217.3.106", "comment": "", "time": 268}

In this request, in the RESPONSE, we have actual explicit labels of location and then 'lat' and 'lng'. This is under RESPONSE--> content--> text, which is a compltely different area from what I saw in previous requests, but still in there nonetheless. THis has my longitude as -71.1024005 and my latitude as 42.3769931.

Next I looked at cttransit.com

This also had explicit mention of my latitude and longitude:

{"pageref": "cttransit.com", "startedDateTime": "2020-11-30T16:56:48.792-05:00", "request": {"method": "POST", "url": "https://www.googleapis.com/geolocation/v1/geolocate?key=AIzaSyB2h2OuRcUgy5N-5hsZqiPW6sH3n_rptiQ", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.googleapis.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/json; charset=UTF-8"}, {"name": "Origin", "value": "null"}, {"name": "Content-Length", "value": "470"}, {"name": "Connection", "value": "keep-alive"}], "queryString": [{"name": "key", "value": "AIzaSyB2h2OuRcUgy5N-5hsZqiPW6sH3n_rptiQ"}], "postData": {"mimeType": "application/json; charset=UTF-8", "text": "{\"wifiAccessPoints\":[{\"macAddress\":\"f8-bc-0e-79-59-06\",\"signalStrength\":-36},{\"macAddress\":\"20-f3-75-cf-61-c3\",\"signalStrength\":-47},{\"macAddress\":\"20-f3-75-cf-61-c2\",\"signalStrength\":-50},{\"macAddress\":\"d8-07-b6-af-76-0b\",\"signalStrength\":-50},{\"macAddress\":\"f8-bc-0e-79-59-08\",\"signalStrength\":-55},{\"macAddress\":\"d8-07-b6-af-76-0a\",\"signalStrength\":-62},{\"macAddress\":\"f8-bc-0e-79-59-07\",\"signalStrength\":-66},{\"macAddress\":\"b0-95-75-fb-00-db\",\"signalStrength\":-73}]}", "comment": ""}, "headersSize": 399, "bodySize": 470, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Type", "value": "application/json; charset=UTF-8"}, {"name": "Vary", "value": "Origin"}, {"name": "Vary", "value": "X-Origin"}, {"name": "Vary", "value": "Referer"}, {"name": "Content-Encoding", "value": "gzip"}, {"name": "Date", "value": "Mon, 30 Nov 2020 21:56:49 GMT"}, {"name": "Server", "value": "scaffolding on HTTPServer2"}, {"name": "Cache-Control", "value": "private"}, {"name": "X-XSS-Protection", "value": "0"}, {"name": "X-Frame-Options", "value": "SAMEORIGIN"}, {"name": "X-Content-Type-Options", "value": "nosniff"}, {"name": "Access-Control-Allow-Origin", "value": "null"}, {"name": "Access-Control-Expose-Headers", "value": "vary,vary,vary,content-encoding,date,server,content-length"}, {"name": "Alt-Svc", "value": "h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\""}, {"name": "Transfer-Encoding", "value": "chunked"}], "content": {"size": 87, "mimeType": "application/json; charset=UTF-8", "text": "{\n  \"location\": {\n    \"lat\": 42.3769771,\n    \"lng\": -71.102352\n  },\n  \"accuracy\": 20\n}\n", "comment": ""}, "redirectURL": "", "headersSize": 649, "bodySize": 91, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": 0, "dns": 24, "connect": 238, "send": 0, "wait": 414, "ssl": 210}, "serverIPAddress": "172.217.13.74", "comment": "", "time": 678}

Under response --> content --> text we get "location\": {\n \"lat\": 42.3769771,\n \"lng\": -71.102352\n }

The other interesting thing is there is a reference to the Google geolocation API, which could be another key thing we look out for: https://www.googleapis.com/geolocation. It is found in the full URL of the post request.

In BING I found the "latitude" and "longitude" and "altitude" keywords:

{"pageref": "bing.com", "startedDateTime": "2020-11-30T17:00:33.591-05:00", "request": {"method": "POST", "url": "https://www.bing.com/fd/ls/lsp.aspx", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.bing.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Referer", "value": "https://www.bing.com/search?q=businesses+near+me&FORM=R5FD"}, {"name": "Content-Type", "value": "text/plain;charset=UTF-8"}, {"name": "Content-Length", "value": "1403"}, {"name": "Origin", "value": "https://www.bing.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Cookie", "value": "_EDGE_S=mkt=en-us&F=1&SID=0A652B9FF6CF61E80242240DF73260FE; _EDGE_V=1; MUID=0E4E0EE244226507068C017045DF648F; SRCHD=AF=NOFORM; SRCHUID=V=2&GUID=B40EE92824BA453D9312001CBC77B279&dmnchg=1; SRCHUSR=DOB=20201130&T=1606773614000; _SS=SID=0A652B9FF6CF61E80242240DF73260FE&R=10&RB=0&GB=0&RG=200&RP=5; _HPVN=CS=eyJQbiI6eyJDbiI6MSwiU3QiOjAsIlFzIjowLCJQcm9kIjoiUCJ9LCJTYyI6eyJDbiI6MSwiU3QiOjAsIlFzIjowLCJQcm9kIjoiSCJ9LCJReiI6eyJDbiI6MSwiU3QiOjAsIlFzIjowLCJQcm9kIjoiVCJ9LCJBcCI6dHJ1ZSwiTXV0ZSI6dHJ1ZSwiTGFkIjoiMjAyMC0xMS0zMFQwMDowMDowMFoiLCJJb3RkIjowLCJEZnQiOm51bGwsIk12cyI6MCwiRmx0IjowLCJJbXAiOjJ9; MUIDB=0E4E0EE244226507068C017045DF648F; SRCHHPGUSR=CW=1280&CH=758&DPR=2&UTC=-300&DM=1&WTS=63742370414&HV=1606773630&BRW=M&BRH=M; ipv6=hit=1606777215805&t=4; _RwBf=mtu=0&g=0&cid=&o=2&p=&c=&t=0&s=0001-01-01T00:00:00.0000000+00:00&ts=2020-11-30T22:00:29.6509106+00:00&ssg=0; ABDEF=V=12&ABDV=12&MRNB=1606773629497&MRB=0; dsc=order=Maps"}], "queryString": [], "postData": {"mimeType": "text/plain;charset=UTF-8", "text": "<ClientInstRequest><Events><E><T>Event.ClientInst</T><IG>65C18DDB89C94673B3E1D765A731B826</IG><TS>1606773630587</TS><D><![CDATA[{\"feature\":\"MC\",\"action\":\"A\",\"data\":{\"CP\":{\"latitude\":42.38482666015625,\"longitude\":-71.09466934204102,\"altitude\":327839.4722814907,\"altitudeReference\":0},\"H\":0,\"A\":327839.4722814907,\"MM\":\"Mercator\",\"T\":1606773630587},\"T\":\"CI.MapsAction\",\"FID\":\"CI\"}]]></D></E><E><T>Event.ClientInst</T><IG>B306CB9915DB4DBABF0D715B7662F9E7</IG><TS>1606773630589</TS><D><![CDATA[{\"feature\":\"v8MapControl\",\"action\":\"Perf\",\"overlay\":\"maps\",\"data\":{\"resourcesLoadTime\":149,\"renderTime\":730,\"serpPLT\":776,\"segment\":\"Local\"},\"T\":\"CI.ClientClick\",\"FID\":\"CI\"}]]></D></E><E><T>Event.ClientInst</T><IG>B306CB9915DB4DBABF0D715B7662F9E7</IG><TS>1606773630592</TS><D><![CDATA[{\"feature\":\"v8MapControl\",\"action\":\"Perf\",\"overlay\":\"maps\",\"data\":{\"renderDataTime\":3,\"segment\":\"Local\"},\"T\":\"CI.ClientClick\",\"FID\":\"CI\"}]]></D></E><E><T>Event.ClientInst</T><IG>65C18DDB89C94673B3E1D765A731B826</IG><TS>1606773630592</TS><D><![CDATA[{\"feature\":\"MapControl\",\"action\":\"MapsLoaded\",\"data\":{\"scenario\":\"HybridMultiAnswer\",\"service\":\"Local\"},\"T\":\"CI.ClientClick\",\"FID\":\"CI\"}]]></D></E><E><T>Event.ClientInst</T><IG>B306CB9915DB4DBABF0D715B7662F9E7</IG><TS>1606773632664</TS><D><![CDATA[{\"feature\":\"TM\",\"action\":\"A\",\"data\":{\"isSlideEnabled\":true},\"T\":\"CI.ClientClick\",\"FID\":\"CI\"}]]></D></E></Events></ClientInstRequest>", "comment": ""}, "headersSize": 1348, "bodySize": 1403, "comment": ""}, "response": {"status": 204, "statusText": "No Content", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Access-Control-Allow-Origin", "value": "*"}, {"name": "X-MSEdge-Ref", "value": "Ref A: AC283208BD9148C79CF951EBDD260359 Ref B: BLUEDGE0720 Ref C: 2020-11-30T22:00:33Z"}, {"name": "Date", "value": "Mon, 30 Nov 2020 22:00:33 GMT"}], "content": {"size": 0, "mimeType": "", "comment": ""}, "redirectURL": "", "headersSize": 200, "bodySize": 0, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": -1, "dns": -1, "connect": -1, "send": 0, "wait": 40, "ssl": -1}, "serverIPAddress": "131.253.33.200", "comment": "", "time": 41}

One thing to note here is the lat / long data is a few decimal points different from the previous requests, so there is not a clear across the board exact number between apps.

So in all I have found the following keywords: longitude lng latitude lat altitude location geolocation geo_location delivery_zip googleapis.com/geolocation

This is a good place to start on geolocation, I think. Perhaps what we can do for the exact geolocation of a device.

I also tried xfinity for the motion sensor you mentioned.

I didn't see any mention of motion sensors. but it did have my location. It had geolocation = 1 here:

{"pageref": "xfinity.com", "startedDateTime": "2020-11-30T17:16:51.209-05:00", "request": {"method": "POST", "url": "https://www.xfinity.com/akam/11/pixel_ee71c0a", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.xfinity.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/x-www-form-urlencoded"}, {"name": "ADRUM", "value": "isAjax:true"}, {"name": "Content-Length", "value": "2852"}, {"name": "Origin", "value": "https://www.xfinity.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://www.xfinity.com/"}, {"name": "Cookie", "value": "PSC=UCID=944923c2-f22e-4b74-a697-7678d8133742&CTY=Somerville&ST=MA&Z=02143&EX=False&REC=N&RC.MKT=5079; SC=RC.USID=749618ca-0971-4c67-8684-04f204fe36ce&VA=1025&CTY=Somerville&ST=MA&Z=02143&ISP=rcn&GEO=True&RC.MKT=5079&L1ID=11599&L2ID=5051&L3ID=6132&L4ID=3846; ASP.NET_SessionId=fiekbgiidydkrgqepipfdtii; SC_ANALYTICS_GLOBAL_COOKIE=a342c076b9dc4527845cd748ab3acd6c|False; Affiliate=DOT_COM; Channel=WEB; www-prd_wc=KOBLFHEE; AKA_A2=A; ak_bmsc=AE7E2A410E245AB5A6C7FC39A9F10C0D1724279C322300004D6FC55FDFE07821~plIMMDvi4Ivk3kePky9GNFJlpcfiZ/9L2Z34PzgNU2nLRD1XcqG/ntCHE80K5vnfpHCq64KHi90qmxhpSj8fwqdDdsWd+oQi3Uzq1qFejMxH6Rqi3qkBnLIPTfZO76Vpt1iQhsFP5hYOxl6CIxuUDDhsikb8ld7RHwI2En3GmJDodu5RrMteeBJEoBLiVB/RdFvu0rFrDXfKW4Tdt6zEUOLD7Q+v7HjB8xf8tFKXxcZMxMZURzzXo2T+C0W0JXflNSxYTYv6WstgZ9ctrBfCKqTf3JQrnGUeKhfxB3a8V1YJpf4alCD63h/7Qf6LfEtqgnXukYgC/CoeIkqMXAyCruow==; bm_sz=D47B9595E232C455ECA9D9DA3D19605A~YAAQnCckFwSaYhd2AQAAj8U6GwmJUhqa1aT1m/H7PZLyTap+UKDtaBFXWPwoKmUs1/KRA6ShNSUjmvlhpPtD1snntHOolMNxmh/kYTQU/OLGSxsx423vhzvvTuEQ/+/E9slnxReZZm7cY9JdwGGisc5zptGnViGnvpaYxkKctFStskckVvlSkxN06g32yBGj; _abck=57FB4A1F1A29686A7A492455BB16A867~0~YAAQnCckF36aYhd2AQAAEtw6GwQWi4JMAY8IfFGmrV2gGzL5M91ztlMvxzQ6QYcs+M21Cq2hR5RsJTLNNaxgK0bpFmHCihJkfI04nGCay/PNdAGlW6lI9G6aIpRBAGGYiDWkxQymzpjrVx9sAujmv9FnEUtXRycJKSbgKruoX+r98nCQqdu/8CHKmeKqIBhEzCuxVSdh4GXMf7PSxxyXjF3canGFQ1wEFo/L2vAphyzK0PfO7RhEkUPMHHfzGrfIa5G6dNjWMg1xVaIO1KJPWrIctth2G+BadFfyyCTXFEKRkTnQikg9spEz1AOm4OH3nHqArpwbaEPouDbLZ5hAWyt0cQmHanU=~-1~||-1||~-1; bm_sv=555DABB412A80A9D8441AECDBABFEBD0~+fXHMZIQCx1ir5uFMemZm6VKrkjuxqVoipRn0+Vfheye8dn68bBmkqEGHK9lxrnUJs2m7uroXkbc+uY+wO7tAYA0W8XaO3JNRefXVDN0o0K+COKowIiRKDi1p+AhaaADQh+YVTTgWQ2R2H39vdHNyl5/r/RGKXmZWKzKpa05RaM=; AMCV_DA11332E5321D0550A490D45%40AdobeOrg=359503849%7CMCIDTS%7C18597%7CMCMID%7C19672755694540869832061993999456874992%7CMCAAMLH-1607379410%7C7%7CMCAAMB-1607379410%7C6G1ynYcLPuiQxYZrsz_pkqfLG9yMXBpb2zX5dvJdYQJzPXImdj0y%7CMCOPTOUT-1606781810s%7CNONE%7CvVersion%7C5.0.1; mbox=session#787721f18890460fa2f4b3e0884baa57#1606776471; s_pers=%20s_dfa%3D%7C1606776410628%3B; at_check=true; AMCVS_DA11332E5321D0550A490D45%40AdobeOrg=1"}], "queryString": [], "postData": {"mimeType": "application/x-www-form-urlencoded", "params": [{"name": "ap", "value": "true", "comment": ""}, {"name": "bt", "value": "0", "comment": ""}, {"name": "fonts", "value": "4,14,15,16,21,22,23,43,47,48,49,50,51", "comment": ""}, {"name": "fh", "value": "cdf3f65934ab41fd05bcf4701c9ac7ab665ba37c", "comment": ""}, {"name": "timing", "value": "{\"1\":50,\"2\":152,\"3\":253,\"4\":363,\"5\":464,\"6\":569,\"profile\":{\"bp\":0,\"sr\":1,\"dp\":4,\"lt\":0,\"ps\":1,\"cv\":29,\"fp\":0,\"sp\":0,\"br\":0,\"ieps\":1,\"av\":0,\"z1\":7,\"jsv\":1,\"nav\":0,\"nap\":4,\"crc\":0,\"z2\":1,\"z3\":1,\"z4\":1,\"z5\":1,\"z6\":0,\"fonts\":62},\"main\":6284,\"compute\":50,\"send\":631}", "comment": ""}, {"name": "bp", "value": "", "comment": ""}, {"name": "sr", "value": "{\"inner\":[1280,758],\"outer\":[1280,832],\"screen\":[4,4],\"pageOffset\":[0,0],\"avail\":[1440,900],\"size\":[1440,900],\"client\":[1280,758],\"colorDepth\":24,\"pixelDepth\":24}", "comment": ""}, {"name": "dp", "value": "{\"XDomainRequest\":0,\"createPopup\":0,\"removeEventListener\":1,\"globalStorage\":0,\"openDatabase\":0,\"indexedDB\":1,\"attachEvent\":0,\"ActiveXObject\":0,\"dispatchEvent\":1,\"addBehavior\":0,\"addEventListener\":1,\"detachEvent\":0,\"fireEvent\":0,\"MutationObserver\":1,\"HTMLMenuItemElement\":1,\"Int8Array\":1,\"postMessage\":1,\"querySelector\":1,\"getElementsByClassName\":1,\"images\":1,\"compatMode\":\"CSS1Compat\",\"documentMode\":0,\"all\":1,\"now\":1,\"contextMenu\":null}", "comment": ""}, {"name": "lt", "value": "1606774610576-5", "comment": ""}, {"name": "ps", "value": "true,true", "comment": ""}, {"name": "cv", "value": "fb1df2e3f14da83d955799ae20c68696a2efb3b4", "comment": ""}, {"name": "fp", "value": "false", "comment": ""}, {"name": "sp", "value": "false", "comment": ""}, {"name": "br", "value": "Firefox", "comment": ""}, {"name": "ieps", "value": "false", "comment": ""}, {"name": "av", "value": "false", "comment": ""}, {"name": "z", "value": "{\"a\":250027159,\"b\":1,\"c\":0}", "comment": ""}, {"name": "zh", "value": "", "comment": ""}, {"name": "jsv", "value": "1.5", "comment": ""}, {"name": "nav", "value": "{\"userAgent\":\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0\",\"appName\":\"Netscape\",\"appCodeName\":\"Mozilla\",\"appVersion\":\"5.0 (Macintosh)\",\"appMinorVersion\":0,\"product\":\"Gecko\",\"productSub\":\"20100101\",\"vendor\":\"\",\"vendorSub\":\"\",\"buildID\":\"20181001000000\",\"platform\":\"MacIntel\",\"oscpu\":\"Intel Mac OS X 10.15\",\"hardwareConcurrency\":2,\"language\":\"en-US\",\"languages\":[\"en-US\",\"en\"],\"systemLanguage\":0,\"userLanguage\":0,\"doNotTrack\":\"unspecified\",\"msDoNotTrack\":0,\"cookieEnabled\":true,\"geolocation\":1,\"vibrate\":1,\"maxTouchPoints\":0,\"webdriver\":true,\"plugins\":[]}", "comment": ""}, {"name": "crc", "value": "{\"window.chrome\":\"-not-existent\"}", "comment": ""}, {"name": "t", "value": "f8dcad4cd33b8492671e8ee611c5a497448ecb10", "comment": ""}, {"name": "u", "value": "df071d33ceea54f192dfeff1afbaf243", "comment": ""}, {"name": "nap", "value": "11133333331333333333", "comment": ""}, {"name": "fc", "value": "true", "comment": ""}], "comment": ""}, "headersSize": 2566, "bodySize": 2852, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Type", "value": "text/html"}, {"name": "Content-Length", "value": "0"}, {"name": "Expires", "value": "Mon, 30 Nov 2020 22:16:51 GMT"}, {"name": "Cache-Control", "value": "max-age=0, no-cache, no-store"}, {"name": "Pragma", "value": "no-cache"}, {"name": "Date", "value": "Mon, 30 Nov 2020 22:16:51 GMT"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Set-Cookie", "value": "ak_bmsc=AE7E2A410E245AB5A6C7FC39A9F10C0D1724279C322300004D6FC55FDFE07821~pl63MrlwyVQ6OwLZ3q+6HMBq2Chf1jjcLwwUHmHNv13/aQIwEzskvhqgZBOv62XYEhtg5TvrrvEv+2KA2t+fa6QNKP6ulVzU5b28vMwh/LwYXAYuZTeDgQdtUC9PZopov+IiYTp7rW7sN9kpd0JItIt61ppaDvAKeFhzk6RI42vRzgjJonYEmWEuhU4MEIkbJ/ypeDa3Sb8mpt+WmTXsIemK6VDzRkt/Q2zzgVz9WHXqpUsMpCQ3yCj5kU+TxHmE6COFiyolfM2JbCA0kAxAomyClmmSUoYxaQedhebN9W24w=; expires=Tue, 01 Dec 2020 00:16:45 GMT; max-age=7194; path=/; domain=.xfinity.com; HttpOnly; Secure"}], "content": {"size": 0, "mimeType": "text/html", "text": "", "comment": ""}, "redirectURL": "", "headersSize": 719, "bodySize": 0, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": -1, "dns": -1, "connect": -1, "send": 0, "wait": 24, "ssl": -1}, "serverIPAddress": "104.77.245.23", "comment": "", "time": 24}

It also had mention of Somerville, and my rough zip code:

{"pageref": "xfinity.com", "startedDateTime": "2020-11-30T17:16:51.209-05:00", "request": {"method": "POST", "url": "https://www.xfinity.com/akam/11/pixel_ee71c0a", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "www.xfinity.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/x-www-form-urlencoded"}, {"name": "ADRUM", "value": "isAjax:true"}, {"name": "Content-Length", "value": "2852"}, {"name": "Origin", "value": "https://www.xfinity.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://www.xfinity.com/"}, {"name": "Cookie", "value": "PSC=UCID=944923c2-f22e-4b74-a697-7678d8133742&CTY=Somerville&ST=MA&Z=02143&EX=False&REC=N&RC.MKT=5079; SC=RC.USID=749618ca-0971-4c67-8684-04f204fe36ce&VA=1025&CTY=Somerville&ST=MA&Z=02143&ISP=rcn&GEO=True&RC.MKT=5079&L1ID=11599&L2ID=5051&L3ID=6132&L4ID=3846; ASP.NET_SessionId=fiekbgiidydkrgqepipfdtii; SC_ANALYTICS_GLOBAL_COOKIE=a342c076b9dc4527845cd748ab3acd6c|False; Affiliate=DOT_COM; Channel=WEB; www-prd_wc=KOBLFHEE; AKA_A2=A; ak_bmsc=AE7E2A410E245AB5A6C7FC39A9F10C0D1724279C322300004D6FC55FDFE07821~plIMMDvi4Ivk3kePky9GNFJlpcfiZ/9L2Z34PzgNU2nLRD1XcqG/ntCHE80K5vnfpHCq64KHi90qmxhpSj8fwqdDdsWd+oQi3Uzq1qFejMxH6Rqi3qkBnLIPTfZO76Vpt1iQhsFP5hYOxl6CIxuUDDhsikb8ld7RHwI2En3GmJDodu5RrMteeBJEoBLiVB/RdFvu0rFrDXfKW4Tdt6zEUOLD7Q+v7HjB8xf8tFKXxcZMxMZURzzXo2T+C0W0JXflNSxYTYv6WstgZ9ctrBfCKqTf3JQrnGUeKhfxB3a8V1YJpf4alCD63h/7Qf6LfEtqgnXukYgC/CoeIkqMXAyCruow==; bm_sz=D47B9595E232C455ECA9D9DA3D19605A~YAAQnCckFwSaYhd2AQAAj8U6GwmJUhqa1aT1m/H7PZLyTap+UKDtaBFXWPwoKmUs1/KRA6ShNSUjmvlhpPtD1snntHOolMNxmh/kYTQU/OLGSxsx423vhzvvTuEQ/+/E9slnxReZZm7cY9JdwGGisc5zptGnViGnvpaYxkKctFStskckVvlSkxN06g32yBGj; _abck=57FB4A1F1A29686A7A492455BB16A867~0~YAAQnCckF36aYhd2AQAAEtw6GwQWi4JMAY8IfFGmrV2gGzL5M91ztlMvxzQ6QYcs+M21Cq2hR5RsJTLNNaxgK0bpFmHCihJkfI04nGCay/PNdAGlW6lI9G6aIpRBAGGYiDWkxQymzpjrVx9sAujmv9FnEUtXRycJKSbgKruoX+r98nCQqdu/8CHKmeKqIBhEzCuxVSdh4GXMf7PSxxyXjF3canGFQ1wEFo/L2vAphyzK0PfO7RhEkUPMHHfzGrfIa5G6dNjWMg1xVaIO1KJPWrIctth2G+BadFfyyCTXFEKRkTnQikg9spEz1AOm4OH3nHqArpwbaEPouDbLZ5hAWyt0cQmHanU=~-1~||-1||~-1; bm_sv=555DABB412A80A9D8441AECDBABFEBD0~+fXHMZIQCx1ir5uFMemZm6VKrkjuxqVoipRn0+Vfheye8dn68bBmkqEGHK9lxrnUJs2m7uroXkbc+uY+wO7tAYA0W8XaO3JNRefXVDN0o0K+COKowIiRKDi1p+AhaaADQh+YVTTgWQ2R2H39vdHNyl5/r/RGKXmZWKzKpa05RaM=; AMCV_DA11332E5321D0550A490D45%40AdobeOrg=359503849%7CMCIDTS%7C18597%7CMCMID%7C19672755694540869832061993999456874992%7CMCAAMLH-1607379410%7C7%7CMCAAMB-1607379410%7C6G1ynYcLPuiQxYZrsz_pkqfLG9yMXBpb2zX5dvJdYQJzPXImdj0y%7CMCOPTOUT-1606781810s%7CNONE%7CvVersion%7C5.0.1; mbox=session#787721f18890460fa2f4b3e0884baa57#1606776471; s_pers=%20s_dfa%3D%7C1606776410628%3B; at_check=true; AMCVS_DA11332E5321D0550A490D45%40AdobeOrg=1"}], "queryString": [], "postData": {"mimeType": "application/x-www-form-urlencoded", "params": [{"name": "ap", "value": "true", "comment": ""}, {"name": "bt", "value": "0", "comment": ""}, {"name": "fonts", "value": "4,14,15,16,21,22,23,43,47,48,49,50,51", "comment": ""}, {"name": "fh", "value": "cdf3f65934ab41fd05bcf4701c9ac7ab665ba37c", "comment": ""}, {"name": "timing", "value": "{\"1\":50,\"2\":152,\"3\":253,\"4\":363,\"5\":464,\"6\":569,\"profile\":{\"bp\":0,\"sr\":1,\"dp\":4,\"lt\":0,\"ps\":1,\"cv\":29,\"fp\":0,\"sp\":0,\"br\":0,\"ieps\":1,\"av\":0,\"z1\":7,\"jsv\":1,\"nav\":0,\"nap\":4,\"crc\":0,\"z2\":1,\"z3\":1,\"z4\":1,\"z5\":1,\"z6\":0,\"fonts\":62},\"main\":6284,\"compute\":50,\"send\":631}", "comment": ""}, {"name": "bp", "value": "", "comment": ""}, {"name": "sr", "value": "{\"inner\":[1280,758],\"outer\":[1280,832],\"screen\":[4,4],\"pageOffset\":[0,0],\"avail\":[1440,900],\"size\":[1440,900],\"client\":[1280,758],\"colorDepth\":24,\"pixelDepth\":24}", "comment": ""}, {"name": "dp", "value": "{\"XDomainRequest\":0,\"createPopup\":0,\"removeEventListener\":1,\"globalStorage\":0,\"openDatabase\":0,\"indexedDB\":1,\"attachEvent\":0,\"ActiveXObject\":0,\"dispatchEvent\":1,\"addBehavior\":0,\"addEventListener\":1,\"detachEvent\":0,\"fireEvent\":0,\"MutationObserver\":1,\"HTMLMenuItemElement\":1,\"Int8Array\":1,\"postMessage\":1,\"querySelector\":1,\"getElementsByClassName\":1,\"images\":1,\"compatMode\":\"CSS1Compat\",\"documentMode\":0,\"all\":1,\"now\":1,\"contextMenu\":null}", "comment": ""}, {"name": "lt", "value": "1606774610576-5", "comment": ""}, {"name": "ps", "value": "true,true", "comment": ""}, {"name": "cv", "value": "fb1df2e3f14da83d955799ae20c68696a2efb3b4", "comment": ""}, {"name": "fp", "value": "false", "comment": ""}, {"name": "sp", "value": "false", "comment": ""}, {"name": "br", "value": "Firefox", "comment": ""}, {"name": "ieps", "value": "false", "comment": ""}, {"name": "av", "value": "false", "comment": ""}, {"name": "z", "value": "{\"a\":250027159,\"b\":1,\"c\":0}", "comment": ""}, {"name": "zh", "value": "", "comment": ""}, {"name": "jsv", "value": "1.5", "comment": ""}, {"name": "nav", "value": "{\"userAgent\":\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0\",\"appName\":\"Netscape\",\"appCodeName\":\"Mozilla\",\"appVersion\":\"5.0 (Macintosh)\",\"appMinorVersion\":0,\"product\":\"Gecko\",\"productSub\":\"20100101\",\"vendor\":\"\",\"vendorSub\":\"\",\"buildID\":\"20181001000000\",\"platform\":\"MacIntel\",\"oscpu\":\"Intel Mac OS X 10.15\",\"hardwareConcurrency\":2,\"language\":\"en-US\",\"languages\":[\"en-US\",\"en\"],\"systemLanguage\":0,\"userLanguage\":0,\"doNotTrack\":\"unspecified\",\"msDoNotTrack\":0,\"cookieEnabled\":true,\"geolocation\":1,\"vibrate\":1,\"maxTouchPoints\":0,\"webdriver\":true,\"plugins\":[]}", "comment": ""}, {"name": "crc", "value": "{\"window.chrome\":\"-not-existent\"}", "comment": ""}, {"name": "t", "value": "f8dcad4cd33b8492671e8ee611c5a497448ecb10", "comment": ""}, {"name": "u", "value": "df071d33ceea54f192dfeff1afbaf243", "comment": ""}, {"name": "nap", "value": "11133333331333333333", "comment": ""}, {"name": "fc", "value": "true", "comment": ""}], "comment": ""}, "headersSize": 2566, "bodySize": 2852, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Type", "value": "text/html"}, {"name": "Content-Length", "value": "0"}, {"name": "Expires", "value": "Mon, 30 Nov 2020 22:16:51 GMT"}, {"name": "Cache-Control", "value": "max-age=0, no-cache, no-store"}, {"name": "Pragma", "value": "no-cache"}, {"name": "Date", "value": "Mon, 30 Nov 2020 22:16:51 GMT"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Set-Cookie", "value": "ak_bmsc=AE7E2A410E245AB5A6C7FC39A9F10C0D1724279C322300004D6FC55FDFE07821~pl63MrlwyVQ6OwLZ3q+6HMBq2Chf1jjcLwwUHmHNv13/aQIwEzskvhqgZBOv62XYEhtg5TvrrvEv+2KA2t+fa6QNKP6ulVzU5b28vMwh/LwYXAYuZTeDgQdtUC9PZopov+IiYTp7rW7sN9kpd0JItIt61ppaDvAKeFhzk6RI42vRzgjJonYEmWEuhU4MEIkbJ/ypeDa3Sb8mpt+WmTXsIemK6VDzRkt/Q2zzgVz9WHXqpUsMpCQ3yCj5kU+TxHmE6COFiyolfM2JbCA0kAxAomyClmmSUoYxaQedhebN9W24w=; expires=Tue, 01 Dec 2020 00:16:45 GMT; max-age=7194; path=/; domain=.xfinity.com; HttpOnly; Secure"}], "content": {"size": 0, "mimeType": "text/html", "text": "", "comment": ""}, "redirectURL": "", "headersSize": 719, "bodySize": 0, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": -1, "dns": -1, "connect": -1, "send": 0, "wait": 24, "ssl": -1}, "serverIPAddress": "104.77.245.23", "comment": "", "time": 24}

This also had ST=MA, so state = Massachusetts, CTY=SOmerville (City), GEO=True, ISP=RCN. This was all under the request headers.

Therefore, we can potentially add even more keywords to our location searches.

The one thing I will say, however, is that it is pretty clear the differences between how each website set up their requests is HUGE and there does not seem to be much consistency. So, to be sure we are looking for the right keywords for locatiions we should likely add a few more sites, but it does take some time to manually look through this stuff. It is a lot of text!

rgoldstein01 commented 3 years ago

2) To come: trying things the other way around 3) credit card checks seem to be a more specific instance I will try after these more general keywords 4) This is pretty simple. Basically in the URL you can see exactly where these requests are going. Most are just do the server, but for example the one above from CTTransit you can see one of them is to Google's geolocate tool. This will be one of the easiest things to parse, as I can just create a listof all the different URLS we observe.

rgoldstein01 commented 3 years ago

OK so now trying things the other way around. I tried just signing into yahoo and then going through a few pages.

There wasn't much here I did not expect. EVery call being made was to a Yahoo URL. My email was in a bunch of different requests but it doesn't look like it was ever being sent to a third party. I will say it was pretty clear they were fingerpringint my browser, as in one of the requests here was a value: {\"language\":\"en-US\",\"colorDepth\":24,\"deviceMemory\":\"unknown\",\"pixelRatio\":2,\"hardwareConcurrency\":2,\"timezoneOffset\":300,\"timezone\":\"America/New_York\",\"sessionStorage\":1,\"localStorage\":1,\"indexedDb\":1,\"cpuClass\":\"unknown\",\"platform\":\"MacIntel\",\"doNotTrack\":\"unspecified\",\"plugins\":{\"count\":0,\"hash\":\"24700f9f1986800ab4fcc880530dd0ed\"},\"canvas\":\"canvas winding:yes~canvas\",\"webgl\":1,\"webglVendorAndRenderer\":\"Intel Inc.~Intel(R) Iris(TM) Plus Graphics 640\",\"adBlock\":0,\"hasLiedLanguages\":0,\"hasLiedResolution\":0,\"hasLiedOs\":0,\"hasLiedBrowser\":0,\"touchSupport\":{\"points\":0,\"event\":0,\"start\":0},\"fonts\":{\"count\":27,\"hash\":\"d52a1516cfb5f1c2d8a427c14bc3645f\"},\"audio\":\"35.7383295930922\",\"resolution\":{\"w\":\"1440\",\"h\":\"900\"},\"availableResolution\":{\"w\":\"900\",\"h\":\"1440\"},\"ts\":{\"serve\":1606866092163,\"render\":1606866092869}}

Next I tried destructoid.com. One thing in specific i was looking for here was whether I could find mention of my logged in emaiil address in any requests to a URL that wasn't destructoid. While I did find requests to sites like Google Analytics, none of my personal data was ever sent there. So there wasn't much there all in all.

Jalopnik I created an account through my Google account so there were a ton of requests going to Google with my email but that is to be expected. The unfortunate thing about a lot of Google requests is most of them are encyrpted within the HTTPS requests themselves and basically unreadable. Interestingly enough while I was on the website with Firefox firefox gave me a notification that the website was trying to fingerprint. I said no so none of the data was present in the requests but if I had said yes it certainly would have been there.

Here's an example of a Google request:

{"pageref": "jalopnik.com", "startedDateTime": "2020-12-01T19:03:59.368-05:00", "request": {"method": "POST", "url": "https://accounts.google.com/_/lookup/accountlookup?hl=en&_reqid=68640&rt=j", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "accounts.google.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "X-Same-Domain", "value": "1"}, {"name": "Google-Accounts-XSRF", "value": "1"}, {"name": "Content-Type", "value": "application/x-www-form-urlencoded;charset=utf-8"}, {"name": "Content-Length", "value": "4712"}, {"name": "Origin", "value": "https://accounts.google.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://accounts.google.com/o/oauth2/auth/identifier?client_id=835741439380-la2837f0st8q361cu6tv5k1dbn1m8sn8.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fkinja.com%2Fapi%2Fprofile%2Faccount%2Foauth%2Fgoogle&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email&state=1606867434238%3A-DqSkojyDGC-bBnWvoA_EdB8pkAX_VsuyqNEQ1P1rrA%3D&flowName=GeneralOAuthFlow"}, {"name": "Cookie", "value": "__Host-GAPS=1:L3DbihVPCII8UYtHj-ddxolyV18jzw:NfcUw2cWlKhAkiDy; NID=204=U5nDBYiSu2tW-swmzaLTPoKGx6fPxLOgn1EHnmCiyeK1x8meQRjyIF4fT8iEEd8bLvQHK6KMvbHBu3L6mXfzR_TAxlv8fvDOTezQW3FOXXg_MXK2OLHb8HlK3fzGeRBgJ8D-vXPTgFPHJvP-vpOqPe65x0Ijxiyne96uVsEpAZc"}], "queryString": [{"name": "hl", "value": "en"}, {"name": "_reqid", "value": "68640"}, {"name": "rt", "value": "j"}], "postData": {"mimeType": "application/x-www-form-urlencoded;charset=utf-8", "params": [{"name": "client_id", "value": "835741439380-la2837f0st8q361cu6tv5k1dbn1m8sn8.apps.googleusercontent.com", "comment": ""}, {"name": "scope", "value": "https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email", "comment": ""}, {"name": "service", "value": "lso", "comment": ""}, {"name": "continue", "value": "https://accounts.google.com/signin/oauth/consent?authuser=unknown&part=AJi8hAN-5-u9qVyRJlaquhnAM3v3zczS0N5jpMV6NhFOZ1QUPcmzWDlQnTbMR3tX7drjZuPFc9ZysGe1MFb4uCyPUZALvh04LuHFKnIwXJsfYEm-DqFHvF3ALsAyeTSVjuCJBii55BiYjne7Xcb2ff_XAcXgN6Me0j9Cb0DLb6yd_pHV8WzA570HvNOu88Vy9NQyxuQ6Q0AEEupUHdFlOzcmHxMtoPeDRIlDKXKcVIoeijoxBTLB5oZLlM4UPT8jq649gi5ueIU91N4sXJU7TyIyhYCsxMX_eZaucQOF1A-SISPa9Nmpcen6AfBsRy69PNO_I9i4I4XB564vLmNx2OAV3qTzN2QqoQVeVnjfjM6ecuoVWMR7CDQDWtDimBXd12Pfuo9y6ETPxDMs5f3TOeAOI1z8n8dfVIZGYpzYRPExJVuRmKnZgob15auUhXKEDTfU7TlYQ1aO&as=S-888431266%3A1606867434684758#", "comment": ""}, {"name": "f.req", "value": "[\"christian.betherly\",\"AEThLlxnJtmp7gESk0__1rhe_zPcx7_KJv5h1iFeAAt2G-POUGfihJN5IiRlMKcJVURZBUTFrfeyun7_ze5b9TbG5atlGPHrY2CltjX4DRSA2ieHkLSQdaLjdXwbW3xvBaszeQFBVsY9Pioou308xaLOLQpSaSe-b4Jv8ZzmCfotWB5Kid8DmE15M3Gl6emLQXjKxHKSBHTuWm7ICUCD69UYcc1g0mK6tA\",[],null,\"US\",null,null,2,false,true,[null,null,[2,1,null,1,\"https://accounts.google.com/signin/oauth?client_id=835741439380-la2837f0st8q361cu6tv5k1dbn1m8sn8.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fkinja.com%2Fapi%2Fprofile%2Faccount%2Foauth%2Fgoogle&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email&state=1606867434238%3A-DqSkojyDGC-bBnWvoA_EdB8pkAX_VsuyqNEQ1P1rrA%3D\",null,[],4,[],\"GeneralOAuthFlow\",null,[]],1,[null,\"835741439380-la2837f0st8q361cu6tv5k1dbn1m8sn8.apps.googleusercontent.com\",[],\"!ChQyQWQzVzlsYnhqU3Nia2dSNXJoWBIfSTFTdVlGN2FZbHNWQU9FaERQUmpBazFLempNTVloYw\u2219AF-3PDcAAAAAX8grajI5ClgmJ1oCsXutFRy2RCtF4Yb1\",null,null,null,null,null,null,null,\"https://kinja.com\",\"S-888431266:1606867434684758\",false,null,null,null,null,null,null,null,null,14,null,null,[],true,null,null,[],[],null,true,null,null,true],null,null,null,true],\"christian.betherly\",null,null,null,true,true,[]]", "comment": ""}, {"name": "bgRequest", "value": "[\"identifier\",\"!ammlaUDNAAUrmx8XK0J2UV4TKxO76FjWdwcXUoKg7wIAAAA5UgAAABJoAQcKAS_nWsn3cb2IzmRbFL2iqvQNoAsL8djJC7UpSJUkbyYhDBZRUzT1SU4y2QHSbsFziLfzjyO-YOJ_NxiwfeKkPRKSrZHr465qFmEeTyBFy9411sLXIwzb_KjlRknPIx4huGhrImMf6OyQjWNOD5aXcg5VrF_K1IfN6CJb4DoogXFhyhogM1olS7qnRBLRMmDdwpXwNbEeJoyYnSTHldzQYQqm060KLA17jQuQYJDaSP4bhZyku5lqOgi86-c0MWNkJRWrgN6JkIY3go4IkePBTz4AoBxFSDlh2PK0lXXjl1T2UmCZ0I4MWoygLSYS9owU72PPMIH3nrMYknCIpEQHHBsNy3zqBHjJmG58AS4dITLxuWtV2v0ygPS3d-NjVjFw4feqgmIyQUOqY0PA0mefg5SZAgrb4HkZRH1hwLafLKMOX19dBUJtM-zQT3cbpYYw8jGsI8h8m-CSmIuB52D2F2_LeWE6xQggU04DmGWobuNJa3SM6eXZdfqxxunhbsHfmmN6lzdTLVAhEximh58MXU42e1nd1JL5d8eR7-2mFoP2E4yIFlax59GT2MMe4fjUTOtzI5dlJSYHoGPBfpFvaAQh_DqBCIVR1n1G0hdrQkt5PK8f5fkzFdLKI-SbMbzQV1wkvxJ1T90GpUQYFtcrHXtwF-1_hA_4vhV0rL58HlTgEfLRXGRiXw-c-2_3aBL5T0XD0iGM5f2hFaY4rsi2zcAuZbaWw8Bepy9N5fqtLQSf3WkYJXWxDGxzsI_FOiYq8raqbO9bGc9DFbbtx687edCWa-6SOaJYV0ICAGObEIXVDDjAAecheWE8pNzwBCLGT-FH2pg-TTXDyVouWwH_UwIMCNX8nFs2qd1xQFIdgGDTngqkzHp1ybq5Rk8qLlM6RgRC4zaXcEXisEVV9QMQ-mbs8gW9GnX36FceZSUd90-aEhB5wkWqWND1-pne9-MsoMvX9Xous4Yd3ieJqo7Sg4c69qXhqjGT3ygofP1BskZXThqIqypL75CXRVCMq-HFlqCJVSe_wL5Nd1SJXDTJCQWeYP3Io6r_DfVNL63OOay1d6eOQnKe5MkeEDHTwBrXEqswNY5biQ47Ta5LDwk\"]", "comment": ""}, {"name": "at", "value": "AFoagUXM4XYHUm9-GcRXtENo18XeNaEsMg:1606867434763", "comment": ""}, {"name": "azt", "value": "AFoagUXM4XYHUm9-GcRXtENo18XeNaEsMg:1606867434763", "comment": ""}, {"name": "cookiesDisabled", "value": "false", "comment": ""}, {"name": "deviceinfo", "value": "[null,null,null,[],null,\"US\",null,\"835741439380-la2837f0st8q361cu6tv5k1dbn1m8sn8.apps.googleusercontent.com\",[],\"GeneralOAuthFlow\",\"!ChQyQWQzVzlsYnhqU3Nia2dSNXJoWBIfSTFTdVlGN2FZbHNWQU9FaERQUmpBazFLempNTVloYw\u2219AF-3PDcAAAAAX8grajI5ClgmJ1oCsXutFRy2RCtF4Yb1\",[null,\"835741439380-la2837f0st8q361cu6tv5k1dbn1m8sn8.apps.googleusercontent.com\",[],\"!ChQyQWQzVzlsYnhqU3Nia2dSNXJoWBIfSTFTdVlGN2FZbHNWQU9FaERQUmpBazFLempNTVloYw\u2219AF-3PDcAAAAAX8grajI5ClgmJ1oCsXutFRy2RCtF4Yb1\",null,null,null,null,null,null,null,\"https://kinja.com\",\"S-888431266:1606867434684758\",false,null,null,null,null,null,null,null,null,14,null,null,[],true,null,null,[],[],null,true,null,null,true],null,null,null,null,0,null,false]", "comment": ""}, {"name": "gmscoreversion", "value": "undefined", "comment": ""}, {"name": "checkConnection", "value": "youtube:1254:1", "comment": ""}, {"name": "checkedDomains", "value": "youtube", "comment": ""}], "comment": ""}, "headersSize": 1177, "bodySize": 4712, "comment": ""}, "response": {"status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Type", "value": "application/json; charset=utf-8"}, {"name": "X-Frame-Options", "value": "DENY"}, {"name": "Cache-Control", "value": "no-cache, no-store, max-age=0, must-revalidate"}, {"name": "Pragma", "value": "no-cache"}, {"name": "Expires", "value": "Mon, 01 Jan 1990 00:00:00 GMT"}, {"name": "Date", "value": "Wed, 02 Dec 2020 00:03:59 GMT"}, {"name": "X-Content-Type-Options", "value": "nosniff"}, {"name": "Content-Encoding", "value": "gzip"}, {"name": "Strict-Transport-Security", "value": "max-age=31536000; includeSubDomains"}, {"name": "X-XSS-Protection", "value": "1; mode=block"}, {"name": "Server", "value": "GSE"}, {"name": "Set-Cookie", "value": "__Host-GAPS=1:Zt0yHo-IBsCrmnODwjPHQwRwX43WNQ:NWwR41AMV-EfE6Px;Path=/;Expires=Fri, 02-Dec-2022 00:03:59 GMT;Secure;HttpOnly;Priority=HIGH"}, {"name": "Alt-Svc", "value": "h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\""}, {"name": "Transfer-Encoding", "value": "chunked"}], "content": {"size": 1231, "mimeType": "application/json; charset=utf-8", "text": ")]}'\n\n[[[\"gf.alr\",1,\"AEThLlyYZ1x0Bq97eI2PXhROn3qj5B2ZolLS_v6UAOI2zL8BlUhBSgGs5fky051Y5WftCO7Sdqg6zH1SJ9A5qdWU4QZ7jkqcxVbac8bb1Pr7GIQpUnUJWfcmfkjg-pLsEWBkgRWZu7jCHREaFkSJkwsIJf9uxgmlrOOTst8V6ii97GZkHokVoRs-rlTX4wbhpjGnuT_i_GOIqPMWoShtcXtA1bRYrM59TEveD5UhWC6zwJxmtkwOvuXFQR_MFCa4YjYqPRhFUQewOSBkvtHvAxg0siVw7BlnzhNW3QssfPo8VE7LkbI15QeFKUD4hIyMestQyMf682OJYrESo_bRC4ZrK0gp2tABCiVczAfd279zDSdDLmnu72UFeAJaxgsdhyfcYOBHxNlOATQB202TelXM_jZC-9w3xBQowKlZodYe45YbIEGYHmhY72YLeeuerXepgta7zdXCUJsgCFuP1xTg89dfqij_7sOZhWoQfkaygmrjxqFNz28\",[[\"christianbetherly@gmail.com\",null,null,null,null,\"christian.betherly@gmail.com\",\"gmail.com\",null,null,2]\n]\n,null,null,null,[\"gf.sisr\",1,null,null,[[[null,null,\"FIRST_AUTH_FACTOR\",1,null,\"INITIALIZED\",null,null,1,7,1,1,null,null,null,null,\"christian.betherly@gmail.com\",\"https://lh3.googleusercontent.com/-XdUIqdMkCWA/AAAAAAAAAAI/AAAAAAAAAAA/AMZuuckflWAKGiml_cnkcXW0IMvgx6Njng/photo.jpg\",null,null,1,null,[]\n,null,null,null,null,1,{\"1001\":[1]\n,\"5001\":[]\n}]\n]\n,0,1,1,1,1,1,null,[null,[3]\n,null,null,3]\n,[null,null,null,null,0]\n]\n]\n,null,null,null,null,null,null,null,null,null,null,null,null,[]\n]\n,[\"gf.ttu\",0,\"AM3QAYY_3FlyrUmr4xsuLfW_KbWRWV3xdJPXgjQXf8FtQOxp7_XG5Rp_kOtIpCmG\"]\n,[\"e\",3,null,null,1231]\n]]", "comment": ""}, "redirectURL": "", "headersSize": 772, "bodySize": 781, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": -1, "dns": -1, "connect": -1, "send": 0, "wait": 633, "ssl": -1}, "serverIPAddress": "172.217.12.173", "comment": "", "time": 633}

As you can see there is a bunch of unreadable encyrpted stuff.

Next I tried latimes.

This was interesting for a few reasons. First, to sign up they asked me for my zip code, but then i could not find my zip in any of the requests I had. It seems weird they'd ask that and not store that info so my guess is that they encoded it.

The authorization to sign up was regular besides that. nothing else to see. But there were requests sent to youtube, google,e and a bunch of other ad networks. Here is one sent to Rubicon, an online ad company (https://rubiconproject.com/):

{"pageref": "latimes.com", "startedDateTime": "2020-12-01T19:13:18.538-05:00", "request": {"method": "POST", "url": "https://prebid-a.rubiconproject.com/event", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Host", "value": "prebid-a.rubiconproject.com"}, {"name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:83.0) Gecko/20100101 Firefox/83.0"}, {"name": "Accept", "value": "*/*"}, {"name": "Accept-Language", "value": "en-US,en;q=0.5"}, {"name": "Accept-Encoding", "value": "gzip, deflate, br"}, {"name": "Content-Type", "value": "application/json"}, {"name": "Content-Length", "value": "1565"}, {"name": "Origin", "value": "https://www.latimes.com"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Referer", "value": "https://www.latimes.com/"}], "queryString": [], "postData": {"mimeType": "application/json", "text": "{\"eventTimeMillis\":1606867997611,\"integration\":\"dmpbjs\",\"version\":\"4.11.0\",\"referrerUri\":\"https://www.latimes.com/\",\"referrerHostname\":\"www.latimes.com\",\"channel\":\"web\",\"wrapperName\":\"20520_latimes_prod\",\"session\":{\"id\":\"6a698773-d3bc-4226-8c5e-b0272844ac5f\",\"pvid\":\"ed4ca875\",\"start\":1606867993578,\"expires\":1606889593578},\"auctions\":[{\"clientTimeoutMillis\":1000,\"samplingFactor\":1,\"accountId\":20520,\"adUnits\":[{\"adUnitCode\":\"google-ad772717ba-453c-4cd3-937f-2b61494c4ea4\",\"transactionId\":\"547b41f3-8d8b-4ccd-81f4-0b56ee325ba2\",\"mediaTypes\":[\"banner\"],\"dimensions\":[{\"width\":300,\"height\":600},{\"width\":300,\"height\":250}],\"adserverTargeting\":{},\"gam\":{\"advertiserId\":4744543393,\"adSlot\":\"/21787098806/web.latimes/homepage\"},\"pbAdSlot\":\"/21787098806/web.latimes/homepage\",\"bids\":[{\"bidder\":\"rubicon\",\"bidId\":\"15ee2191fb8b9e88\",\"status\":\"no-bid\",\"source\":\"client\",\"clientLatencyMillis\":1007,\"params\":{\"accountId\":20520,\"siteId\":267796,\"zoneId\":1327982}},{\"bidder\":\"openx\",\"bidId\":\"1754bc3861f6e1a\",\"status\":\"no-bid\",\"source\":\"client\",\"clientLatencyMillis\":444},{\"bidder\":\"ix\",\"bidId\":\"19cbb4afe733cfa8\",\"status\":\"no-bid\",\"source\":\"client\",\"clientLatencyMillis\":1008},{\"bidder\":\"ix\",\"bidId\":\"201f789a951d293\",\"status\":\"no-bid\",\"source\":\"client\",\"clientLatencyMillis\":1008},{\"bidder\":\"criteo\",\"bidId\":\"2237080e0434c198\",\"status\":\"no-bid\",\"source\":\"client\",\"clientLatencyMillis\":1008},{\"bidder\":\"appnexus\",\"bidId\":\"242904d8431a366\",\"status\":\"no-bid\",\"source\":\"client\",\"clientLatencyMillis\":1008}],\"status\":\"no-bid\",\"accountId\":20520,\"siteId\":267796,\"zoneId\":1327982}]}]}", "comment": ""}, "headersSize": 385, "bodySize": 1565, "comment": ""}, "response": {"status": 202, "statusText": "", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Date", "value": "Wed, 02 Dec 2020 00:13:18 GMT"}, {"name": "Content-Type", "value": "application/json;charset=UTF-8"}, {"name": "Content-Length", "value": "61"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Vary", "value": "Origin"}, {"name": "Vary", "value": "Access-Control-Request-Method"}, {"name": "Vary", "value": "Access-Control-Request-Headers"}, {"name": "Access-Control-Allow-Origin", "value": "*"}], "content": {"size": 61, "mimeType": "application/json;charset=UTF-8", "text": "{\"httpStatus\":202,\"message\":\"Event accepted for processing.\"}", "comment": ""}, "redirectURL": "", "headersSize": 267, "bodySize": 61, "comment": ""}, "cache": {}, "timings": {"comment": "", "receive": 0, "blocked": 0, "dns": 0, "connect": 484, "send": 0, "wait": 30, "ssl": 451}, "serverIPAddress": "54.209.150.172", "comment": "", "time": 515}

In the request it is definitely sending information about my browser, as well as information about myself like my "adID"

All in all not much really to find here. I think location is definitely something we can look for, as well as email. Aside from that, I do not see many other things that stick out (right now)

SebastianZimmeck commented 3 years ago

Not sure if this is norm but this is where this was located.

I do not think there is a standard format. Maybe, some convention, but essentially every site can do it differently.

The other interesting thing is there is a reference to the Google geolocation API, which could be another key thing we look out for: https://www.googleapis.com/geolocation.

I agree that should be another useful feature to look for. So we have:

  1. location keywords/regex in HTTP requests headers and bodies,
  2. request of location permission use in the browser, and
  3. use of geolocation APIs.

These could be used as an ensemble of heuristics to make the call whether a first or third party is collecting location data.

So, to be sure we are looking for the right keywords for locatiions we should likely add a few more sites, but it does take some time to manually look through this stuff.

Yes, and at some point once we have analyzed enough, we can then converge towards a techique.

My email was in a bunch of different requests but it doesn't look like it was ever being sent to a third party.

Identifying email addresses should be doable, for example, via a regex or a Python library. One tricky point could be that we would not know whether that is an email address of the user or of the service (e.g., info@companyx.com). Maybe, one approach could be to say, if it is a POST request we assume that an email address is posted to the server and for a GET request nothing is collected or disclosed because something is being sent to the user. In other words, take into account the type of HTTP request (which may be something to consider in other parts of the analysis as well).

I will say it was pretty clear they were fingerpringint my browser

Why do you think so? Did Firefox show a warning message (as shown here)? If so, I wonder how Firefox is identifying fingerprinting; just checking the use of the HTML 5 canvas API? I will open a separate fingerprinting issue.

rgoldstein01 commented 3 years ago

Per issue #15 we should include and begin to look for HTTPS requests that contain browser related info for fingerprinting. ie colorDepth\":24,\"deviceMemory\":\"unknown\",\"pixelRatio\":2, etc.

rgoldstein01 commented 3 years ago

Will also add a specific URL watch for the following URL: https://www.facebook.com/tr/ as we know this is directly related to a tracking pixel.

SebastianZimmeck commented 3 years ago

@rgoldstein01, do you mean that Facebook is using https://www.facebook.com/tr/ exclusively for the tracking pixel? That would be great and make it very easy to keep track of.

Edit: OK, I see now, per your other comment, the answer is yes.

rgoldstein01 commented 3 years ago

@SebastianZimmeck We should do some further analysis to confirm, but from my initial and bit deeper look into things, I believe so. I agree, this is good news.

SebastianZimmeck commented 3 years ago

@notowen333, I have added you to this issue as @rgoldstein01 mentioned that he could use some help analyzing the HTTP requests. We can discuss details on Tuesday.

As the location analysis is our most successful functionality so far, it may make sense that we develop it further, see where we get, and possibly use it as a blueprint for other functionalities.

rgoldstein01 commented 3 years ago

I think I will create either a .yaml or .json file to begin our network keyword "checklist", but I'd just like to document here on this issue the keywords I think are relevant so far:

Location keywords:
longitude
lng
latitude
lat
altitude
location
geolocation
geo_location
delivery_zip

URL:
https://googleapis.com/geolocation

fingerprinting:
screenSize
platform
timezone
contentLang (en-US)
canvas
fonts
adblock
deviceMemory
pixelRatio
colorDepth
screenWidth
screenHeight
screenDepth
webGL

FB Pixel URL:
https://www.facebook.com/tr/

I think we need to look through a few more sites that use fingerprinting to either add or refine to that list, as I am sure there are more.

In addition to adding this YAML file, I need to add to make a few tweaks to the current tool as of now it just outputs all of the HTTP requests into a JSON file for me to read manually. Of course, for our actual tool, we will want these things to be automated, and I think I will most likely just keep the logs as a JSON object in python as I am parsing through requests. Then, when the developer is done browsing, I can just analyze the JSON of network requests within python as an object, rather than its own file. Not a big deal but just writing out my process.

Another note on URLs: I assume, like Google's geolocation API, there are other URLS we could ID as being specifically realted to location or some other relevant fact, so we should be on the lookout for those as well. For example, this list here: https://blog.api.rakuten.net/top-ip-geolocation-apis/ has the top 10 geolocation APIs. Google is the first, but we could maybe just add all 10 of these to our list?

On a similar note, on that same site, I found a list of the top 10 best ads APIs, so we could also look into these API URLS to include, as well. https://blog.api.rakuten.net/top-10-best-ads-apis/

SebastianZimmeck commented 3 years ago

On the fingerprinting, the most important clue may be the use of the HTML5 Canvas API. You can test your browser here. There you can also see how much each detected feature contributes to a unique fingerprint (the higher the bits, the more).

rgoldstein01 commented 3 years ago

Issue #20 should full under this, as well. Will add a feature that checks all the URLs for their protocols. If it is just HTTP, will flag.

rgoldstein01 commented 3 years ago

Going to begin a YAML file for some keywords. Of course, we can change formats etc. if we see fit.

rgoldstein01 commented 3 years ago

Closing this for now as we buff up the yaml file I may open again.