Another bunch of potential trackers missing

[IzzySoft]

I've added a new virtual column to my tracker table and found that there are 104 entries which are not covered by Exodus. Table data are based on definitions by Appbrain plus from my own scanner. Condition for the virtual column is_tracker is:

  case when exodus_id is not null OR category='ats'
       OR tags like '%_ads,%' OR tags like '%,analytics,%' OR tags like '%,mediation,%'
       OR types like '%,ad,%' OR modwarn_id is not null then 1 else 0 end

So I did a select * from libs where is_tracker=1 and exodus_id is null; using the INTO OUTFILE clause of MariaDB (sorry, the JSON functions are not yet available in 10.0 – and no, I cannot simply upgrade but will do eventually). I've added a header line, and attached the results as .gz here:

zz_exodus.csv.gz

I guess not all of them are worth being added (some are almost completely unknown/unused) – but things like amazon_aws_metrics or amazon_mobile_analytics probably are :wink:

Related for reference: #31

[pnu-s]

Hi @IzzySoft

Thanks a lot! I'll check how many matches we have in the apps already scanned by exodus and see what we can do from there

[IzzySoft]

Thanks! I don't expect you'll pick them all. One third would be a good rate I guess – but I thought to send them all nevertheless (couldn't tell which third you'd like :rofl:).

[pnu-s]

FYI @IzzySoft there are 10 trackers which seem good candidates for an integration into exodus.

For the rest, some didn't have package name so I didn't look into them, some didn't match with our list of apps in exodus, some where already in exodus, and the rest I didn't have time to review yet :)

[IzzySoft]

Thanks for looking into it @simpnu! Yeah, I expected something along those lines. Those without package name come solely from Appbrain (and I haven't found a match yet for them). Some are so unknown that even Appbrain (who lists them) couldn't find a hit (so I wonder why they list them at all)…

Btw: I've noticed that one of my "mergers" failed for a while until shortly after I sent you the extract. If you want, I can send you "fresh data" for the above, which might have more package names matching (that's what the merge script does: try to match Appbrain stuff with IDs/packageNames from Exodus and me). Just let me know, I still kept the steps for creating the CSV so it should be a thing of 2 minutes. But probably doesn't change much.

[IzzySoft]

While you are still processing this, let me add my latest find right away:

{
 "id": "/com/google/cloud/audit",
 "details": "writes audit log entries to Google Cloud Services to help answer the questions of „who did what, where, and when?”",
 "name":"Cloud Audit Logs",
 "type":"Mobile Analytics",
 "url":"https://cloud.google.com/logging/docs/audit"
}

[jawz101]

Are you just looking for buzzwords? Here are a few partial strings that I think of when I try to identify 3rd party undesirables:

'%beacon%'
'%geofenc%'
'%lytic%'
'%report%'
'%metric%'
'%segment%'
'%locat%'
'%upload%'
'%proximity%'
'%advert%'
'%interstitial%'
'%segment%'
'%mediat%'
'%sdk%'

[jawz101]

I added about 30 to my list of things to maybe add to ETIP (which has like 800 companies that may or may not end up being trackers.

https://github.com/jawz101/potentialTrackers/blob/master/potentialTrackers.csv

[IzzySoft]

Are you just looking for buzzwords?

Nope. Those are libraries my own scanner identified. For yet-unknown-to-me packages I then try to find background information. If they turn out to be trackers, I mark them such – and from time to time bring up small collections of them here.

@pnu-s as this issue was dormant now for 2 month, just a little heads-up that you wanted to pick some of the candidates :wink:

[jawz101]

Would @pnu-s try to check some of these? My link above has tons I slowly sift through. You could probably flip the website column as well & see if that yields any additional matches. I'm trying to get through the "Retail & Proximity Marketing" category 1st since they probably do more location-based tracking.

ad.nugg.android.
biz.clickky.ads_sdk.
cn.domob
cn.smartmad
com.adfonic
com.adgoji.
com.adition.android.sdk.
com.adknowledge
com.admarvel
com.admob
com.adsmogo
com.adwhirl
com.adx
com.amazon.analytics
com.amazonaws.metrics
com.amazonaws.mobileconnectors.amazonmobileanalytics
com.aol.mobile.sdk
com.aol.metrics
com.appannie.
com.appcelerator.
com.appfireworks
com.appflood
com.papaya
com.appia.sdk.
com.applifier
com.appmachine.
com.apprupt.sdk
com.appscend.
com.aquafadas.dp.reader.sdk.
com.arubanetworks.meridian.
com.attendify.android.app.
com.azetone.
com.bee7.sdk
com.brandify.brandifymobilesdk
com.brightroll
com.bugsense
com.burstly
com.burtcorp.sdk
com.buzzfeed.ads.
com.chalkdigital.cdads.
com.cloudinary.
com.contentful.
com.contentsquare.android.sdk
com.coremedia
com.crosspromotion
com.cxense.cxensesdk
com.datorama.
com.desygner.
com.digg.
com.dimelo.dimelosdk
com.directtap
com.dnb.vipps.
com.docusign.
com.facebook
com.fgl
com.fractionalmedia.sdk
com.freshchat.consumer.sdk
com.freshdesk.
com.fusepowered
com.getjar
com.ghostery.privacy.appnoticesdk
com.gingersoftware.
com.grammarly.
com.greystripe
com.growmobile
com.hootsuite.compose.sdk
com.hp.android.notificationsdk
com.hybris.mobile.
com.ifttt.
com.igaworks.adbrixtracersdk
com.impactradius.
com.inneractive
com.integralads.avid.library.
com.intercom.
com.ionicframework.
com.iperceptions.iperceptionssdk
com.iqzone
com.iterable.iterableapi.
com.janrain.android.
com.jumptap
com.kahuna
com.kaltura.
com.kamcord
com.kinvey.
com.kodak.kioskconnect.
com.kofax.mobile.sdk
com.kontagent
com.kony.sdk
com.ktplay
com.layer.lsdka
com.layer.sdk
com.leadbolt
com.livechatinc.
com.liveperson.api.sdk
com.loginradius.sdk
com.loopme.
com.magnetic.
com.mediabrix
com.medialets
com.medium.android.
com.microsoft.advertising
com.microsoft.azure.mobile.analytics
com.millennialmedia
com.mindbodyonline.ironhide.
com.mintegral
com.mobileapptracker
com.mobpartner.android.publisher.
com.mocoplex
com.mologiq
com.mtraction.mtractioninapptracker
com.nanigans
com.nec.
com.netbiscuits.bild.android.
com.nielsen.app.sdk
com.nuance.nmsp.client.sdk
com.omniata
com.oplytic.oplyticsdk
com.outfit7
com.parsely.parselyandroid
com.percolate.
com.phunware.
com.playhaven
com.pocketchange
com.pollfish
com.pontiflex
com.prime31
com.progress.
com.quark.mobileiq.
com.quora.android.
com.radiumone
com.relay42.sdk
com.revmob.ads.
com.rhythmone.ad.sdk
com.rokolabs.sdk
com.rovio.fusion.cloudservicesnativesdk
com.sas.
com.scala.mamp.
com.selligent.sdk
com.sensorberg.sdk
com.sessionm
com.sharethrough.android.sdk
com.shazam.
com.shopify.sdk
com.skyhookwireless.
com.smartfoxserver
com.splunk.mint.
com.sprinklr.collaboration.
com.squareup.
com.steema
com.sudouest.android
com.surveymonkey.surveymonkeyandroidsdk
com.swelen.ads.
com.tapcontext
com.tapit
com.tappx.sdk
com.tapsense
com.tatvic.lib.
com.telesign.mobile.
com.teliver.sdk
com.threatmetrix
com.thunkable.
com.timgroup
com.treasuredata.android.
com.tremorvideo
com.trialpay
com.trophit.
com.truvie
com.twilio.
com.ubermedia.
com.umbel.
com.usabilla.sdk
com.usablenet.android.
com.usebutton.sdk
com.useinsider.insider.
com.uservoice.uservoicesdk
com.validity.fingerprint.
com.vdopia
com.vervewireless.advert
com.vimeo.android.analytics.
com.vpon
com.vwo.mobile.
com.waps
com.wikia.
com.wix.
com.wiyun
com.wooboo.adlib_android
com.woodwing.
com.xamarin.forms.
com.yelp.android.analytics.
com.yieldmo.sdk
com.yume
com.zendesk.sdk
com.zenjoy.ads
com.zestadz.android
com.zumobi.
com.woopra.tracking
im.getsocial.sdk
io.applink.applinkio.AppLinkIO
io.augur.
io.prismic.
io.rover.
it.partytrack.sdk
jp.co.cyberagent
jp.co.cyberz
mediba.ad.sdk.android
net.gree
net.metaps
net.nativo.sdk
org.restlet.
ru.wapstart.plus1.sdk

[pnu-s]

Hi @IzzySoft and @jawz101

Currently most of our members are not very available due to the holiday seasons, but we'll try to take a look at this when we have some time :) Thanks to both of you for your very precious help on this

[pnu-s]

To give you an update, I just reviewed a bunch of the recent additions to ETIP and selected 10 good candidates to be imported into exodus. This will need a second review and then we'll push them there.

Then I plan to take a look at the remaining trackers in ETIP and your list @jawz101, but this all takes time :disappointed:

[IzzySoft]

Thanks @pnu-s!

[eighthave]

@rivkakarasik did you come across any of these in your recent search?

[jfoucry]

@jawz101 Hello, sorry to came like a hair on Yul Buner's head :-)

I will make a small script with all your entries and check how many times we found each of them in our dataset to have a better idea of good candidates.

Btw @pnu-s this script could be useful to "bathc" detection for new tracker. A list in a file and a loop to check each line… Did you think it make sens?

[jawz101]

@jfoucry thank you.

In light of your work, it would also be nice to have the ability to submit apps for scanning from the Android app. With approx. 80,000 in the Exodus library, the Exodus library represents about 2% of the apps on Google Play Store. It would be nice to have easier ways to scan apps besides keying in package names on the Exodus website.

option to submit an app on-device analysis

[pnu-s]

Btw @pnu-s this script could be useful to "bathc" detection for new tracker. A list in a file and a loop to check each line… Did you think it make sens?

I cannot agree more with you, that's why I already have that script written somewhere. I'll try to take some time to show you someday :)

@jawz101 I agree with you about the importance of such a feature (the first option being the easy one - at least for a first implementation). The issue here is that the only person from Exodus Privacy working on the Android app (and who wrote most of it) doesn't have any free time to work on it. So we have a blocking point to improve the app currently :/

[jfoucry]

@pnu-s is your script on the production machine? If yes, just send me a mail with the path, I will have look on it

[jawz101]

@pnu-s I've been in contact with @PerfectSlayer of the Adaway project about asking if someone involved in the Exodus Project might be able to take over maintenance of the Adaway list. Conversely, I wonder if he might be interested in pitching in on the Android app. And, coincidentally, he's French :P

And @Izzysoft if you'd ever be interested in participating in the Adaway blocklist you definitely have the understanding of the mobile landscape.

[pnu-s]

@jawz101 That's very cool, thanks (again) for everything you are doing! :+1: :1st_place_medal: