dubstard
commented
1 year ago Apologies for the cold mentions, just an idea i had. I know how to do it on my machine, but not via js and i think it would be neat to have this baked right in.
cat phishfort_whiteliost.txt | sort -u
cat raw_URL_list_for_blocking.txt
cat raw_URL_list_for_blocking.txt| grep -vwE '^.*\b(1in.ch|1inch.exchange|1inch.io|aave.com|acala.network|airswap.io|algoexplorer.io|ampleforth.org|aurora.tari.com|axieinfinity.com|badger.finance|balancer.exchange|bancor.network|bflgroup.ae|binance-cn.com|binance.charity|binance.co|binance.co.ug|binance.com|binance.je|binance.org|binance.us|binance.vision|binance.zendesk.com|binancecn.net|binancelite.com|binancezh.co|binancezh.com|binancezh.net|bisq.network|bit-z.com|bitcoin.co.th|bitfinex.com|bitforex.com|bitkub.com|bitkubnext.com|bitkubnext.exchange|bitmex.com|bitso.com|bittrex.com|blockchain.com|bolha.com|bx.in.th|cex.io|chain.link|chainlink.com|changelly.com|changenow.io|coinbase.com|coindash.io|coindesk.com|coingecko.com|coinmarketcap.com|compound.finance|consensys.net|cryptocompare.com|cryptokitties.co|cscs.ng|curve.fi|cyanre.co.za|decentral.ca|defipulse.com|degentrilogy.com|dether.io|dex.ag|dfinity.org|dharma.io|digifinex.com|district0x.io|duneanalytics.com|dydx.exchange|eharmony.com|eos.io|eth.link|etherdelta.com|etherealsummit.com|ethereum.org|etherscan.io|ethfinex.com|ethgasstation.info|ethglobal.co|ethpool.org|exchange.idex.io|exodus-dev.io|exodus-prod.io|exodus-services.io|exodus-stage.io|exodus.com|exodus.io|exodus.netlify.com|exodus.workers.dev|f2pool.com|facebook.com|fantom.network|fkc.bank|fortmatic.com|ftx.blue|ftx.com|ftx.cool|ftx.digital|ftx.io|ftx.page|ftx.soy|ftx.tech|ftx.us|gate.io|gemini.com|gilded.finance|gitcoin.co|github.com|github.io|gmail.com|gnosis.pm|golem.network|google.com|hblabs.com.ng|hbng.biz|hbng.com|hbng.org|hbnggroup.biz|hbnggroup.com|hbngonline.com|hbnng.com.ng|helpag.com|hitbtc.com|hk.ftx.tech|holaplex.com|idax.pro|idex.io|idex.market|jaxx.io|karura.info|katana.roninchain.com|kattana.io|kia.com|klar.mx|kraken.com|kusama.network|kyber.network|kyberswap.com|lbank.info|ledger.com|liquid.com|localbitcoins.com|localbitcoins.net|localcryptos.com|localethereum.com|looksrare.org|luno.com|magicalcryptoconference.com|makerdao.com|matcha.xyz|matic.network|medium.com|metamask.io|mooniswap.exchange|mstable.org|multis.co|myalgo.com|mycrypto.com|myetherwallet.com|mymonero.com|netlifyglobalcdn.com|nett7.com|nexo.io|nexusmutual.io|numer.ai|nuo.network|oasis.app|octopus.ng|oex.com|okex.com|opensea.io|oppo.ae|oppo.bg|oppo.bh|oppo.co.ke|oppo.com.bd|oppo.com.eg|oppo.com.hk|oppo.com.kz|oppo.com.lk|oppo.com.mm|oppo.com.ng|oppo.com.ph|oppo.com.ro|oppo.com.uz|oppo.hk|oppo.hk.com|oppo.hu|oppo.id|oppo.lk|oppo.lol|oppo.ma|oppo.ph|oppo.pk|oppo.qa|oppo.sa.com|oppo.sg|oppo.tm|oppo.za.com|oppopad.com|originprotocol.com|pancakeswap.ai|pancakeswap.blog|pancakeswap.com|pancakeswap.cz|pancakeswap.finance|pancakeswap.info|pancakeswap.love|paradex.io|parity.io|paxful.com|paxful.com.cn|paypal.com|phantom.app|phishfort.com|polkadot-statue.io|polkadot.com|polkadot.io|polkadot.network|poloniex.com|post.at|radarrelay.com|raydium.io|recordedfuture.com|reddit.com|ripple.com|saga.co.uk|shapeshift.com|shapeshift.io|skymavis.com|solana.com|solana.foundation|solanahackerhouse.com|solanamonthly.com|solanaweekly.com|sphere.finance|spherefinance.store|spherefinance.xyz|stakedao.org|staratlas.com|substrate.dev|substrate.io|sushiswap.org|sushiswapclassic.org|switch.ag|switcheo.network|sybil.org|synthetix.exchange|synthetix.io|tari.com|techdata.com|tenx.tech|thehashmasks.com|thepanomirror.com|tokenlists.info|tokenlists.org|tornado.cash|totalcoin.io|trezor.io|trubi.io|trustwallet.com|tymebank.co.za|tymedigital.co.za|tymedigital.com|unipig.exchange|unisocks.exchange|uniswap.exchange|uniswap.finance|uniswap.info|uniswap.io|uniswap.org|uniswap.pink|upbit.com|uphold.com|usscyber.com|valr.com|wallet.mymonero.com|wallet.roninchain.com|wallet.trezor.io|walletconnect.com|walletconnect.org|walletsrecovery.org|web3.foundation|workers.dev|wsce.world|www.hbng.com|xliquidus.com|xmr.to|y.at|yam.finance|yearn.finance|yellowcard.io|ygov.finance|zapper.fi|zb.com|zerion.io)\b.*$' >Filtered_URL_list_for_blocking.txt
#Filtered_URL_list_for_blocking.txt should now contain only domains which are not listed aboveI tested this "PoC" with two URLs
#!/usr/bin/env node
require('shelljs/global');
cat('phishfort_whiteliost.txt').sort('-u');
cat('raw_URL_list_for_blocking.txt');
cat('raw_URL_list_for_blocking.txt').grep('-vwE', '^.*\b(1in.ch|1inch.exchange|1inch.io|aave.com|acala.network|airswap.io|algoexplorer.io|ampleforth.org|aurora.tari.com|axieinfinity.com|badger.finance|balancer.exchange|bancor.network|bflgroup.ae|binance-cn.com|binance.charity|binance.co|binance.co.ug|binance.com|binance.je|binance.org|binance.us|binance.vision|binance.zendesk.com|binancecn.net|binancelite.com|binancezh.co|binancezh.com|binancezh.net|bisq.network|bit-z.com|bitcoin.co.th|bitfinex.com|bitforex.com|bitkub.com|bitkubnext.com|bitkubnext.exchange|bitmex.com|bitso.com|bittrex.com|blockchain.com|bolha.com|bx.in.th|cex.io|chain.link|chainlink.com|changelly.com|changenow.io|coinbase.com|coindash.io|coindesk.com|coingecko.com|coinmarketcap.com|compound.finance|consensys.net|cryptocompare.com|cryptokitties.co|cscs.ng|curve.fi|cyanre.co.za|decentral.ca|defipulse.com|degentrilogy.com|dether.io|dex.ag|dfinity.org|dharma.io|digifinex.com|district0x.io|duneanalytics.com|dydx.exchange|eharmony.com|eos.io|eth.link|etherdelta.com|etherealsummit.com|ethereum.org|etherscan.io|ethfinex.com|ethgasstation.info|ethglobal.co|ethpool.org|exchange.idex.io|exodus-dev.io|exodus-prod.io|exodus-services.io|exodus-stage.io|exodus.com|exodus.io|exodus.netlify.com|exodus.workers.dev|f2pool.com|facebook.com|fantom.network|fkc.bank|fortmatic.com|ftx.blue|ftx.com|ftx.cool|ftx.digital|ftx.io|ftx.page|ftx.soy|ftx.tech|ftx.us|gate.io|gemini.com|gilded.finance|gitcoin.co|github.com|github.io|gmail.com|gnosis.pm|golem.network|google.com|hblabs.com.ng|hbng.biz|hbng.com|hbng.org|hbnggroup.biz|hbnggroup.com|hbngonline.com|hbnng.com.ng|helpag.com|hitbtc.com|hk.ftx.tech|holaplex.com|idax.pro|idex.io|idex.market|jaxx.io|karura.info|katana.roninchain.com|kattana.io|kia.com|klar.mx|kraken.com|kusama.network|kyber.network|kyberswap.com|lbank.info|ledger.com|liquid.com|localbitcoins.com|localbitcoins.net|localcryptos.com|localethereum.com|looksrare.org|luno.com|magicalcryptoconference.com|makerdao.com|matcha.xyz|matic.network|medium.com|metamask.io|mooniswap.exchange|mstable.org|multis.co|myalgo.com|mycrypto.com|myetherwallet.com|mymonero.com|netlifyglobalcdn.com|nett7.com|nexo.io|nexusmutual.io|numer.ai|nuo.network|oasis.app|octopus.ng|oex.com|okex.com|opensea.io|oppo.ae|oppo.bg|oppo.bh|oppo.co.ke|oppo.com.bd|oppo.com.eg|oppo.com.hk|oppo.com.kz|oppo.com.lk|oppo.com.mm|oppo.com.ng|oppo.com.ph|oppo.com.ro|oppo.com.uz|oppo.hk|oppo.hk.com|oppo.hu|oppo.id|oppo.lk|oppo.lol|oppo.ma|oppo.ph|oppo.pk|oppo.qa|oppo.sa.com|oppo.sg|oppo.tm|oppo.za.com|oppopad.com|originprotocol.com|pancakeswap.ai|pancakeswap.blog|pancakeswap.com|pancakeswap.cz|pancakeswap.finance|pancakeswap.info|pancakeswap.love|paradex.io|parity.io|paxful.com|paxful.com.cn|paypal.com|phantom.app|phishfort.com|polkadot-statue.io|polkadot.com|polkadot.io|polkadot.network|poloniex.com|post.at|radarrelay.com|raydium.io|recordedfuture.com|reddit.com|ripple.com|saga.co.uk|shapeshift.com|shapeshift.io|skymavis.com|solana.com|solana.foundation|solanahackerhouse.com|solanamonthly.com|solanaweekly.com|sphere.finance|spherefinance.store|spherefinance.xyz|stakedao.org|staratlas.com|substrate.dev|substrate.io|sushiswap.org|sushiswapclassic.org|switch.ag|switcheo.network|sybil.org|synthetix.exchange|synthetix.io|tari.com|techdata.com|tenx.tech|thehashmasks.com|thepanomirror.com|tokenlists.info|tokenlists.org|tornado.cash|totalcoin.io|trezor.io|trubi.io|trustwallet.com|tymebank.co.za|tymedigital.co.za|tymedigital.com|unipig.exchange|unisocks.exchange|uniswap.exchange|uniswap.finance|uniswap.info|uniswap.io|uniswap.org|uniswap.pink|upbit.com|uphold.com|usscyber.com|valr.com|wallet.mymonero.com|wallet.roninchain.com|wallet.trezor.io|walletconnect.com|walletconnect.org|walletsrecovery.org|web3.foundation|workers.dev|wsce.world|www.hbng.com|xliquidus.com|xmr.to|y.at|yam.finance|yearn.finance|yellowcard.io|ygov.finance|zapper.fi|zb.com|zerion.io)\b.*$').to('Filtered_URL_list_for_blocking.txt');
//Filtered_URL_list_for_blocking.txt should now contain only domains which are not listed above
409H
Hi,
When I accidentally block a good site I feel horrible! So an idea came to me:
Not sure if this is even possible. I would like to suggest the following extra check in the CI/CD automated pipeline, that parses and checks each pull request for known trusted sites in the newly proposed contents in the
blacklistarray and would subsequently prevent them from getting blocked, even if they are not whitelisted per se in MetaMask's own whitelist!As many organizations have a bunch of legitimate domains, apart from their main one, it is extremely time consuming to check each one individually, in order not to accidentally block a legitimate resource.
For example:
1inch
For 1inch the main URL is 1inch.io But those are also not harmful:
As they are registered by the 1inch team to prevent cyber squatters from getting them.
Pancake
For Pancake swap the main URL is
pancakeswap.financeAll those are legit! Same goes for
pancakeswap-lotterystats.info- again not an official resource by the pancake team, but not harmful in any way, community developed and driven.We communicate with some of the organizations DEXs, DAOs and etc and I can check with them directly before blocking somehting: I also refer to the projects official page etc, but sometimes there are fan made pages which are also perfectly legitimate
Uniswap
Another example -
Uniswap.fishis not official, but is not a scam!So I would like (if possible) to use this individually vetted whitelist maintained by PhishFort (where Metamask is a customer AFAIK).
cc @409H @legobeat @danfinlay