Time for a serious look at Proof of Work change

[shaolinfry]

I believe it's time to seriously review the proof of work algorithm used in Monero in light of the very serious consequences we have all witness with mining centralization in the Bitcoin community.

Mining centralization takes some obvious and non obvious forms. Miners can centralize the network simply by accumulating a majority of hashrate which may be easier to do when there is only specialized hardware from limited sources.

The second form of centralization is more insidious, which is also currently observed in the Bitcoin mining ecosystem where one company commands the monopoly on ASIC hardware supply. This is detrimental to decentralization because the company is able to exert economic force against both competitors and it's own customers. Because they are the major supplier, new players can be quickly starved out of the market through anti-competitive pricing designed to suffocate a new company. In the case of customers, since there is an unlimited demand for mining, and a scarcity of equipment, mining equipment customers can be coerced economically with threats of ceasing new sales. All these things encourage and enforce a cartel that is both difficult to see, and difficult to tame.

It is unclear if the Monero proof of work can be optimized by specialized hardware.

Clearly the best defense against mining hardware monopoly is to make it uneconomical or impossible for specialist hardware to be created for Monero. If the proof of work is only viable on commodity hardware, such as GPU, it's much harder for a manufacture to dominate because GPUs have a wide range of applications and thus plenty GPUs available in the world from a diverse set. CPU only algos are obviously problematic due to botnets from hacked computers. GPUs don't suffer this problem since GPU hijack would be very obvious on PCs.

In any case, I believe Monero is in a precarious situation and given the clear lessons from Bitcoin, we should take proactive action to ensure Monero does not become the center of a similar situation.

[othexmr]

Agree on the principle.

We have different choice In an advent of a malicous miner takeover or wants to enforce his rules on the community:

1) Modify CryptoNight slightly - it uses 4 different hashing algos, we could replace or modify them slightly so break unflexible hardware, this should be easy to deal with for our own CPU and GPU mining software.

2) Use sth like Cuckoo, which claims to be harder to do on HW, which is think is totally unproven. It also has the downside of being bad for pool mining, i am sure smaller miners would't like this.

3) Switch to something really asic friendly like Blake, Skein or Keccak (which we use already), lowering the costs for asic miners to join the game and letting the market deal with it.

4) Come up with system that makes ASICS harder to do, like using the blockchain as a scratchpad for calculations In all cases the community should decide that they want to support, UASF style, miners have only 1 job - to validate transactions and mine blocks.

The main issue with every change is, how do we switch while the blockchain keeps running? we could allow 2 PoW for a while and gradually reduce the amount of blocks the old one is allowed to mine.

Slightly unrelated but we should also work on a p2pool like system to counter miner centralization, not sure how to make this scaleable tho as the variance in ordinary p2pool systems is hard to cope with for smaller miners.

[fluffypony]

We've investigated this before, mostly around Cuckoo Cycle, and at some point it fell by the wayside. I support reopening this. I think that, at a minimum, we'd need to be able to provide:

• reasonable GPU mining kernels
• fast validation to prevent DoS risk
• use both as a mining PoW and as an on-handshake PoW

Some urgency might not be a bad idea, as the window in which we can make such broad and sweeping changes is narrowing.

[ghost]

What really matters is what we should call the legacy chain?

[CameronRuggles]

This might be a bit too radical/off topic but I think one issue that might be important to consider in PoW is the competitive exclusion principle: http://en.wikipedia.org/wiki/Competitive_exclusion_principle

"In ecology, the competitive exclusion principle... is a proposition... that two species competing for the same limiting resource cannot coexist at constant population values. When one species has even the slightest advantage over another, the one with the advantage will dominate in the long term. This leads either to the extinction of this competitor or to an evolutionary or behavioral shift toward a different ecological niche. The principle has been paraphrased in the maxim "complete competitors cannot coexist".

I think that miners are equal to different species that are direct competitors for the limiting resource which is the block reward. If the competitive exclusion principle is correct and applicable here, I think miner centralization might prove inevitable unless you can make it where there is effectively several different resources. This may still result in miner centralization, but likely only for one specific resource and not all.

If you imagine a coin that has multiple PoW algorithms: ones optimized for GPUs, CPU, ASIC1 type 1, ASIC type 2, etc it might result in a better distribution of miners: some would be botnets, some would be ASIC farms, and others consumer grade hardware.

Another thought might be to try to make mining linked closely with a volatile resource. If electricity prices fluctuated wildly among populations, perhaps that could prove to be useful. An electricity intensive mining algorithm optimized for consumer hardware and a low electricity consumption ASICs might be less likely be owned/controlled by the same entities.

[kenmasters21]

Brainstorming and may be "forking" the main practical issue in hand, im am INTP can't help it...

Do you think "tangle" type configuration (like IOTA) can be suitable and robust enough to fulfill the main function of Money- to be a storage of value that can be deferred through space/time? [https://www.youtube.com/watch?v=T2FJ9hH66b8]- being immutable, decentralized, widely connected enough, and attack proof?

A meshnet network of small devices that do their own validation and PoW simultaneously with a micro ASIC.

Decentralization could be forced as a function of being rewarded by being the farthest away from concentration of these micro meshnet ASICs, therefore also expanding network/signal coverage and harder to become fallible through centralization. Don't know how could this could be spoof-free though (GPS location can be spoofed in cellphones currently)

edit: PS: this micro ASIC should be so simple to produce that you could massively 3D print it if you wanted to anywhere in the world with the bare minimum materials.

[hyc]

Coincidentally, this paper came to my attention today - Proof of Work without all the Work https://arxiv.org/pdf/1708.01285.pdf

Worth reading.

[Makeone11]

One potential attack vector that opens up once you opt for "decentralization" is that the amount of hardware out there that can be put to use in attacking the network becomes a lot higher.

If you optimize for gpu(s) the investment cost of attacking the network isn't lost if monero goes down. If you compare that with dedicated ASICS the investment cost of attacking is lost if the network is harmed (either they find something new to mine using the same ASIC hardware or they're stuck with a brick).

[gerardomoscatelli]

Maybe this is a stupid idea but wouldn't a small reward paid for running a full node also prevent centralization? We run full nodes as a hobby and to verify blockchain's integrity but most people don't.

[JollyMort]

Cuckoo can work with pool mining if a cuckoo round is at low enough difficulty. Worth reading: https://github.com/ignopeverell/grin/blob/master/doc/pow/pow.md

[dEBRUYNE-1]

@gegemos That may cause centralization, because people will put their node on services that guarantee the most uptime in order to maximize their rewards.

[zachherbert]

Taking an opposite viewpoint, at Sia we've decided to embrace ASICs. Mining centralization is definitely an issue, but so is network security and making sure incentives are properly aligned across all users on the network.

As we wrote in our blog post (https://blog.sia.tech/choosing-asics-for-sia-b318505b5b51), GPU mining is a "false panacea that ultimately leaves a cryptocurrency far more vulnerable to attack."

One main issue is 51% attacks. On GPUs, that will always be a serious risk, as a few large Ethereum GPU pools could coordinate to attack Monero's network. Additionally, we don't know for sure how many millions of GPUs sit in the hands of private companies, research centers, and so on.

There is also a question of whether any algorithm is truly ASIC-resistant.

Another main issue is incentives – we believe it's more healthy if miners use ASICs that can only mine a single coin. That way, they can't simply switch back and forth on their GPUs to whatever coin is most profitable at the time, and are more invested in that coin's success.

At Sia, we decided to start a subsidiary called Obelisk to build the first ASIC miners for Siacoin and raise money via a presale. The idea is that, if we can widely distribute the initial hashrate, that should give a good foundation to keep away centralization. Of course we will have to keep making better chips, but we would continue to widely distribute them via presales and do our best to stay ahead of the competition.

This is of course going to be a huge decision for Monero, and I'd recommend that you observe how our Obelisk project at Sia goes over the next several months. We'll be shipping out ASIC miners by June 2018, so this group may be very interested in seeing how the switch goes.

[hyc]

I kinda like the idea of tying mining more closely to a node, requiring blockchain lookups to crunch the PoW. Really, why are miners separate from nodes in the first place?

Right now operating a node is a pure expense, totally uncompensated. It would make sense to get at least some kind of payment, periodically. Binding mining to nodes will raise the hardware requirements for mining, making it less feasible to integrate entirely in a single chip. It will also raise the CPU cost of running a node, making it too expensive to run 24/7 on the majority of cloud providers.

[fluffypony]

Moderator note: deleted long off-topic garbage post from anonymint.

[kim0]

I very much support bringing down the walls between "miners" and "users". All users should be miners and vice versa. I would like to suggest we consider biocryptics towards that goal. Basically, deriving keys from biometric information (nothing leaves your house though). The main benefit is that each human can run a single miner, not n miners where n is the number of cores they have access to. I do understand this approach can sound too wild, but let's give it a chance

For more details, please check out the Cicada whitepaper .. I'm also quoting perhaps the most interesting bits below.

The "Human Unique Identifier" (HUID) or "Single Identifier" or "Secure Identity Number" is an ID unique to each human on the planet. This allows us to prevent Sybil attacks and ensure everyone has a voice in the system. The HUID must be unique, incontrovertible, incorruptible, and not centrally stored or administered.

The HUID uses biometric markers to create public/private key pairs, guarded by passwords, which in turn are used to create IDs and linked sub-IDs. To be very clear, what we are talking about here is biocryptics NOT biometrics. Biocryptics are the intersection of biometrics and cryptography. The HUID is created by generating a public/private key pair from an iris scan and storing that ID in an identity blockchain, known as the HUID chain.

The HUID has the special advantage of being largely immune or at least incredibly resistant to Sybil attacks, because it is unique to each person and cannot be reused in the system. To create the public/private key pair, we scan for biometric markers known as "minutiae points." We identify these critical minutiae points to create a "template" of the points, convert them into a hash, and use them as seeds for a pseudo-random number generator to create a public/private key pair, along with salt to create additional randomness.

This public private/key pair will then be used to create an "info wallet." This is similar to a bitcoin wallet, but used for storing personally identifiable information (PII). The public key is run through a function to create a unique numerical ID for a person. The public/private key pair will include a built-in requirement to create a strong password, utilizing a rainbow table of prohibited weak passwords, enforced by the blockchain, to ensure good security practices.

[fluidvoice]

I wonder if it would be helpful to have a feature where all miners are not anonymous to the users/nodes such that a node may have a blacklist of miners (which would require miner identities I guess) to exclude one's Tx rewards going to "bad actors". This is a sort of voting power given to nodes.

[pierce403]

It is unclear if the Monero proof of work can be optimized by specialized hardware.

Is that the real issue? It seems to me that Monero is one of the few coins out there that is not just being mined with a GPU, but there are several people out there mining on CPUs, and they aren't even at a significant disadvantage. This is a very positive sign that the memory hardness of CryptoNight is working, and that ASICs would be uneconomical. Other poorly thought out PoW algorithms (I'm thinking of things like PrimeCoin, MemoryCoin, etc) got optimized to hell and left everyone else out in the dust.

Monero is old enough and high profile enough that if building ASICs was economical, I think we would have seen movement in that direction by now. It clearly isn't as asymmetric as Bitcoin where even 90nm ASICs could blow GPUs out of the water. Remember that SHA256 was designed to be super cheap in hardware, it was one of NIST's requirements for the SHA competition, and CryptoNight was very much designed to be expensive in hardware. Of course someone could design CryptoNight ASICs, but I doubt they would be able to compete with the economies of scale we get from commodity GPUs.

I'm a fan of algorithms like Yescrypt, Lyra2, Argon, etc, which have had a lot of work done on them in academia to show that they would be uneconomical to put into hardware, but weakening them to enable GPU mining would be really tricky and dangerous (look at the last couple Vertcoin PoW forks). I've been impressed with how well CryptoNight has held up to attempts to shortcut the algorithm, and I think that unless there is really solid evidence that someone magic'd up a way to do it super cheap in hardware, I think switching PoW would do more harm than good.

[shelby3]

Monero is old enough and high profile enough that if building ASICs was economical, I think we would have seen movement in that direction by now.

That is not necessarily the case if that someone who had the deep pockets required was preferring to keep it secret so it can run Monero as a Sybil attacked honeypot. How would we know? I posited the only way we can know is to have transaction revenue much greater than block reward and observe if the orphan rate skyrockets. If not, the mining is 51% controlled—otherwise inconclusive. I’m not advocating running such a potentially self-destructive experiment.

Of course someone could design CryptoNight ASICs, but I doubt they would be able to compete with the economies of scale we get from commodity GPUs.

Afaik, having the lowest hardware unit cost (due to high volume manufacturing) its amortization is largely irrelevant because it is a small component of the mining cost, which is dominated by electrical efficiency even though profitable lifespan is limited by Moore’s law. So the only economies-of-scale required is sufficient ROI on mining to payback the capital cost of the ASIC R&D and fab setup. This threshold may or may not have been met yet given Monero is not even a $billion market cap yet, and considering likely higher capital costs you allude to.

Note that GPUs can be re-purposed to continue their utility beyond their mining profitability lifespan.

[ghost]

No, don't fix what is not broken.

go back to Litecoin.

[QuickBASIC]

@zachherbert We don't know the makeup of those Ethereum pools, but aren't you assuming that the miners in those pools would want to be complicit in committing a 51% attack. How many of those pools are made up of individual miners as opposed to centralized datacenters controlled by a company or individual?

[shelby3]

Coincidentally, this paper came to my attention today - Proof of Work without all the Work https://arxiv.org/pdf/1708.01285.pdf

Actually I thought of that conceptually before when I was trying to devise a solution for the liveness-gets-stuck issue that I mentioned about Byteball, but didn’t bother to fully develop the model, because it has a very obvious and fatal flaw because they ostensibly didn’t model the economics of it. Their model is the provability that it can’t be gamed algorithmically. But afaics, they didn’t model the economic ramifications of their algorithm.

Their algorithm is essentially scaling the amount of PoW difficulty (that all mining node ID’s must have to survive a PoW challenge round) by the rate of changes to the ID set. So assuming there is no attacker, then everyone agrees to play nice then the difficulty remains low. But the specific flaw is its communism because it steals from those who have greater or low-cost hashrate and redistributes to the marginal miners, because every good or bad ID has the same weighted vote. Of course the same entity can create more than one ID to spread its hashrate, but this is attackable because if the threshold of their splits are exceeded by an attacker who issues too many ID joins/deletes per round, then the split IDs are deleted by the challenge round and amplify the attacker’s effect. So the economic implications are amplification instability else communism. We must understand the economics of decentralized consensus.

Also it appears to me that it requires some trusted setup on the initial randomness to create a non-gamed ID member set for the committee which acts as the “server”. There may be other issues, as this is brand new so peer review is presumably lacking.

[b-g-goodell]

Here's a quick pre-amble (3 brief points), then I'm going to comment on folks' individual responses. My entire point of view here is to look at decentralization and egalitarian mining as a matter of financial barriers to new users mining with a hashrate comparable to the current per-user hashrate, and a matter of long-term trends towards coalition mining. TLDR: lethos3 and pierce403, in my estimation, are correct.

Disclaimer: I am happy to be wrong, and I would rather be corrected than to continue disseminating false information. So ask questions and call me out on my mistakes.

1) High variance is bad. All other things being equal, a system with high variance of per-user hashrate isinherently more centralized than a system with low variance (in fact, this could be taken as a definition of centralization). The disparity between CPU/GPU/ASICs hashrates is orders of magnitude difference. Very few people would mine with a CPU at 1 H/s if they can mine with a GPU at 1 MH/s, and no one woul mine with a CPU at 1 H/s if the average per-user hashrate is 1 MH/s. Similarly, very few people would mine with a GPU at 1 MH/s if they can mine with a mining ASIC at 1 GH/s, especially if the average per-user hashrate is 1 GH/s. For an egalitarian mining process, every user should have approximately the same hashrate and therefore every user should have approximately the same equipment.

2) CPUs are egalitarian because many people already have them. We certainly can't provide every interested user in the world a big fancy ASIC mining rig, but every interested user has access to at least one CPU. In the interest of decentralization, then, is to determine a mining scheme that is not significantly easier using a GPU or an ASIC compared to a CPU.

3) CryptoNight forces low efficiency in all equipment. The CryptoNote inventors developed CryptoNight, which is a hashing algorithm designed to i) require large swaths of cache so that no GPU or ASIC can complete a computation without going through the CPU and ii) require so much cache so as to not fit in a computer's L1 or L2 cache, only fit in a computer's L3 cache. Accessing L3 costs more time than any other way of accessing memory. Importantly, this is a fundamental property of modern computer design, and until a fundamental redesign of computer architecture takes place, this cache will always be slow.

That is to say: CryptoNight was designed to force every user, GPU or ASIC or CPU, to utilize the slowest part of the computer in every hash computation. Moreover, all modern human computers will be vulnerable to this slowdown, and will continue being vulnerable to it until a major overhaul of computer architecture takes place. Take 1, 2, and 3 together and I don't see a rather elegant implementation of a decentralized mining process.

IOW, I don't anticipate a long-term trend toward mining cartels due to PoW with Monero unless computer architecture starts to change.

Now, in response to comments:

shaolinfry: "I believe it's time to seriously review the proof of work algorithm used in Monero in light of the very serious consequences we have all witness with mining centralization in the Bitcoin community." The centralization of Bitcoin is directly due to ASICs and the impossibility for a CPU or even GPU miner to break even.

"Miners can centralize the network simply by accumulating a majority of hashrate which may be easier to do when there is only specialized hardware from limited sources." In this case, the word 'simply' is tricky. Like saying "all we need for fusion is to simply get two protons very close together." Accumulating a majority of hashrate right now can only be accomplished by state actors or large botnets mining on equipment without the owner's consent. In the first case (state actors), I don't consider this a reasonable threat model for several reasons (I can elaborate). The second case, I think, is an inevitable consequence of egalitarian, decentralized mining. In fact, I think this is a value of decentralized computing: if many coalitions are capable of launching an attack but any one coalition needs more than 50% of the network to make such an attack, then any one coalition is less likely to succeed.

"The second form of centralization is more insidious, which is also currently observed in the Bitcoin mining ecosystem where one company commands the monopoly on ASIC hardware supply." Disregarding the monopoly-ness, the sheer existence of specialized hardware that costs more than a CPU and is orders of magnitude more efficient leads to centralization. Add on top of that the idea that AMD might be the ones truly in control of a mining network... yeah.

"It is unclear if the Monero proof of work can be optimized by specialized hardware." <--- Completely incorrect. See above.

"If the proof of work is only viable on commodity hardware, such as GPU, it's much harder for a manufacture to dominate because GPUs have a wide range of applications and thus plenty GPUs available in the world from a diverse set." CPUs are also commodity hardware. Moreover, the hierarchy of minability goes like this: anything that can be mined on an ASIC can be mined on a GPU, and anything that can be mined on a GPU can be mined on a CPU. It's not really possible to construct a mining system that operates on a GPU but not a CPU or an ASIC. If you a computation can be efficiently performed with a GPU, an ASIC can usually be designed to do the same thing but more efficiently. If we design a mining game that is mine-able on GPUs much more efficiently than CPUs, and if Monero then sees a price increase, then ASICs would be just around the corner.

"CPU only algos are obviously problematic due to botnets from hacked computers. " I think this is a natural and inevitable consequence of any egalitarian system that has no punishment associated with collusion. In particular, the problem with botnets is that user systems have been compromised, not that a large swarm of computers are validating transactions, or that a single entity is in control of the swarm. If we are concerned about a botnet controlled by a single entity coming in and rewriting our blockchain or selfishly mining, the solution is more competition between botnets, not less.

othexmr: "Modify CryptoNight slightly - it uses 4 different hashing algos, we could replace or modify them slightly so break unflexible hardware, this should be easy to deal with for our own CPU and GPU mining software." If we are concerned about the hashing algorithms being gamed into greater efficiency, you are correct that swapping them around would break inflexible hardware. However, the bottleneck for CryptoNote mining is not in the four hashing algos that contribute to CryptoNight, the bottleneck is in the usage of the L3 cache. So a user who cleverly designs something that can handle the four hashing algorithms is still sunk in the water because his computer architecture is what's slowing him down, not computation of many hashes.

"Switch to something really asic friendly like Blake, Skein or Keccak (which we use already), lowering the costs for asic miners to join the game and letting the market deal with it." If we want to put up a financial barrier to new users running a node while centralizing mining power with the richest of Monero miners, then sure.

"Come up with system that makes ASICS harder to do, like using the blockchain as a scratchpad for calculations" <--- I don't know how you could use the blockchain as a scratchpad for calculations. But I know that CryptoNight is already more ASIC resistant than almost any other option around (short of designing our own hash functions).

fluffypony: "I think that, at a minimum, we'd need to be able to provide: reasonable GPU mining kernels, fast validation to prevent DoS risk, use both as a mining PoW and as an on-handshake PoW." Providing a GPU miner that is markedly more efficient than the CPU is a centralization move. The short-term benefit would be more miners (for a time... until all the CPU miners are flushed out), but the long-term cost of this would be that no one will mine with CPUs anymore.

CameronRuggles: Your observation about competitive exclusion between species is important. There is a behavioral version describing competitive exclusion between behavioral strategies and individuals deploying different strategies. The idea is not that only one species remains, but only one behavioral strategy remains. In our case, each miner is an individual, the resource being fought over is a nonce that makes a block's hash sufficiently small, and the strategy is the equipment they employ while mining. The competitive exclusion principle, then, implies that only one strategy for finding nonces (i.e. type of mining equipment) will remain after a sufficiently long period of time.

I have some ideas of what it might look like to compete for many resources. As you say, a mining game using more than one cryptographic hash function could possibly work... but it would require some thought. I'm not convinced this would be the right approach, but I find the idea interesting.

Makeone11: "If you optimize for gpu(s) the investment cost of attacking the network isn't lost if monero goes down. If you compare that with dedicated ASICS the investment cost of attacking is lost if the network is harmed (either they find something new to mine using the same ASIC hardware or they're stuck with a brick)."

If a GPU can do it, an ASIC can do it better.

gegemos: "Maybe this is a stupid idea but wouldn't a small reward paid for running a full node also prevent centralization? We run full nodes as a hobby and to verify blockchain's integrity but most people don't." <--- I think I fully support providing slightly larger block rewards to full nodes. As hyc points out, running a full node is pure expense. If we can come up with a way to offset the cost without rewarding users with enough personal resources to run a full node, I would support that, but I'm not convinced it's possible.

Maybe we could have a a block reward bonus on top of the usual block reward, where the bonus is inversely proportional to the number of full nodes on the network. Fewer nodes -> bigger bonus -> more nodes -> lower bonus -> fewer nodes -> ...

zachherbert: "Taking an opposite viewpoint, at Sia we've decided to embrace ASICs. Mining centralization is definitely an issue, but so is network security and making sure incentives are properly aligned across all users on the network." This is a logical way to go, maybe... except it puts up a barrier to an arbitrary user participating in the network, which in the end leads to fewer non-colluding users participating in the network, which risks all that netsec you were hoping for. After all, fewer coalitions mining means any one coalition has an easier time hitting 51%. Your idea about incentives and miners using ASICS that can only mine a single coin does the same thing, by reducing the total number of participating users and hence reducing the total number of coalitions. Having said that, I can't see the future of an ASIC-based POW system that doesn't resemble the current problem in the bitcoin universe. That doesn't mean it can't exist, it means I have a failure of imagination.

kim0: "I would like to suggest we consider biocryptics towards that goal." Interesting idea. Identity-based encryption, where keys come from some arbitrary data like your e-mail address or a photograph of you (or a scan of your iris), has experienced some theoretical problems in the past and recently, where cryptanalysis is quite effective. Having said that, presuming we find a secure system... I'm not sure how you can verify that, say, an iris scan used to generate a pair of keys, actually came from a human instead of a random iris generator. If I can code up a piece of software that randomly generates realistic iris information, I can feed each randomly generated iris into the keygen and run as many bots as I like. As technology improves, we can also assume that arbitrarily realistic iris scans could be simulated by computers. In order to fix a problem like that, usually cryptographers introduce trusted third parties or certificate-based systems, where some authority determines if a real human is behind the iris. It's still an interesting idea.

pierce403, lethos3: Agreed.

shelby3: I am super happy to talk with you if you have a specific concrete proposal for POW. After all, in a high-txn-fee-with-respect-to-block-rewards environment, you are correct that PoW doesn't operate too well. I will also engage with you about your claimed honeypot situation if you identify all of your assumptions, fix the ones that are blatantly incorrect, and if you develop any verifiable concrete numbers on the complexity of solving the combinatorial problems associated with de-anonymizing a cryptonote blockchain. However, I will not engage with you if the conversation will resemble something like "Like it or not, you are going to use my solutions!!1 Checkmate, son!!112 Fluffypony is gestapo1l1khj." Your choice.

[iamsmooth]

@pierce403 can you elaborate on this:

look at the last couple Vertcoin PoW forks

for those of us who don't follow Vertcoin

What happened there and what did you learn from it?

[peronero]

Not sure how to take seriously any 'decentralize mining' proposal that would centralize mining in two US-based corporations subject to export regulations that already restrict the proliferation of high-end hardware along political lines. Keeping commodity CPUs and architectures such as ARM, POWER, and OpenRISC competitive with the most powerful chips is surely a better approach to 'decentralization'...

[fluffypony]

Removed another derailing post. Just a reminder to anyone passing by that this is a moderated issue on the research repo, and requires serious and/or academic responses. Known contributors preferred, but this repo is open and input from anyone is welcome. That said, derailing and nonsense will be moderated in order to keep the conversation on-topic.

[bigreddmachine]

Please use markdown when replying to people, especially in big blocks. Most notably, "quote" someone by beginning their quote on a new line with a leading "greater than sign" followed by a space. Otherwise this is super tedious to read.

(A bit off topic but in the interest of the discussion... @fluffypony feel free to delete if you think too spammy.)

[ghost]

Would it be possible to implement a cpu/gpu combination algorithm? That is to say tie it mostly to cpu but with the requirement that a certain amount of gpu power be involved in the calculations as well but only to a certain extent.

Could this not mitigate the botnets since most infected computers probably don't have a dedicated gpu and igpus are not that powerful to begin with anyways?

[hyc]

That would also eliminate a lot of non-PC devices (phones etc.) as they tend to have poor GPU driver support for OpenCL/generic compute.

[bigreddmachine]

@b-g-goodell, referring to @fluffypony's point and your response:

I think that, at a minimum, we'd need to be able to provide: reasonable GPU mining kernels, fast validation to prevent DoS risk, use both as a mining PoW and as an on-handshake PoW.

Providing a GPU miner that is markedly more efficient than the CPU is a centralization move. The short-term benefit would be more miners (for a time... until all the CPU miners are flushed out), but the long-term cost of this would be that no one will mine with CPUs anymore.

I believe the point he was making is that if we switch PoW and we switch to something that can still be mined with a GPU, we should make sure we have mining software that has been relatively optimized, otherwise someone can take advantage with private mining software and that defeats the purpose of the switch.

---

On the topic of CPU+GPU mineable algorithms, one thing I'd like to bring up here is that a consequence of the economics of Cryptonight mining being somewhat similar on both CPU and GPU is that this drives towards a state where GPU mining is actually not economical from an opportunity cost perspective, unless Monero's mining rewards are much greater than that of other coins that are only economical on GPUs. For example, today it might be profitable to mine ZEC, ETH, or XMR with a GPU. However, the fact that XMR is also profitable on CPU drives the reward per hash down relative to ZEC and ETH because CPUs can only mine XMR. As a result, anyone GPU mining XMR is incurring losses in terms of opportunity cost. If XMR rewards were much more valuable than everything else, this effect would lessen, as most hashrate would point to XMR regardless. But that's not the case today, nor likely in the near future, and therefore GPUs will tend towards those coins that are only GPU mineable.

[fluffypony]

I believe the point he was making is that if we switch PoW and we switch to something that can still be mined with a GPU, we should make sure we have mining software that has been relatively optimized, otherwise someone can take advantage with private mining software and that defeats the purpose of the switch.

Correct - anyone who was mining during the artforz time period (BTC and/or LTC's predecessors, Tenebrix and Fairbrix) will know how icky things get when a subset of miners have access to a GPU miner and the rest don't.

[mbarkhau]

I think Bram Cohen has an idea he calls "Proof of Space" which is worth pursuing. He hasn't published any details yet, but I have outlined my understanding of the idea here: https://gist.github.com/mbarkhau/00129f99e19cf28cbfb2cdf8c58c5f60

[bigreddmachine]

@hyc said:

kinda like the idea of tying mining more closely to a node, requiring blockchain lookups to crunch the PoW. Really, why are miners separate from nodes in the first place?

Right now operating a node is a pure expense, totally uncompensated. It would make sense to get at least some kind of payment, periodically. Binding mining to nodes will raise the hardware requirements for mining, making it less feasible to integrate entirely in a single chip. It will also raise the CPU cost of running a node, making it too expensive to run 24/7 on the majority of cloud providers.

I'm not sure I follow your point, can you elaborate/clarify? Are you arguing miners should be required to run a node, or that node operators should be compensated just for running a node?

Mining does require a node. Pooled mining allows individual workers to delegate who runs that node, and maybe that's not ideal (though maybe it is). I know you know that, given the extensive work you've done on the various mining softwares, which is part why I'm confused...

[hyc]

@bigreddmachine I was looking for a way to bind them together more tightly. E.g., so that if you're not running a node on the same box as your miner, the latency of talking to a remote node will significantly reduce your hashrate.

And yes, looking for ways to compensate people who just operate nodes. The simplest approach is to tell node operators to mine, and then it's a non-issue, they get compensated just like any miner.

[olarks]

I see no reason why Cryptonight PoW should be changed in the near future. I was an advocate for Cuckoo Cycle, but with recent performance improvements in Cuckoo Cycle the gap between CPU miners and GPU miners has widened by a considerable margin entirely assimilating CPUs, compromising PoW egalitarianism.

The arguments for an egalitarian PoW like Cryptonight stem around Cryptonight being very accessible to both CPU and GPU miners making it very decentralized, but prone to a 51% by a large coordinated attack from supercomputers and large botnets.

The arguments for ASIC friendly PoWs like SHA256 stem around reduced risk of 51% attacks because presumably the ASICs are not as accessible as GPUs or CPUs. However, ASICs present a large centralization risk for entities who are fortunate enough to have easy access to ASICs, typically China, pushing out all other miner competition with large warehouse mining operations. Ultimately, resulting in closed door meetings of miners as present in Bitcoin agreeing on segwit2x.

In my opinion having an egalitarian PoW like Cryptonight is far more valuable despite 51% attack risks because over time the the network hashrate will grow as seen in Monero in the past year growing from sub 20MH/s to over 100MH/s, minimizing 51% risks.

As long as miners are responsible and distribute their hashrate to smaller pools, then a centralization of hashrate in a single pool will never be present and centralization issues that are so common in ASIC friendly coins will not weigh down on Monero.

If Cryptonight ASICs are ever produced and are able to greatly increase efficiency over GPU and CPU mining breaking PoW egalitarianism then a new PoW should be seeked out.

[mbarkhau]

Maybe I'll just go over the motivation for the "Proof of Space" concept without getting into the implementation details.

In short, I think it is possible to use disk space as a resource for an alternative (or additional) proof of work system. Each node could dedicate 10-100GB of disk space to participate in the mining of new blocks and not require any specialized hardware. I think unused disk space is a much more distributed resource and I think it would contribute greatly to keep mining decentralized.

The devil is in the details of course and I won't claim to have thought everything through. But I think the core idea has merit and would be happy to go over it with somebody who has a better idea of what implementation issues there might be.

[olarks]

@shelby3 If you have nothing else to contribute to this discussion and are only going to continue to derail the discussion at hand in this github issue please post elsewhere.

[banastas2]

Can someone clearly define "mining centralization" -> Is it too much hash rate by geography or pool or ownership? Is it possible to design and algo that understands geography?

A good approach is to devise a strategy to attack the perceived/foreseen problem by creating a system that "forces" change of protocol under certain circumstances.

I like the idea of compensating nodes at a variable rate that increases as the concentration of mining (however defined) increases. Problem is intelligent coding that 'understands' centralization.

My 2 cents as a complete outsider/non-programmer... :)

[benkloester]

Peter Todd's distinction between "mining - the act of validating and constructing new blocks, and hashing - the act of solving proof-of-work problems." is useful to think about. Centralization/concentration of either presents risks, but there may be natural reasons for the latter to decentralize (eg heat dissipation), so the former is more likely to be a focus of centralization.

Making pools impossible probably prevents this but comes with its own trade-offs.

Paul Stztorc also has a decent discussion of this question.

When I think of miner centralization, I tend to think of what Paul calls "Managerial Miner Concentration". Ie the concentration of power with few decision-making agents.

[apertamono]

@mbarkhau Bram Cohen wasn't the first to invent Proof of Space. It's the algorithm used by Burstcoin. They call it Proof of Capacity. I think he wasn't aware of that. And it leads to hard-disk farms instead of GPU farms. You need multiple terabytes to be a competitive miner. That's not much of an improvement compared to GPU mining. A CPU-focused algorithm is better for decentralization.

[mbarkhau]

@ProkhorZ Thanks for the tip. I found this whitepaper which I will go over: https://eprint.iacr.org/2013/796.pdf

I'm not convinced that the centralization story is the same as for CPU/GPU/ASIC focused PoW methods. It seems to me that lots of people have idle disk capacity which they could put to use for mining at a marginal cost of practically zero. Maybe I'm missing something, but wouldn't it be hard to compete with that if you have to spend money to buy disks for your farm, when hundreds of thousands of computers are each contributing say 107GB (my current idle disk space). I also don't think the story is the same wrt. hardware specialization. The most cost efficient online storage technology (Ferromagnetic HDD) is the same that is already being used in consumer hardware and there are no specialized technologies such as ASIC which would give an orders of magnitude advantage to centralized miners.

[apertamono]

@mbarkhau Yes, I agree that Proof of Capacity is more decentralized than ASICs. But it's unlikely that you'd get 100 casual miners for every 10 TB miner. The UX would have to be very user-friendly.

@othexmr said:

We have different choice In an advent of a malicous miner takeover or wants to enforce his rules on the community

Since the discussion started with lessons learned from Bitcoin mining, let's take into account that we may not recognize an attacker before it's too late. For example, the Bitcoin community doesn't agree whether Bitcoin is being attacked or defended by Bitmain.

When I think through the strategy 'modify the algo just a bit to put an attacker on the wrong foot', the faster you change it, the better, so the optimal version is using multiple algos at the same time, the way DigiByte uses 5 competing algos separately.

@hyc Using the blockchain as a scratchpad is a fascinating idea. But wouldn't that make miners vulnerable to DDOS attacks?

As an altcoin historian with a limited understanding of cryptography, I don't see any obvious improvements to CryptoNight. I wouldn't want to change anything if I didn't know that Monero mining is extremely centralized already at the pool level: two pools control 51% of the hashrate at the moment.

[mbarkhau]

@ProkhorZ The marginal cost of 10TB across 100 casual miners is practically zero. Install the application so it boots at startup (maybe select a mining pool) and forget about it. The marginal cost of 10TB at todays prices for a centralized miner is ca 250 €.

[apertamono]

@mbarkhau Does it matter that competitors who can't scale up have zero cost? (Actually hidden/subsidized cost.) Vegetable gardens don't make industrial farming unprofitable.

[mbarkhau]

@ProkhorZ Vegetable gardens have a decidedly non-zero cost because people's time isn't free. Count the hours together you spend on the tomatoes in your garden and even at minimum wage they end up pretty expensive. The HDD based mining on the other hand is actually zero, neither hidden nor subsidized. At least that is the case if you only dedicate storage to mining, which you wouldn't have used otherwise anyway. The more significant cost is probably network traffic to participate in the mining.

[JollyMort]

HDDs are the bottleneck of modern PCs. Having some additional I/O load would kill PCs usefulness whereas you can easily throttle CPU/GPU mining use and multitask. I'd never use my HDD to mine. For those with SSD, it's better, but it would be of interest to see how many write operations mining would cause as it would hasten the SSD's inevitable death. Then again, it's expensive and I need my precious space for storing actual files. Again, would not SSD mine. I think the HDD mining concept is rather misguided, sorry to say.

[mbarkhau]

@JollyMort The I/O load for responding to a single challenge is a single seek operation (assuming the index of hashes is in memory) and reading maybe one kilobyte of data. The number of challenges corresponds to the number of blocks generated, so for monero one seek every two minutes. In other words, the I/O load is minimal.

The main I/O load is generated during (re)populating of the hash database, this might only be required every few weeks or so and then the CPU load would be more problematic than the disk I/O.

The idea is Proof of Space, not Proof of Throughput or Proof of IOPS.

[ghost]

It's simply too late now, people are welcome to launch an altmonero with a different PoW. I have not seen a rebuttal of the points brought forth by @b-g-goodell, there is nothing wrong with Cryptonight currently, and for the foreseeable future, as it makes sure the slowest parts of the machine need to process the hashes.

[moneromooo-monero]

there is nothing wrong with Cryptonight currently,

Cryptonight is slow to verify.

[ghost]

Worth note @shaolinfry is the same guy that out of nowhere created the UASF movement in Bitcoin which aims to bypass the nakamoto consensus that relies on miners and nodes on the same page and not on war against each other (a war that nodes lose because they don't actually secure the ledger) and opened a dangerous precedent of nodes unilaterally ''modifying'' a network, he started this over at Litecoin and after running it to the ground with Segwit abandoned the development there and appeared here, he clearly has a hidden agenda here and doesn't give 2 piconero to Monero.

Cryptonight is slow to verify.

and it works.

[kenmasters21]

I'm a nobody but I vouch for the creation of a new coin in which all problems that plague the current ones are overcome, and if be needed, create a new one again and again until the evolution of a system that maximizes freedom and the disposal of private property, boundless, and wealth preservation is perfected. Slaves no more to concentrated malicious powers, whether governments, miners, mafias or anybody.

[Gingeropolous]

@moneromooo-monero said

Cryptonight is slow to verify.

What is the transaction rate at which this will actually become a problem? This seems to me like a similar "problem" as the block size. As long as technology advances, so will the verification speed. Cryptonight does what a lot of other aspects of Monero do - tie a critical element of the network's function to an aspect of the physical world to void the need of human intervention. I don't fully understand the mechanisms of cryptonight, but from what I do understand, it exploits a fundamentally challenging aspect of computing, in general, to achieve its egalitarian nature - the communication between a CPU and memory. Basically, if a new technology is developed that drastically increases the speed between a processing unit and its memory, this technology will permeate every aspect of computing - consumer CPUs, server CPUs, GPUs, phone CPUs, toaster SOCs, you name it.

Thus, i don't think hashing centralization is a problem. However, I am concerned about mining (block formation) centralization. Pooled mining. At one end of the spectrum, I say we move to cuckoo cycle (once its been figured out to work as I hope it does). If its prevents pooled mining, great. The problem is that a large entity could just control a lot of CPUs, so in effect it could end up with a very unequal distribution of hashing and mining power. I.e., 5000 individuals with 1 hash/s each and one individual with 5000 h/s. And will those small miners continue mining without seeing any rewards?

@pierce403 said

Monero is old enough and high profile enough that if building ASICs was economical, I think we would have seen movement in that direction by now.

As mentioned elsewhere, its possible this could occur without us knowing. Though the adage "selling pick axes during a gold rush is more profitable than mining gold" applies here, so perhaps they don't exist.

In conclusion, I think cryptonight is currently adequate to inhibit ASIC development. However, I think a plan should be in place to be acted upon in the event of ASIC development, or perhaps a plan to just fork PoW every n years. In my mind, this has sort of been an unwritten social contract in Monero to date, but I think the deterrent nature of such a policy would add to the inhibition of ASIC development. If a potential ASIC developer knows that upon public release of their ASIC (or in n years) that Monero will shift PoW, they might not make an ASIC. And even if they did create an ASIC, they would have to adopt a some sort of "stealth" policy to avoid detection, which could be in the form of distributing their hashrate to different pools or slowly spinning up their hardware over multiple years.

I don't have a conclusion regarding pooled mining. We have yet to see how smart mining will play out, and we really don't have a means to actually measure its effectiveness (besides public pool tracking and inferring unknown hashrate to be solo miners). I think p2pool would be of great benefit to the monero network infrastructure and should be the focus of development and research. I also speculate that the mining infrastructure will change as the use of monero increases and the block reward decreases. It may be that, in the future, mining is not economical enough for professional miners, and instead private, solo miners may dominate the network. Part of the reason block rewards exist is to boot strap the network by distributing the currency. After this primary emission, the behavior of the network may be vastly different than it is today. Finding small blocks may be advantageous because they propagate the network faster, but a big pool will want to mine big blocks so they can pack them full of transactions.

Maybe a way to combat pooled mining is to find a way to increase the latency between a hasher and the pool. I think this was what @hyc was hinting at.