Plan how to change the proof-of-work

[str4d]

At some point in the future, we may have to change the Proof-of-Work in order to maintain our desired mining properties and goals. We should have a plan for how to do this.

[str4d]

Summary of discussion between myself and @daira:

Any PoW change would require a transition period, during which both PoWs are accepted on the network. Given that the block header would also need to be changed, the easiest way would probably be to increment the block header version, and then have a period in which both block headers are accepted.

To ensure a smooth transition, we'd probably want to have a ramp-up (similar to mining slow start) where the minimum percentage of blocks that must be new PoW (enforced by changes to the difficulty algorithms) increases over a few months to, say, 30-40% (high enough to encourage migration, but not enough to risk 51% attacks). Beyond that, the ratio of blocks would be allowed to float, and then once the new PoW passes, say, 95% of blocks, we would drop the old PoW (adding a checkpoint for the change into newer versions of Zcash).

We could safely replace the old PoW with the new PoW in the zcashd miner at the start of the transition period. If we get to the point of needing to change the current PoW, it's highly likely that the miner in zcashd is being used very infrequently. Migrating the zcashd miner would also provide an easy entry-point to the new PoW.

[str4d]

This thread has a similar proposal for changing a PoW via a sunset period, although it specifies a definite end to the window instead of letting it float.

[eldentyrell]

This is fun to think about, but it is unwise to assume that a mechanism like this will work when needed. I can only think of one nontrivial altcoin that completed a PoW change (vertcoin), and its market cap crashed immediately afterwards, never to recover.

So you probably need to assume that you have to get the PoW right the first time.

For starters, all the sound+sensible forking mechanisms I've seen involve a phase where miners advertise support for a new proposal. When enough of the hashpower advertises support, the change takes effect and noncompliant blocks get orphaned. This is needed to minimize the otherwise-large risk of a chain split on the first block where the change takes effect. In this case even with a smooth "ramp-up" the change from 0% of blocks using new-PoW to 0.00001% using new-PoW is a forking change, so the first block of the ramp would require a supportadvertisement-commit process regardless of how slow the ramp is (unless you're proposing something different than a supportadvertisement-commit process, in which case please explain). If you're planning on using a PoW change to dislodge a group of highly-invested industrial-scale miners, they would be unlikely to vote against their own interests.

I can pick other nits, but that's the first that comes to mind.

[daira]

The thing that needs to be committed to before the first new-PoW block is that almost everyone can verify the new PoW. The purpose of the ramp is that at that point, not all miners will be able to mine blocks with the new PoW (those using the core software will, but there will be independent mining implementations), so an immediate switch could indeed cause a collapse in hash rate.

Note that it's possible to indicate support for verifying the new PoW by a soft-fork: suppose we have evidence that most of the network already supports verifying the new PoW. At that point we can introduce a type of block indicating support that is a non-standard block according to the old rules -- so it will be accepted in the chain without any hard-forking risk -- but if the longest chain includes that block then we can be confident that most miners were able to verify it. Note that even if there's an entrenched coalition of miners that would vote against the switch, there's little they can do to stop it (in particular, every individual miner in that group has an incentive to defect from the coalition in order to obtain any mining reward in the short term, if they are able to perform the switch to verifying the new PoW at all, despite it reducing their long-term profit).

[eldentyrell]

The thing that needs to be committed to ... is that almost everyone can verify the new PoW.

... and also the exact block height at which to:

• accept the first block which uses newPoW (i.e. not orphan it)
• orphan any otherwise-valid block simply because it used oldPoW

It's a synchronization issue.

If there's a ramp, you still have to synchronize the first block of the ramp, no matter how slow it is.

every individual miner in that group has an incentive to defect from the coalition

It seems like we have different threat models in mind here.

str4d opened this issue in order to "change the Proof-of-Work in order to maintain our desired mining properties and goals", mainly ASIC-resistance. In that scenario, the "group" consists of those miners who are using oldPoW-specific ASICs. They cannot defect since their ASICs do not mine newPoW (and if they did, there would be no point in attempting the change!).

If by "defecting" you're suggesting that oldPoW-only miners have nothing to lose by mining an oldPoW block on top of the very first-ever newPoW block as part of a slow ramp, I cannot agree with your analysis. If a large portion of the hashpower stands to lose immense sunk costs they will be ready to orphan not just one-block-long forks but two-block-long forks as well. If miners with sunk costs manage to produce three blocks then the defector's block is orphaned and they lose their block reward. The defector takes a very real risk. Perhaps that risk is worth it, but it's not as simple as risk-free defection -- if it were then I'm sure bitcoin would have replaced SHA256 by now.

suppose we have evidence that most of the network already supports verifying the new PoW

By "the network", I assume you mean most nodes on the network or most users of zcash, rather than most of the hashpower, right? In such a case, where most of the hashpower is hostile to the change due to sunk costs, what form would this evidence take? Miners will reject attempts to inject it into blocks, unless you take steps to conceal it. That can of course be done, but the details need to be spelled out -- they may introduce further issues and attacks. Like most of the other issues surrounding this question, I'm sure this can be solved. But make no mistake, this is a novel and untested scheme (which is exciting!). It's great to include novel and exciting new ideas in zcash, but if the PoW-switching technology is novel and untested (as all of them are) then it isn't really a fallback for mistakes in PoW design. So by all means, add the cool new feature, but don't think it will satisfy the need that spawned the search for it.

TL;DR: every ASIC-thwarting-PoW-switching scheme I know of is much more unreliable than equihash. That doesn't mean you shouldn't include them or try to improve them. It means they aren't "insurance" against flaws in zcash's PoW.

[daira]

@eldentyrell wrote:

... and also the exact block height at which to:

• accept the first block which uses newPoW (i.e. not orphan it)

Yes, kind of. As for any other fork, there is an activation condition for this. The exact block height at which it activates does not have to be known in advance, as long as it is unambiguous.

Define a "signalling block" to be a block containing a specific signal that is non-standard (but valid) according to the old rules. Concretely, let's suppose that the new-PoW acceptance activation occurs N blocks after the first signalling block. (More elaborate conditions are possible, but the analysis below assumes this one.)

• orphan any otherwise-valid block simply because it used oldPoW

The important thing is that the difficulty of the old PoW ramps up until it becomes uneconomic. It's not strictly necessary to have a hard cut-off for the old PoW; the complexity cost of keeping the ability to verify it is small (and it's desirable to retain support for verifying the whole chain history).

str4d opened this issue in order to "change the Proof-of-Work in order to maintain our desired mining properties and goals", mainly ASIC-resistance. In that scenario, the "group" consists of those miners who are using old PoW-specific ASICs. They cannot defect since their ASICs do not mine newPoW (and if they did, there would be no point in attempting the change!).

I think you've misunderstood the proposal, or possibly I didn't explain it well. They absolutely can defect.

The problem we're trying to solve is that we need to achieve a supermajority of miners able to verify (not mine) the new PoW, before allowing it in the blockchain. An ASIC miner is certainly technically capable of verifying the new PoW; that is cheap and can be done in software. The difficulty is that it's not sufficient to handwave that a supermajority of miners could do so; we have to be able to see direct evidence of that supermajority in the blockchain. And without some other economic factor, ASIC miners have a strong incentive to pretend that they can't do this verification, in order to block the change and protect their mining profits.

The proposal circumvents this by performing a soft fork to a situation in which the ASIC miners will lose short-term revenue (on the old PoW) unless their blocks commit to their having the ability to verify the new PoW. After the soft fork, a majority will be mining previously-nonstandard signalling blocks that make this commitment, and so the longest chain will include such a block. Miners that follow the old rules cannot mine on the longest chain, and so they will forfeit their mining revenue. To obtain that revenue, a miner must mine on top of signalling blocks. This is evidence that they must have been upgraded, from which we infer (whether they like it or not) that they can verify the new PoW.

(It is of course possible that they upgraded just to accept the signalling blocks and not to verify the new PoW. In that case they're intentionally not following the protocol. Nothing actually forces miners to perform PoW verification anyway, and some Bitcoin miners don't. The security of PoW-based consensus does not depend on all miners performing PoW verification. However, it's important that large-scale mining pools do so as demonstrated by the 2015 BIP66 block chain fork. I personally hope that we support P2Pool or similar decentralized pool protocols quickly, to make such problems less likely.)

If by "defecting" you're suggesting that oldPoW-only miners have nothing to lose by mining an oldPoW block on top of the very first-ever newPoW block as part of a slow ramp, I cannot agree with your analysis. If a large portion of the hashpower stands to lose immense sunk costs they will be ready to orphan not just one-block-long forks but two-block-long forks as well. If miners with sunk costs manage to produce three blocks then the defector's block is orphaned and they lose their block reward. The defector takes a very real risk. Perhaps that risk is worth it, but it's not as simple as risk-free defection -- if it were then I'm sure bitcoin would have replaced SHA256 by now.

This is mistaken: Miners in the coalition will refuse to mine on top of signalling blocks. But as long as the coalition, plus miners stuck on old software versions, control less than half of the mining power, they cannot win. In order to prevent the activation condition for accepting the new PoW, they have to ensure that the longest chain contains no signalling blocks. The coalition and the old-version miners will mine their own orphan chain branching from just before the first signalling block, but if they fall behind the rest of the network, they will lose all mining revenues on that chain.

Now look at this from the point of view of a defector. If the longest chain is significantly ahead of the coalition chain at any point, then the defector's risk is minimized by building on the longest chain. The coalition is unlikely to attempt to orphan blocks after the first signalling block on the longest chain, because that does not achieve their goal, and any mining power expended in doing so cannot contribute to building the coalition chain. Therefore, the defector may assume that the risk of that happening is not much greater than in a normal mining situation.

It's great to include novel and exciting new ideas in zcash, but if the PoW-switching technology is novel and untested (as all of them are) then it isn't really a fallback for mistakes in PoW design.

Absolutely not! I'm not sure how what I said came across otherwise. We need to get Equihash right.

[eldentyrell]

control less than half of the mining power

@daira, it sounds like we have different scenarios in mind. You're assuming 51% of the old-PoW hashpower supports the switch. I agree that a switch will work under those circumstances. I also don't think that scenario matters.

By the time you figure out that somebody's built a working equihash ASIC, they and their customers will most certainly control far more than 51% of the hashpower. Aside: I think it's a little misleading to call the customers of a chip vendor a "coalition" -- there's no requirement for them to be in communication with each other in any way.

If you're not trying to handle that situation then I think we're on the same page here.

Also, handling this specific situation (where 51% of the hashpower supports your change) requires no advance pre-genesis-block planning. So I don't think there's any need to plan ahead for a 51%-supported future change at this point; if you've got that level of support then there aren't any technological barriers... just don't do anything stupid like Ethereum did.

and some Bitcoin miners don't....2015 BIP66 block chain fork

Actually this was the result of miners failing to verify well-formedness of the blocks they were building on top of (i.e. all the other rules except for "has enough PoW"). They were still checking the PoW, since failing to do so opens them up to a simple DoS attack that is very lucrative for their competitors.

Edit: in fact in bitcoin you can't build on top of a block without [doing 99% of the work involved in] checking its PoW. The blocks are identified by their SHA-256 hash, and checking the PoW is just counting the number of leading zeroes in the same hash. I've been focused on cryptocurrencies with separate identification/PoW hashes for so long that this now seems weird...

[tromp]

Have you considered launching with multi-PoW support as in Myriadcoin ? Then changing or deprecating particular PoWs is just a matter of inflating its difficulty, which could be implemented as either a soft or hard fork. Note that each PoW maintains its own difficulty in Myriadcoin.

[eldentyrell]

Have you considered launching with multi-PoW support as in Myriadcoin ? Then changing or deprecating particular PoWs is just a matter of inflating its difficulty, which could be implemented as either a soft or hard fork.

Neat idea.

Just to clarify for anybody else reading (I'm sure @tromp already knows this), the deprecation/inflation soft fork still requires 51% hashpower support, so it still doesn't help with the situation where you wake up one morning to discover that an ASIC exists.

However using multiple algorithms might postpone that unhappy morning or give advance warning of it (e.g. if the distribution of hashpower across the algorithms suddenly skews strongly). Then again it might not; there are already 130nm X11 ASICs available for purchase that implement 11 different hashing algorithms. It depends on a lot of factors and details.

[tromp]

Thanks, eldentyrell, for pointing out that 51% of the blocks is needed in a PoW deprecating softfork. But note that with 3 different PoWs, the independent dynamic difficulty adjustments will ensure that each gets roughly 1/3 of the blocks, even if you increase hashpower of one tenfold. So if miners of the other 2 PoWs support the soft-fork, they can outvote the third.

[str4d]

@tromp @eldentyrell See #129.

[tromp]

You may find it "too complicated" (which I disagree with), but it certainly simplifies any future needed PoW changes.

[daira]

It is too complicated unless we find that we need it, and in any case that option was rejected months ago (early April), which is when it would need to have been decided on to get into 1.0.

[Edit: it was actually rejected earlier than I thought, in June 2015.]

As I wrote here:

Multi-algorithm likely doesn't help with centralization, since large/centralized miners will mine whichever algorithm is most favourable to them at any given point. It may be slightly easier to add another algorithm, but that can be done anyway starting with a single algorithm.

[str4d]

We are now planning several hard forks (HF0 followed by the Sapling upgrade), and will therefore be changing consensus rules, as well as potentially data formats (e.g. #2071). Thus changing or extending the PoW can be done either in concert with another change, or during its own hard fork (via whatever mechanism we introduce with HF0).

Mining pools and solo miners, as operators of Zcash full nodes, will already need to migrate to new software that supports the new consensus rules, and in doing so can add support for a new PoW. For them, a hard switch vs a ramp-up mechanism won't make much difference. The miners that use mining pools, OTOH would be best notified by their mining pools. A ramp-up might be more preferable for them, but with sufficient lead time and the presence of miners with both algorithms, a hard switch could also work.

The Stratum ZIP specifies that the block version is a switch for subsequent parameters, and existing implementations MUST ignore jobs with versions other than 4. So we can trivially use a different block version (or some more complex parsing method involving treating it as a bitfield) to support the new PoW over the same Stratum connection as the current one. Whether or not we also use a new different field for indicating PoW type would depend on how we implement hard forks (e.g. #2255, #2286, #2308).

[bitcartel]

For reference, the PoW change merged for BIP 148 UASF hard fork: https://github.com/BitcoinHardfork/bitcoin/pull/1

[luke-jr]

The correct PR is BitcoinHardfork/bitcoin#5 (it's not merged yet), and it has nothing to do with BIP148 or UASF...

[mreid2005]

Sorry, for the mid-west accent crawling in, but Y'all realize that the Major Miners, that can afford to hire dedicated developers to "solve" the ASIC resistance through FPGA design emulation, can then afford to contract what some in the manufacturing industry call a short-run of ASICS that they never sell to the public.. and no reason for anyone to be the wiser.

Well, some will assume and point fingers etc. and accuse some group or another, but unless someone actually touches said ASIC. it can only presume to exist.

However, if the getwork, and submitwork code requires signing not only by the application that requests the work but the underlying device that submits a possible solution/partial solution, along with some unique characteristic that would be generated by the device performing the proof of work. for ex: the plug_in_play vendor_id that was captured when a device is initialized by the host os. This is somewhat immutable, and it will stand up to variance analysis. if someone builds an ASIC, but spoofs the vendor_id of a gtx 1070. the POW submission rate will stand out from the median of all vendor_id's matching some subset of a gtx1070 gpu. In which case identifying the existence of a "private" ASIC becomes possible.

bear with my disjointed thinking, but it's early morning, my kids are sick and so is the wife. but I've been pondering this very question myself. How to prove a thing exists, despite the lack of evidence that such a thing may exist.

Figure out a way, to validate the endpoint not the aggregator.

Furthermore to facilitate continued decentralization:

Hire a true blue PR/Social Networking firm with some of that 20% dev fee, and make Zcash fall off of everyone's lips as easily as Bitcoin does and you won't have to worry about centralization for at least 2 or 3 more years. Because everyone and their brother will point some hashing power at it.

Here's a straightforward marketing gimmick someone should be able to pull off with a little video skills and acting ability.

STEP 1: Develop a Professional GUI Interface built into the wallet that interfaces with the top 2 or 3 ZEC miners agnostically. and have the top 10 mining pools ready to choose from through a drop down.
STEP 2: Create a one, two , three how easy can it be tutorial called "How to make your own spending money with your Husband/Boyfriends Gaming Computer" and target Stay-At-Home Mom's and etc.. (yes I know others will see the video, but they will be like "what the hell" she better not, because that's what I'm going to do")

include a how to move your earnings from the wallet to your bank account (examples) or how to spend it on overstock.com etc...

and see what happens.

I'm pretty sure you won't have to worry about centralized mining for awhile.

Sorry for belaboring the point but that 85+ million possible dollars the core dev's are sitting on, y'all should be able to pay to put something together that is a hell of a lot slicker than what even NICEHASH has going on. Which will further help keep your blockchain solution decentralized. Ya Ya, I know some zcash foundation put 80k up for grabs, ooh, aah, someone pat them on the back 0.001 percent of the dev fee up for grabs. and opensource is freaking great. sure. hard to put deadlines on development when waiting for someone to deliver something for free.

It's funny, Bitcoin is the only one that didn't kick it's miners in the teeth, or try to kick them in the teeth when they figured out a better way to support the network, for a piddling reward. that now has immense value though possibly transitory.

Eth, Zcash, all y'all so worried about innovation. that your planning ahead of time to kick it in the teeth if it rears it's ugly head.

Just think what if "you found a legitimate way to save 10 percent on your capital gains tax next year... and oops seconds after you implement it, the IRS (insert your local country equivalent) submits a new ruling 2 days after tax season starts, but retro-active enough that you have to update your tax software to account for the new rules.

I'm not pro or anti asic's, fpga's, AMD vs Nvidia, but I have a small investment in mining gear not enough to solo mine, but enough to be glad that I'm doing what I'm doing as a supplement. But definitely feel like even us miners mining through a pool, the pool's centralize the effort as much as anything else.

Was there actually a point, not really. just my mad ramblings of coming up to speed the last three months, building rigs and getting a handle on the subtleties for myself and others here in the Midwest, and watching the rules change by the day on various blockchains.

[h4x3rotab]

Bitmain just announced their Cryptonight ASIC. Time to consider a PoW backup plan seriously? https://www.reddit.com/r/Monero/comments/84kkw2/antminer_x3_220kh_at_550w/

[bitcartel]

...and Bitmain just announced their EtHash ASIC for Ethereum. https://twitter.com/BITMAINtech/status/981169456411021312

There is a good discussion here: https://github.com/ethereum/EIPs/issues/958

[daira]

My opinion on mining centralization and ASIC resistance: https://forum.z.cash/t/let-s-talk-about-asic-mining/27353/137

[h4x3rotab]

Makes sense but I doubt slightly tweaking the parameters of Equihash can help keep ASIC away. Shouldn't we do more planning than just changing N & K?

[tromp]

The current parameters are putting Zcash at risk of a single-chip ASIC, which is the worst possible outcome.

One of the goals of a memory hard PoW should be to force off-chip communication to separate memory chips, so as the make ASICs face the same memory bottleneck that GPUs do, and minimize their advantage. Equihash at (200,9), allowing for a 144MB solver, fails to serve that goal, unlike (144,5), which would need 2.5GB of memory.

Could Zcash invest resources and stretch schedules to increase memory requirements in Overwinter or Sapling?

[str4d]

Could Zcash invest resources and stretch schedules to increase memory requirements in Overwinter or Sapling?

We just cut v1.1.0-rc1 today, which will next week become the release with the Overwinter activation height, so a PoW change will not happen in Overwinter.

I doubt it would be feasible for Sapling either. Assuming that block header parsers are implemented correctly, it shouldn't require changing them (the Equihash solution in the header is explicitly a variable-length field), but anything in the ecosystem that validates the proof-of-work would need to be updated. It would also require changes to consensus rules, which are already frozen at the specification level for Sapling.

That said, the necessary consensus rule and implementation changes necessary to implement such a switch should be figured out, because that would make it much easier to reason about deployment overheads and timelines.

Equihash at (200,9), allowing for a 144MB solver, fails to serve that goal, unlike (144,5), which would need 2.5GB of memory.

I agree that (144,5) would be significantly preferable parameters:

• Still inside the RAM limits of modern GPUs.
• The solution data in the block header would drop from 1344 bytes to 100 bytes (plus the length field of the vector in both cases).
• BLAKE2b calls in verification reduced to 6.2% of present (63 for (144,5) vs 1023 for (200,9)), significantly improving header verification performance.

The open-source miners all support this parameter set IIRC. It would be great if someone could evaluate the closed-source miners to determine what they support.

[tromp]

The solution data in the block header would drop from 1344 bytes to 800 bytes (plus the length field of the vector in both cases).

It should drop to 2^5 25-bit indices, or exactly 100 bytes.

The open-source miners all support this parameter set IIRC. It would be great if someone could evaluate the closed-source miners to determine what they support.

I'm not aware of any open-source solver supporting (144,5) other than my own. But given that (192,7) miners already exist (eg http://cryptomining-blog.com/tag/equihash-192-7/) with the same 24 bit digits as (144,5) has, performance should be roughly similar...

[str4d]

It should drop to 2^5 25-bit indices, or exactly 100 bytes.

Whoops, forgot to divide by 8. You're right :smile:

[h4x3rotab]

(144,5) will be harder to make an ASIC but that may not be the bottleneck. Note that 144MB is already very hard for a single-chip ASIC. (Btw, I've sent an email to security@z.cash but got no response)

[zookozcash]

I don't necessarily agree that our current goals and requirements are the same as we had two years ago (linked in the original post above).

And I don't necessarily agree that we should support a change to do something today just because it would have been good if we had done it in the first place two years ago. That is: there might be some things that would have been a good idea to start with but aren't a good idea to change to.

I also think that ASIC mining (so-called "ASIC", although at a technical level it might not be actual ASIC, but that doesn't matter because "ASIC" is the word people use to mean custom mining hardware) is better than CPU/GPU mining in one important way, which is that it aligns incentives between the users of the network and the miners. That's important, and it might turn out to be the most important factor!

I'm also skeptical that it is actually possible to prevent ASIC mining. We tried, two years ago, despite several experts telling me that they thought it was impossible. Now it looks like (re the Cryptonight and EthHash miners) the experts were right.

So why did I decide to do it even though multiple experts told me they thought it was impossible? Because I had realized that there is a irreconcilable trade-off between broader-user-base and better-incentive-alignment. CPU/GPU mining gives (or at least potentially can give — see below) broader-user-base, but ASIC mining gives better-incentive-alignment. I figured that we would aim for broader-user-base — because at that point the biggest threat to the success of Zcash was just not gaining a critical mass of early users! — and even if it didn't work, at least we'd still get better-incentive-alignment. I also figured that it would work for at least a few years, and then if it then eventually failed, it would have served its purpose by bootstrapping the Zcash network with a broader base of users.

But on top of all the other skepticisms listed above, I'm most skeptical of all of the idea that CPU/GPU mining promotes decentralization! I think it is entirely plausible that CPU/GPU mining could be centralized (e.g. how do we know that the current GPU-oriented Zcash mining isn't highly centralized? How do we know that the current CPU-oriented Monero mining isn't highly centralized?), and I think it is entirely plausible that custom mining could be decentralized (e.g. hardware miners could be widely available and easy to install and use, and therefore more accessible to more users in more countries and situations than GPU miners).

So in sum, I don't agree with the statement in the original post above that our current mining properties and goals are to make ASIC miners less relatively-efficient.

[daira]

I don't agree that ASIC mining aligns incentives between the users of the network and the miners. It seems that most of the people commenting in https://forum.z.cash/t/let-s-talk-about-asic-mining/27353 don't, either. It seems pretty clear that mining coalitions in other coins act purely in their own interests, regardless of the interests of users.

In particular the argument that ASIC mining firms are highly vulnerable to interference by governments, regulators, or other powerful interests is a serious one.

[zookozcash]

I strongly oppose making any changes to anything that could delay or destabilize Sapling. That definitely includes any possible change to the PoW algorithm, the consensus algorithm, or the design of Sapling. That means the earliest we could deploy a change to the PoW would be in the next Network Upgrade cycle after Sapling. I'm not sure how fast we can execute Network Upgrade cycles — I'd love it if the June→September gap between Overwinter and Sapling (a) works as planned, and (b) is repeatable so that we can start executing Network Upgrade cycles that activate every 3 months.

(Note: there's a development cycle for each Network Upgrade that starts about 6–9 months before the activation date, so having another activation every 3 months doesn't mean you can squeeze the entire development cycle into 3 months, it means you already have to have the next one or two in flight before the first one activates!)

So if we started now working on a PoW change, there's a chance the Network Upgrade that began the cutover to the new PoW could trigger as early as December 2018. There are, of course, 99 reasons why that might not happen in December, starting with that we might have other things that we prioritise doing instead, and also including that we might collectively decide that ASIC miners are good for the network and the users. ☺

[daira]

It was not proposed to change the PoW in Sapling. (I completely agree that doing so would be incredibly risky and would have blown the complexity budget, even if the Sapling consensus rules were not already frozen except to fix any security flaws.)

[zookozcash]

I don't agree that ASIC mining aligns incentives between the users of the network and the miners.

I'm refering to a simple fact, that if your capex has value for things other than mining, then you have less incentive alignment than if your capex doesn't. Note that miners can be repurposed across multiple coins that share a PoW, so the incentive alignment is between the owner of the hardware and the set of coins that share that PoW, not any one coin.

It seems that most of the people commenting in https://forum.z.cash/t/let-s-talk-about-asic-mining/27353 don't, either.

This is just a fact, it's not a matter of popular opinion.

It seems pretty clear that mining coalitions in other coins act purely in their own interests.

Yes, and we should design for the assumption that they will do so, regardless of what technology is used for mining. This is why the better-incentive-alignment argument is important.

[daira]

Mining coalitions, ASIC or not, very clearly have a conflict of interest on any decision about consensus changes that affect block rewards or fees. Their interests may not coincide with users'. To the extent that ASIC mining results in larger mining coalitions (which it certainly has in Bitcoin), that's a problem.

[elkimek]

So @zookozcash simple question. In case that Bitmain or any other ASIC manufacturer announce ASIC for equihash, would you support fork against it or not?

[daira]

The issue was raised, by @zookozcash and @h4x3rotab at least, of whether a parameter change to (144, 5) would be sufficient to prevent ASIC mining, or if it is then for how long.

It doesn't seem economic with current technology to put the amount of memory needed for (144, 5) on the same chip as the Equihash logic. (It might be with "3D" memory technology at some point, although not with the current Intel XPoint technology.) Forcing a split between multiple chips preserves the main limitation in the performance of Equihash, which is memory bandwidth.

My estimation is that a (144, 5) parameter change is likely to delay ASICs long enough to allow time to switch to PoS or a PoW/PoS hybrid if we decided to do that (which is dependent on many other factors).

[bjames301]

@zookozcash I truly believe that all of us who mine and those who truly believe in Zcash would wait till a new POW could be rolled out correctly. I dont think anyone would want you to push off the Sapling as long as the POW change was forth coming. We are reasonable people. :)

[raresolid]

I've supported Zcash for a very long time and I was surprised to see a shift from the core values and pillars as it relates to the ASIC discussion. An organization who shifts their core believes is an organization who doesn't have any beliefs/pillars to begin with. I always believed that Zcash would work to keep the coin decentralised by not letting a couple of large corporations take over the mining of their coin. An example of control is how these large corporations like Bitmain geo-restrict where their units are sold and also the quantities. How does that support an open decentralised coin for everyone to mine? I just don't see it. With GPU mining, anyone can buy a GPU and start mining the coin. This ability alone is enough to allow more and more people to be involved in the mining process. I can not see any valid reason for thinking that ASIC mining equates to a more open and available decentralised coin.

[flame05]

because at that point the biggest threat to the success of Zcash was just not gaining a critical mass of early users!

I feel very embittered from your words @zookozcash , where did this version of you go?

@elkimek made a great question!

[bjames301]

I second that notion!!!!

On Sat, Apr 7, 2018 at 2:26 PM, raresolid notifications@github.com wrote:

I've supported Zcash for a very long time and I was surprised to see a shift from the core values and pillars as it relates to the ASIC discussion. An organization who shifts their core believes is an organization who doesn't have any beliefs/pillars to begin with. I always believed that Zcash would work to keep the coin decentralised by not letting a couple of large corporations take over the mining of their coin. An example of control is how these large corporations like Bitmain geo-restrict where their units are sold and also the quantities. How does that support an open decentralised coin for everyone to mine? I just don't see it. With GPU mining, anyone can buy a GPU and start mining the coin. This ability alone is enough to allow more and more people to be involved in the mining process. I can not see any valid reason for thinking that ASIC mining equates to a more open and available decentralised coin.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/zcash/zcash/issues/1211#issuecomment-379489507, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3onUVWUdQjMuFV0q-8cnnDLIL812ewks5tmQThgaJpZM4Jg__D .

[LordMajestros]

@zookozcash Another simple question. If there was a way to achieve ASIC resistance would you support a fork to do so? I ask because to the best of our knowledge right now it is not possible but I've seen some suggestions on the monero issue tracker that seem promising.

[LordMajestros]

Here is the relevant discussion Idea for ASIC resistance #3545

[daira]

The MyriadCoin approach is complicated and I'm not particularly enamoured of it as a way of achieving ASIC resistance on its own, but support for multiple parameters (rather than an instant switch at a given block height) may be necessary/useful in any case for a PoW migration strategy.

[bitcartel]

Support for multiple equihash parameters doesn't seem necessary, given that the objective is to move away from existing parameters deemed no longer useful.

It might also be harmful. Some disgruntled ASIC miners might choose to disrupt the network with empty blocks, withholding, 51% attacks... before departing for their own chain like "Monero Original" and "Monero Classic".

Monero's recent PoW switch, where mining pools and GPU miners were updated ahead of time (with a few weeks notice) demonstrate it is feasible to execute an instant switch under time constraints.

The removal of ASIC mining hashpower has resulted in slower blocks over the past 24 hours, but this would be less of an issue for Zcash which has a more responsive difficulty adjustment algorithm, based on the last 17 blocks, compared to Monero's retargeting window of 720 blocks.

[LordMajestros]

@daira The MyriadCoin approach is complicated but that's not what I was referring to. See the comments by hyc and tevador. Here is the relevant bit

From @tevador

I started developing a similar concept of an ASIC resistant PoW.

Basically, the idea is to create a virtual instruction set with basic bitwise, arithmetic and floating point operations and some branching (this must be designed carefully to avoid infinite loops). At the start of each new block, the hash of the previous block is expanded into a few hundred random instructions and compiled into machine code (for CPU/GPU). This random program is then applied to the scratchpad (initialized from the current block header). At the end, the scratchpad is hashed and the result is compared to the target.

Therefore, the PoW would change with every block and if the instuction set is broad enough to cover the majority of what current CPUs/GPUs implement in hardware, then any potential ASIC would be basically a CPU/GPU with similar performance.

The specific details can be tuned so the benefits from a custom hardware are negligible - for example, the PoW code should fit into L1I cache, scratchpad fits into L2/L3 cache, 64 bytes of data are processed at a time (whole cache line) etc. A different permutation of opcodes could be chosen for every block to prevent direct hardware decoding.

This approach has some disadvantages, but it's the only one guaranteed to be ASIC resistant.

From @hyc

Sounds a lot like what I outlined here https://www.reddit.com/r/Monero/comments/865vbb/asic_resistance/dw2p4b7/

The basic idea is to use randomly generated code. If you simply select from a handful of known algorithms, eventually someone will build an ASIC that just implements the entire set of algorithms. Using randomly generated code, with an RNG with an intractably large set of states, will defeat that possibility.

My approach is to randomly generate javascript. Developing your own instruction set sounds great but such a development will be immature for quite a long time, leaving countless opportunities for fatal flaws and unexpected optimizations. Using something mature like javascript means most optimization opportunities have already been exploited. (And meanwhile, if anyone does manage a new optimization, it becomes useful to the entire computing community.) I will have deployable code demonstrating this approach in a few days.

[bitcartel]

To help answer this ticket, I spent a few hours on the weekend and have a prototype based on v1.1.0-rc1 which allows equihash parameters to be changed for each network upgrade.

The code is in commit https://github.com/bitcartel/zcash/commit/02397b64b30916d3ff3ab0222b9b197ac3e1b9fb on my branch here: https://github.com/bitcartel/zcash/tree/equihash_upgrade_parameters

Here is the output of solutions when switching parameters in regtest mode: https://gist.github.com/bitcartel/49d1d028e31c482028327ae95debf045

Example usage in regtest mode:

./zcashd -nuparams=5ba81b19:10 -eqparams=5ba81b19:96:5

This activates branch 5ba81b19 at block 10, at which point the equihash parameters switch from (48, 5) to (96, 5). I also tried switching to (144, 5) and using -equihashsolver=tromp which appears to work fine, as does getblocktemplate.

The code should be useful for the community to build on and experiment with different solvers and parameters.

[zawy12]

If you change the POW every block to one of 16 options based on the last 4 bits of the previous block's hash, I can write a difficulty algorithm to handle the different POWs. It would need to look back ~20xN blocks to determine the relative difficulty of each algorithm, then use an existing difficulty algorithm with an N window, adjusted by that relative factor. If you changed the POWs every 6 months, maybe GPU coders would not be able to keep up and CPU mining could be competitive.

@bitcartel N=17 with the 75% tempering makes it act a like N=4x17=68 SMA, plus a 6 block delay. Like you say, a lot faster than Monero. Last summer before the Asics, Monero's 24 hour difficulty did not track hash rate as good as Zcash's 3-hour "window". But this winter, Monero with ASICS providing stability out-performed Zcash. BCH's 24 hour simple moving average ties Zcash. Monero (cryptonote) clones are going to my LWMA which has about 5x fewer delays and 5x fewer "blocks stolen" than Zcash's. Since Zcash has the advantage of being big, I compared them to Hush, which is still a lot bigger than some of them. About 8 switched in last 2 months, and 10 more have commits.

[zawy12]

Cell phones should be able run the following algorithm more efficiently than ASICs and FPGAs at about the same electrical cost as GPUs and CPUs ... if you can write a routine to create a general class of functions from a seed that has the following characteristics:

• Input size equal to output size, say 64 bits.
• The class is complex enough to not be optimizable as a class (hence ASICs and FPGAs are not useful)
• The class has an upper and lower bound on computation time for a given "CPU"

The last item seems to require a similar distribution of operations and a restricted way of using them. This would make achieving the 2nd to last item more difficult. To relax the computation time bounds in the last item without risking miners abandoning slow functions by going to the next nonce, I needed to add an outer loop for the function generator instead of letting it be at the nonce level. It can't be in the inner loop because it can't require a large percentage of the total computation, or it could be implemented in an ASIC. Nonces need to be random.

The POW is then:

myhash = ffx32 
unless (myhash < target) {
   nonce++
   j =  hash(previous block hash + nonce)
   for (1 to 100)  { 
       j++
       function = generate_function( hash( j ) )
       for  (k=1 to 100,000) {
           input=output
           output = function( input )
       }
   }
   myhash = hash(output)
}
if ( myhash < target ) { claim my block }

The goal is to make memory useless and try to make the "CPU" burn as much electricity as possible. This is not more "polluting" or "wasteful" than investing in more hardware to avoid electrical expense.

[ebfull]

There's nothing obligating us to change the PoW or not change it, but we definitely stated ASIC resistance was a quality we were striving for. Ideally we should have compelling evidence if we want to abandon that.

I think we should plan to preserve the status quo intention of ASIC resistance by switching to the (144, 5) parameters in a network upgrade following Sapling. It just so happens that (144, 5) is what we would have prefered to launch with anyway!

[daira]

I reviewed @bitcartel's branch at https://github.com/bitcartel/zcash/tree/equihash_upgrade_parameters . Concept ACK.

[mms710]

This is a duplicate of https://github.com/zcash/zcash/issues/3672