Identify property by issuance transaction

[dexX7]

The current id scheme with ascending numbers based on the complete history is extremely fragile.

Think of the two cases:

1. A client parses all transactions and there is one false interpretation of a simple send, e.g. due to a corrupt file, ... in the worst case the send is not recognized as such and a single branch of the whole system state is changed / black / invalid.
2. A client parses all transactions and there is one false interpretation of a property creation, e.g. due to a corrupt file, ... in the worst case all following transactions are completely changed / black / invalid.

I suggest to support identification of properties by some other mechanism which could be based on the (partial) hash of the creation transaction. I'm not sure, if this is the best approach, but it would be an improvement.

As bonus on top the property identifier would be known before the issuance transaction confirms and this enables chaining of Tx 54: New Property with Managed Number of Tokens and Tx 55: Granting Tokens for a Managed Property. This was sort of discussed in an Omni thread: https://github.com/mastercoin-MSC/omniwallet/issues/1062#issuecomment-60933928.

If the scope and context of an application is isolated, then a client may not even be required to parse all Mastercoin transactions, but only those after and related to the issuance of a property.

[marv-engine]

@dexX7 I agree, we should use an identifier that's not subject to changing. Note that it would have to include, or be combined with, an ecosystem indicator.

[dexX7]

An approach of minimal length hash pieces that are based on the context would work well for the DEx, because the context and scope covers my own orders and it's fair to assume a wallet has knowledge about the own transactions.

I don't really see how this could be applied for property identifiers in general, because any similar approach would imply the requirement of perfect knowledge and you need to know every property that's out there, to determine, if a hash piece is long enough -- which would be an improvement to status quo, but a client that is unaware all properties would be required to use the full hash nevertheless.

Using the transaction hash of the issuance transaction would be the most solid solution which would require 32 byte -- which seems to be a lot. The hash function applied there is SHA256. Another prominent hash function used in Bitcoin is RIPEMD-160 - 20 byte length.

You already mentioned the ecosystem flag. I'm unsure, if you meant to use the flag for the selection or for the issuance. When creating a property the ecosystem flag is currently provided explicitly and when selecting a property (e.g. in a simple send), then the ecosystem can be derived from it's identifier.

I don't really see the need for an explicit ecosystem flag when selecting a property, because it's already given implicitly during the creation. Say for example there is MaidSafeCoin which has the identifier 3.

The identifier itself tells us: it's in the main ecosystem. But what if we would select it via it's creation hash? The creation hash is 86f214055a7f4f5057922fd1647e00ef31ab0a3ff15217f8b90e295f051873a7 and a client could as well fetch the issuance transaction and extract information about it's ecosystem.

---

What do you suggest? Any ideas how we could get away with a shorter identifier?

[dacoinminster]

A corrupt file can cause chaos no matter what system we use. I think I'd rather a corrupt file cause a catastrophic failure (which would be quickly noticed) than a subtle failure.

On Wed, Oct 29, 2014 at 3:57 PM, Marv Schneider notifications@github.com wrote:

@dexX7 https://github.com/dexX7 I agree, we should use an identifier that's not subject to changing. Note that it would have to include, or be combined with, an ecosystem indicator.

— Reply to this email directly or view it on GitHub https://github.com/mastercoin-MSC/spec/issues/280#issuecomment-61019697.

[dexX7]

The corrupt file example was just an example, but the underlying difference where I was getting was that in one case flawless and full knowledge about the whole system is required and in the other only a fraction.

In the broader picture I believe the whole Mastercoin system can be split into "scopes" and "contexts" and a community that uses it's own token and never uses something else than simple send (well, that's probably a bad example) might as well get away with maintaining much less data, namely the "branch" that derived from the issuance of the community token. Hope the idea is clear. :)

[dexX7]

Data integrity is probably a whole different topic on it's own, but equally (if not more) important. I'm already trying to push awareness for this over here: https://github.com/mastercoin-MSC/mastercore/issues/143.

[zathras-crypto]

Playing devil's advocate here but why introduce that level of complexity in the first place (wrt property creation)? Why not simply introduce a new version of the property creation transactions that includes a field for the requested propID. If it's already in use when the TX is parsed it's invalid.

For example I issue a tx creating a property and use '3' as the field for the requested property ID. This tx is invalid and the property doesn't get created because the property ID is already in use.

Alternatively, let's use the example that I'm the popular 888 Casino website and I want to shift to my own crypto-backed tokens for some purpose - I issue a tx creating a property and use '888' as the field for the requested property ID. This tx is valid and the property gets created because the property ID is available.

There would be some (not dramatic) coding changes in Core yep but this seems like a fairly simplistic model spec & code wise to move away from sequential prop IDs (which have various problems I won't re-iterate here).

Thanks Z

[dexX7]

Why not simply introduce a new version of the property creation transactions that includes a field for the requested propID. If it's already in use when the TX is parsed it's invalid.

Unless I miss something, this doesn't seem to solve the issue at all, because "full knowledge" (= every Master transaction processed correctly) is still required to determine, whether a property identifier is valid.

There are actually two seperate aspects:

1. full block chain / knowledge about transactions is required
2. complete and correct knowledge about meta transactions

What I proposed earlier, using RIPEMD-160(transaction id) or a "partial hash piece" as identifier, clears the second point, but I just noticed full block chain knowledge is still required. To solve both, only full transaction hashes seem to be viable.

level of complexity

To what are you referring here? The base idea is to use transaction ids to identify properties - which seems like a straightforward approach, but I'm biased against raw hashes due to the length.

OTOH there are several parts where the transaction size could be reduced significantly (e.g. using multisig output for change), but it doesn't seem to be an issue at this point.

Right now a "standard" simple send has a size of about 340 byte and using hashes instead of property ids comes with an overhead of 28 byte (32 byte hash vs. 4 byte property id) -- not very much imho.

Bitcoin Core 0.9 has "static fees" of x BTC/1000 byte, so it's "for free", because there is quite some space left. With 0.10 it's going to be fees paid per byte, and assuming 0.0001 BTC/kB, the overhead would equal additional costs of 0.0000028 BTC, less than 0.01 of a penny right now.

Let me be clear, I'm biased against those 28 bytes, but in the broader picture even a slightly improved input/change handling would have greater consequences - not even speaking of actually using "low output values" on mainnet, which seems to be the most significant, but unnecessary, cost factor at the moment.

[zathras-crypto]

Unless I miss something, this doesn't seem to solve the issue at all, because "full knowledge" (= every Master transaction processed correctly) is still required to determine, whether a property identifier is valid.

It's not exactly thoroughly thought out :P but I thought of this as only requiring us to know about previous creation transactions, not "full knowledge" as you put it of everything - create-property transaction validity is not dependent on anything other than previous prop creation's (assuming we allowed it to be specified) - I may be wrong though will need to think it through a little more.

To what are you referring here?

Simple change/scope control - allowing a property ID to be specified doesn't involve a large degree of change, but on initially reading your proposed move to property hashes for identification instead of property ID's seemed like a substantially 'bigger' change.

Also just for 2 cents - does your method not require knowledge of previous transactions also? Utilizing the hash is atomic sure but we still need knowledge of previous create transactions to code in collision handling of the generated hash (no matter how unlikely we would need to cater for it I would assume?).

Thanks :) Z

[dexX7]

create-property transaction validity is not dependent on anything other than previous prop creation's

The dependency on previous property creations is the issue I see. ;) And this seems to be a similar problem with user defined ids as well as ascending auto-ids. You're right though, not every Master transaction is required to be processed correctly, but only those involving property creation.

collision handling

Let's stick to transaction hashes as identifier. In this case the property id == transaction id of creation transaction. One would be able to fetch the transaction and even without any other transaction the property information would be available.

The txid of a Bitcoin transaction is a SHA256(SHA256(raw_transaction)) hash, and I think it's safe to assume collision resistance is given.

[ABISprotocol]

History forgotten

[zathras-crypto]

The txid of a Bitcoin transaction is a SHA256(SHA256(raw_transaction)) hash, and I think it's safe to assume collision resistance is given.

Fair point :)

Let's stick to transaction hashes as identifier.

This is a pretty fundamental change in the code (switching property ID's from uint32 to uint256) but could be done. Before we think about code though let's get some more views/feedback in.

Note, transaction malleability could be used to play havoc with anything that involves a property creation and zero conf if we did go this route, so would need to tread carefully for pending/unconfirmed perhaps.

[zathras-crypto]

Ah - think I see an issue - we couldn't treat all creations as atomic because validity for 51 requires prior knowledge (only allowed one active crowdsale).

[dexX7]

Good point. It still narrows the scope of "all 'create property' transactions" down to "all 'create property' transactions of address x".

Or, as alternative and further protocol change, multiple crowdsales per address are allowed and crowdsale participation becomes a more active process where the crowdsale-to-participate-in is explicitly selected. This would be in line with #277 ("I unintentionally participated in a crowdsale") and #261 ("Allow users to specify the specific terms of an offer they are responding to"). Edit: also #132 and #268.

transaction malleability

That's unfortunally true. Depending on the use-case one could chain transactions where either the whole chain is valid or invalidated due to a altered crowdsale transaction (assuming the issuer wants to do something sneaky with unconfirmed transactions for some reason).

The initial overall goal I had in mind was not necessarily to get down to robustness on a single transaction basis, but to prevent brutal butterfly effects, so to speak. (e.g. all property ids off by one number)

This is a pretty fundamental change in the code

Yeah, probably. ;) I just focused on the theory and haven't thought much about potential implementations. The most annoying part is probably to support both schemes at the same time (to process the "old" crowdsales), but this an issue in general and needs to be addressed with every change.