Failure to create datatoken, looks like default artifacts issue

[MantisClone]

Note: this github issue uncovered two problems: (1) network_url polygon (2) default artifacts issue. Issue (1) is in the description below and the first few comments. Issue (2) is in the remaining comments.

--

First reported by Corrie#6921 via Discord: https://discordapp.com/channels/612953348487905282/822085610172317706/836613103839871037

Trying to publish via ocean.py. Changed out the config to look like this:

config = {
   "network": "polygon",
   "metadataStoreUri" : "https://aquarius.polygon.oceanprotocol.com/",
   "providerUri" : "https://provider.polygon.oceanprotocol.com/",
}

But get error message:

AssertionError: The given network_url polygon does not start with either http or wss, in this case a network name is expected >and must be one of the supported networks {'rinkeby', 'kovan', 'mainnet', 'ropsten', 'ganache'}.

Addtional context: https://discordapp.com/channels/612953348487905282/822085610172317706/836681454787231774

config = {
    "network": "polygon",
    "network_url": "https://rpc.polygon.oceanprotocol.com/",
    "metadataStoreUri" : "https://aquarius.polygon.oceanprotocol.com/",
    "providerUri" : "https://provider.polygon.oceanprotocol.com/",
}
wallet = Wallet(ocean.web3, private_key=private_key)
datatoken = ocean.create_data_token("foobar", "fb", from_wallet=wallet)

AssertionError: The given network_url polygon does not start with either http or wss, in this case a network name is expected and must be one of the supported networks {'kovan', 'mainnet', 'rinkeby', 'ganache', 'ropsten'}.

config = {
    "network": "https://rpc.polygon.oceanprotocol.com/",
    "metadataStoreUri" : "https://aquarius.polygon.oceanprotocol.com/",
    "providerUri" : "https://provider.polygon.oceanprotocol.com/",
}
wallet = Wallet(ocean.web3, private_key=private_key)
datatoken = ocean.create_data_token("foobar", "fb", from_wallet=wallet)

WARNING:root:No logs were found for tx 0x9e8ff0580d1a13adcd3e7e9390aeb0602f680dede2214c64c6e1c9e50377994a.
AssertionError: new datatoken has no address

[MantisClone]

The changes in PR #308 so far (up through https://github.com/oceanprotocol/ocean.py/pull/308/commits/fc9e7f1a5e0d424384a3d8ffbe7d5a45763a2d9e) have fixed the initial AssertionError where "polygon" was not in the SUPPORTED_NETWORK_NAMES. Now, it should be possible to specify config = { "network": "polygon" }

However, the second issue remains:

WARNING:root:No logs were found for tx 0x9e8ff0580d1a13adcd3e7e9390aeb0602f680dede2214c64c6e1c9e50377994a.
AssertionError: new datatoken has no address

[trentmc]

The second issue is a sign that it can't see the contract on polygon. That contract info is normally imported from the ocean contract pypi repo. So then the question is: does ocean contact pypi repo have it? If no, fix that. If yes, then track down why it can't see it in Ocean.py

[MantisClone]

Reported by Corrie#6921 via Discord

I get the same error "WARNING:root:No logs were found for tx..." on mainnet using latest version of ocean.py. Can't seem to create a token. Works in Rinkeby.

[MantisClone]

Does ocean contract pypi repo have [address.json]?

Yes. it places it at $VIRTUAL_ENV/lib/python3.8/site-packages/artifacts/address.json.

(venv) david@david-Inspiron-5502:~/projects/ocean.py$ find . -name "address.json"
./venv/lib/python3.8/site-packages/artifacts/address.json

Then track down why it can't see it in Ocean.py

ocean.py can't see it because the default artifacts.path is ~/.ocean/ocean-contracts/artifacts and the address.file is derived from that.

>>> config = {
...     "network": "polygon",
...     "metadataCacheUri": "https://aquarius.polygon.oceanprotocol.com/",
...     "providerUri": "https://provider.polygon.oceanprotocol.com/",
... }
>>> from ocean_lib.ocean.ocean import Ocean
>>> ocean = Ocean(config)
>>> ocean.config.artifacts_path
PosixPath('/home/david/.ocean/ocean-contracts/artifacts')
>>> ocean.config.address_file
'/home/david/.ocean/ocean-contracts/artifacts/address.json'

@trentmc @calina-c What would you say is the appropriate solution here? Should I change the default artifacts.path to be $VIRTUAL_ENV/lib/python3.8/site-packages/artifacts? $VIRTUAL_ENV points to the venv dir.

[calina-c]

I defer to @trentmc on this one. I'm not sure why this happens only now. Shouldn't it affect other networks as well? If we use this derived path, what happens if someone uses the native Python without a venv?

BTW I think this is related to some contracts changes, and it seems artifacts is like a python package in itself. But that's not ok, because "artifacts" is way to generic. It should at least be ocean_artifacts or something like that. Maybe we can import artifacts now? Did you try that in the config?

[trentmc]

I renamed this issue to point out its similarity to previous issues.

Summary

What would you say is the appropriate solution here?

Until recently, the answer would be to point to venv/artifacts/, kinda like what you suggested. That's what it does elsewhere in ocean.py.

But that's problematic of course. For ocean.py to get addresses specified by ocean-contracts, (a) it shouldn't have to point to a directory in a virtual env, and (b) it doesn't do an import, instead looking into a weird directory related to the install. This is a non-standard use of a python library, it's brittle and has poor developer experience.

Issue multi-repo-issue#83 describes (a) and (b) above, and points to a solution to make that cleaner. While the issue has not been closed, it's made enough progress where ocean.py can use the main result. That is...

Config.py should get the default by approach: how ocean-contracts now specifies in its README:

from jsonsempai import magic
from ocean_abis import address #uses json address file from oceanprotocol/contracts/
address.mainnet
{'DTFactory': '0x57317f97E9EA49eBd19f7c9bB7c180b8cDcbDeB9', ...

Details

This is an ocean.py artifacts issue. We've seen several artifacts issues over the last few months; we've slowly been chopping them down. Here's a run-down.

At first ocean.py had its own artifacts directory. This is a problem because it violates DRY and needs to be kept up to date with the addresses in ocean-contracts. The latter wasn't happening. We fixed this in ocean.py#95.

Then, ocean.py#205 found that when CONFIG_FILE value wasn't specified, it wasn't grabbing a good default. Therefore it couldn't find the address file. Fixed in this commit.

In doing issues 95 and 205, I came to understand the https://github.com/oceanprotocol/multi-repo-issue/issues/83

However it seems that in some scenarios, it defaults to looking the address file in ~/.ocean/ocean-contracts/artifacts when it should be looking for something given via ocean-contracts. Multi repo 83 like mentioned above describes the issue and the fix.

[trentmc]

Shouldn't it affect other networks as well? If we use this derived path, what happens if someone uses the native Python without a venv?

I meant to mention: yes it should affect other networks as well. But perhaps not ganache or barge because it might auto-set some stuff, which is why the issue only showed up now versus in unit tests.

I will rename the issue accordingly.

[MantisClone]

2nd issue fixed

I've fixed the issue where the Ocean instance couldn't see the polygon contract addresses in commit https://github.com/oceanprotocol/ocean.py/pull/308/commits/fd35692f346cefb611736350bf83ba9e216fdf86. The fix was to default the artifacts_path property to the ocean_abis folder from the ocean-contracts dependency.

3rd issue revealed

Unfortunately, a third issue has been revealed and I'm stuck. @trentmc @alexcos20 @calina-c Can any of you provide insight on this?

web3.utils.events.get_event_data() raises MismatchedABI("The event signature did not match the provided ABI"). I have no idea why this happens.

Details

The error occurs at web3.utils.events.py::158

def get_event_data(event_abi, log_entry):
    """
    Given an event ABI and a log entry for that event, return the decoded
    event data
    """
    if event_abi['anonymous']:
        log_topics = log_entry['topics']
    elif not log_entry['topics']:
        raise MismatchedABI("Expected non-anonymous event to have 1 or more topics")
    elif event_abi_to_log_topic(event_abi) != log_entry['topics'][0]:
        raise MismatchedABI("The event signature did not match the provided ABI") # <-- error occurs here
    else:
        log_topics = log_entry['topics'][1:]

Here's the full callstack:

get_event_data (/home/david/projects/ocean.py/venv/lib/python3.8/site-packages/web3/utils/events.py:158)
_parse_logs (/home/david/projects/ocean.py/venv/lib/python3.8/site-packages/web3/contract.py:1290)
inner (/home/david/projects/ocean.py/venv/lib/python3.8/site-packages/eth_utils/functional.py:45)
processReceipt (/home/david/projects/ocean.py/venv/lib/python3.8/site-packages/web3/contract.py:1284)
_wrapper (/home/david/projects/ocean.py/venv/lib/python3.8/site-packages/web3/utils/decorators.py:14)
get_token_address (/home/david/projects/ocean.py/ocean_lib/models/dtfactory.py:58)
<module> (/home/david/projects/ocean.py/debug-create-datatoken-polygon.py:29)

Here's the event ABI at the moment the error occurred:

event_abi: {
   "anonymous":false,
   "inputs":[
      {
         "indexed":true,
         "name":"tokenAddress",
         "type":"address"
      },
      {
         "indexed":false,
         "name":"tokenName",
         "type":"string"
      },
      {
         "indexed":false,
         "name":"tokenSymbol",
         "type":"string"
      },
      {
         "indexed":false,
         "name":"tokenCap",
         "type":"uint256"
      },
      {
         "indexed":true,
         "name":"registeredBy",
         "type":"address"
      },
      {
         "indexed":true,
         "name":"blob",
         "type":"string"
      }
   ],
   "name":"TokenRegistered",
   "type":"event"
}

Here's the log_entry at the moment the error occurred. This transaction receipt only contains 1 log entry.

AttributeDict({
   "address":"0x0000000000000000000000000000000000001010",
   "topics":[
      "HexBytes(""0x4dfe1bbbcf077ddc3e01291eea2d5c70c2b422b415d95645b9adcfd678cb1d63"")",
      "HexBytes(""0x0000000000000000000000000000000000000000000000000000000000001010"")",
      "HexBytes(""0x0000000000000000000000007eb023bfbaee228de6dc5b92d0beeb1edb1fd567"")",
      "HexBytes(""0x0000000000000000000000007c7379531b2aee82e4ca06d4175d13b9cbeafd49"")"
   ],
   "data":"0x00000000000000000000000000000000000000000000000000001793103c040000000000000000000000000000000000000000000000000000038d7ea4c680000000000000000000000000000000000000000000000000aff569777d6511ed31000000000000000000000000000000000000000000000000000375eb948a7c000000000000000000000000000000000000000000000000aff5698f10754df131",
   "blockNumber":14189481,
   "transactionHash":"HexBytes(""0xaa3771ad84cdc5372d880f3feb6dc41943ddf8aae03ee66a93c0dc0f7874d7a2"")",
   "transactionIndex":47,
   "blockHash":"HexBytes(""0x89e7ade5aa2f94d5c86370c952ca47c8874bf86d918302fb10029f8acb0971b8"")",
   "logIndex":56,
   "removed":false
})

[trentmc]

Glad to see issue 2 fixed. In a way that makes sense 👍👍.

Re issue 3: Is it getting the abi via the ocean-contracts dependency? It should be. It'll be wrong if it's still looking in the old spot.

[alexcos20]

not sure where is that, but you have 6 topics in abi and only 4 in event.. definitely something does not match

[MantisClone]

The log_entry was LogFeeTransfer. So I think you're right :sweat_smile: something went wrong when I was creating the datatoken https://explorer-mainnet.maticvigil.com/tx/0xaa3771ad84cdc5372d880f3feb6dc41943ddf8aae03ee66a93c0dc0f7874d7a2/logs

I'll try creating a datatoken on ropsten so I can see what a successful datatoken creation looks like.

[MantisClone]

I think it ran out of gas.

[MantisClone]

I created a datatoken on Ropsten using the same script that I used for Polygon and here is the resulting transaction: https://ropsten.etherscan.io/tx/0x403107b7b7f0f56089a08d4c3cbc3ef609a795cb6897c660b1db973dedcc533a

I see that the gas limit was set to 268,583 and that it used 100% of it. 3 logs were created.

My guess is that somehow ocean.py is setting the gas limit too low on polygon. Will investigate further.

[MantisClone]

I executed all the steps in marketplace-flow.md and successfully created this datatoken/pool on polygon: https://market.oceanprotocol.com/asset/did:op:2357C32D8C68514D4186D0e193836156a133F0f3

When I first attempted to create a datatoken on Polygon 7 days ago (https://explorer-mainnet.maticvigil.com/tx/0xaa3771ad84cdc5372d880f3feb6dc41943ddf8aae03ee66a93c0dc0f7874d7a2/internal-transactions), I think web3.eth.estimateGas() was returning estimates that were too low. Between then and now, the polygon gas estimates have been fixed and now everything seems to be working.

Thus, I believe that the "3rd issue" has been resolved, PR #308 can be merged, and this issue (#307) can be closed. CC @trentmc @alexcos20

I've created https://github.com/oceanprotocol/list-purgatory/issues/44 to add the test asset to the purgatory.

[trentmc]

OK with me. So you'll merge the PR and close the issue?

[MantisClone]

Yeah I'll merge it after it's been reviewed.