ProviderNotConnectedError: Not connected to a network provider after ad-hoc connection

[NoahY43619]

Environment information

Debian 11 Python 3.9

• ape and plugin versions:

$ ape --version
0.5.1

$ ape plugins list
  solidity    0.4.0
  infura      0.3.1

• Contents of your ape-config.yaml (NOTE: do not post anything private like RPC urls or secrets!):

$ cat ape-config.yaml
ethereum:
  default_network: ropsten
  ropsten:
    default_provider: infura

Context

Hey there Ape team, and thank you for your work on this project. It's been a lifesaver since Brownie got sunsetted and has worked flawlessly so far.

Apologies in advance if this turns out to be not quite a bug report. I have a suspicion that I might be contorting ape into a use case it was simply not designed to handle.

Here's the situation:

I am using ape as an intermediate between a REST API and Solidity contracts. The API has a set of endpoints allowing to deploy and interact with contracts compiled using ape, and accessed by a Python backend using the standard project.MyContract interface.

This means that at no point am I using ape through terminal calls or the ape console, aside from an initial ape init and ape compile. Rather, all interactions with ape happen from within python modules.

What went wrong?

When trying to call methods from project.MyContract through API calls, I get the error:

ape.exceptions.ProviderNotConnectedError: Not connected to a network provider.

This does not happen while performing unit tests using ape test. I have tried to perform ad-hoc network connections:

with networks.ethereum.ropsten.use_provider("infura"):
    project.MyContract.do_stuff(...)

As well as by manually connecting to the target network:

with ape.networks.ethereum.use_provider("infura") as provider:
    provider.connect()
project.MyContract.do_stuff(...)

Both methods giving the same error quoted above when attempting to call contract methods, although inspecting ProviderContextManager immediately before those calls shows that I am indeed connected using the expected provider.

Am I missing something in the way I am trying to use Ape? More generally, what would be the best practice to create and maintain a connection to a given network in use cases like the one I described?

Please let me know if there is any detail I can provide.

[fubuloubu]

Would be more helpful to see a script of how you're using it so we can reproduce. The first method connecting via context manager should would just fine (by default, ape does not connect until asked), or we could explore running your service as an ape script so you can run it using our built-in ape run cli command, which includes provider connection management as a feature.

[NoahY43619]

Hey @fubuloubu, and thank you for replying so quickly.

Here's a minimal script I cobbled together by removing all non-ape related stuff.

from typing import Any

from ape import accounts
from ape import networks
from ape import project
from fastapi import FastAPI

# Actual ETH accounts with Ropsten funds used here in full script
account = accounts.test_accounts[0]
app = FastAPI()

# Example metadata body. "cid" points to an asset stored on IPFS.
metadata = {
    "name": "test_name",
    "symbol": "test_symbol",
    "cid": "test_cid",
}

@app.post(path="/test")
def make(metadata: dict[str, Any], wallet_address: str) -> dict[str, Any]:

    with networks.ethereum.ropsten.use_provider("infura"):

        # ERC721NFT is an ape-compiled OpenZeppelin ERC721 preset
        contract = account.deploy(
            project.ERC721NFT,
            metadata["name"],
            metadata["symbol"],
            metadata["cid"],
            sender=account,
        )

        response = {
            "wallet_address": wallet_address,
            "contract_address": contract.address,
            "token_name": contract.name,
            "token_symbol": contract.symbol,
            "asset_cid": metadata["cid"],
        }

    return response

Interestingly enough, running this app using uvicorn and submitting a request to the endpoint crashes the server with message:

Segmentation fault (core dumped)

rather than the error described in my previous post, althought this might just be a non-specific uvicorn message obfuscating the real error.

As for running the app directly through the ape console, I don't see why not. I'll give it a try and update the results here.

[fubuloubu]

@NoahY43619 two quick things jump out at me, but I don't think it's related to your issue: contract.name should be contract.name() and contract.symbol should be contract.symbol(), since they both have to call out to the network

[fubuloubu]

Another thing, so you're using a test account that won't have a balance on a public network (even if it's a testnet) so that would probably have an exception. If you used a keystore account, you'd like have to unlock and set_autosign on the account so that it will sign anything your script asks it to.

EDIT: I see your note now, nevermind

[fubuloubu]

I got something working! Not sure how well this works with your use case, but this is a working POC:

scripts/deployer.py:

import click
import uvicorn  # type: ignore  
from fastapi import FastAPI     

from ape import accounts, project  
from ape.cli import NetworkBoundCommand, network_option

app = FastAPI()
account = accounts.load("my-account")  # whatever account alias you have here

@app.post(path="/deploy")
async def deploy():
    contract = project.Test.deploy(sender=account)
    # takes a lot of time to perform a deployment, might be better to pass the txn id back instead
    return {"address": contract.address}

@click.command(cls=NetworkBoundCommand)
@network_option()
def cli(network):
    # the combo of `NetworkBoundCommand` and `network_option` ensures that a network connection is occuring
    dev.set_autosign(True)  # necessary to perform signing operations without confirmation
    uvicorn.run(app)  # programatically start the server, not sure how to use `uvicorn` cli directly

which I run using this command:

$ ape run deployer --network :goerli:infura
Enter passphrase to permanently unlock 'my-account': 
WARNING: Danger! This account will now sign any transaction it's given.
INFO:     Started server process [35101]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

Note that I have to use a public network because that's the one account has a balance on (otherwise, it won't work. area of improvement for sure)

and I can do an http post request to /deploy!

After like 2 minutes waiting on the request to process the deployment/confirmation (since it is a blocking operation), I will get a response from the server containing the deployed address!

[fubuloubu]

Another note: for this kind of use case, it would probably be best to return the deployment txn id and have the user wait for it to process

[fubuloubu]

Usage note, the lag in handling is pretty significant, I don't think this would work very well for a production server. More than happy to hop into Discord and work through it a bit more to identify a better solution, because this is a use case I'd like to support

[NoahY43619]

Hey @fubuloubu ,

You're right about the name() and symbol() typoes - sorry about that.

Aside from the transaction lag, your solution would work perfectly in my case. However, when trying to apply it to the full API rather than the toy example, I get the most puzzling loop of errors I've seen in a while:

  File "{some path}/python3.9/site-packages/ape/api/networks.py", line 233, in __getattr__
    return self.get_network(network_name.replace("_", "-"))
  File "{some path}/python3.9/site-packages/ape/api/networks.py", line 387, in get_network
    if network_name in self.networks:
  File "/usr/lib/python3.9/functools.py", line 969, in __get__
    val = self.func(instance)
  File "{some path}/python3.9/site-packages/ape/api/networks.py", line 176, in networks
    for _, (ecosystem_name, network_name, network_class) in self.plugin_manager.networks:
  File "{some path}/python3.9/site-packages/ape/plugins/__init__.py", line 191, in __getattr__
    for result in results:
  File "{some path}/python3.9/site-packages/ape_ethereum/__init__.py", line 27, in networks
    yield "ethereum", network_name, create_network_type(*network_params)
  File "{some path}/python3.9/site-packages/ape/api/networks.py", line 885, in create_network_type
    class network_def(NetworkAPI):
  File "pydantic/main.py", line 148, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/utils.py", line 657, in pydantic.utils.smart_deepcopy
  File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.9/copy.py", line 205, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/usr/lib/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.9/copy.py", line 137, in deepcopy
    d = id(x)

This block repeats until triggering a RecursionError: maximum recursion depth exceeded while calling a Python object error.

There's a good chance the full API is doing something that conflicts with your solution. However, the thing is a few hundred lines long, so dumping it all here would basically be throwing a haystack at you hoping you'll find the needle. Please feel free to suggest ape components whose use by the API might trigger the issue so that I can look into it/share specific bits of code.

As for discussing the use case on Discord: I'd be glad to. Thanks a lot for being so supportive.

[fubuloubu]

Yeah, this is difficult to debug without any further knowledge of what the code is doing

Happy to jump into our discord, I'll be in the Helpdesk chat for the next 45 mins if you are around

[NoahY43619]

Sorry @fubuloubu , found your reply too late to catch you on Discord.

Turns out the weird loop actually happens whenever I'm trying to run any script using ape run. I am at a loss as to what is happening.

I've moved the discussion on competitively-dumb-questions to see if anyone has ever experienced anything of the sort.

[NoahY43619]

Bit of progress, in a way: just running the following block of code as copied from the docs:

with networks.ethereum.ropsten.use_provider("infura") as provider:
    ecosystem_name = networks.provider.network.ecosystem.name
    network_name = networks.provider.network.name
    provider_name = networks.provider.name
    print(f"You are connected to network '{ecosystem_name}:{network_name}:{provider_name}'.")

Fails with error Process finished with exit code 139 (interrupted by signal 11: SIGSEGV) after hanging for ~5 sec.

[fubuloubu]

These are some bizarre behaviors I have not seen before! Let's definitely make a point to debug in discord on Monday if that would work for you. There might be some environment issues at play