ethpm / py-ethpm

This library is deprecated. ethPM python tooling is now located in web3.py
MIT License
24 stars 13 forks source link

Add configurable IPFS URI backend #37

Closed njgheorghita closed 6 years ago

njgheorghita commented 6 years ago

What was wrong?

Right now we only support fetching packages on IPFS, and only through the INFURA gateway.

Also extend backend so it can swap in a dummy backend that will serve content rather than monkeypatching http requests. (see piper's comment here . . . https://github.com/ethpm/py-ethpm/pull/46)

How can it be fixed?

Add configurable option to allow fetching packages from other gateways & local nodes

njgheorghita commented 6 years ago

@pipermerriam Can you expand a little more on the API you have in mind for the configurable backend? Something similar to the backends used in Populus package management?

pipermerriam commented 6 years ago
from abc import ABC, abstract_method

class BaseIPFSBackend(ABC):
    @abstractmethod
    def fetch_ipfs_uri(self, uri: str) -> str:  # or maybe `bytes`?
        """
        Returns the file contents?
        """
        pass

class IPFSOverHTTPBackend(BaseIPFSBackend):
        def __init__(self, base_uri):
            self.base_uri = base_uri

        def fetch_ipfs_uri(self, uri):
            ipfs_hash = extract_hash_from_uri(uri)
            gateway_uri = self.base_uri + '/' + ipfs_hash
            response = requests.get(gateway_uri)
            return response.content

class LocalIPFSBackend(BaseIPFSBackend):
    ... # knows how to connect to a locally running IPFS node...

class InfuraIPFSBackend(IPFSOverHTTPBackend):
    ... # knows the infura IPFS base uris...

class IPFSGatewayBackend(IPFSOverHTTPBackend):
    ... # knows the URI for the IPFS http gateway

Something like that. Then the same plumbing type code like this from py-evm https://github.com/ethereum/py-evm/blob/master/evm/db/__init__.py for loading the backend class, probably configurable via environment variable.

If we want to be really generic with URIs (which we do) we probably need a method like:

class BaseURIBackend:
    # generic backend that all URI backends come from
    def can_handle_uri(self, uri) -> bool:
        ... # returns `True` or `False` for whether this backend class can handle the given URI

# and then some glue code in the core logic that fetches URIs

def get_backends_for_uri(backends, uri):
    return tuple(backend for backend in backends if backend.can_handle_uri(uri))

# and then some code to loop over the *good* backends and return the fetched contents.  Maybe an exeption that the backends can raise if for some reason they find they can't handle it.

for backend in good_backends:
    try:
        contents = backend.fetch_uri_contents(uri)
    except CannotHandleURI:  # custom exception
        continue
    else:
        return contents
else:
    raise Something('No backends were able to fetch the URI contents')