Interface extending other interface

Suppose I want the following:

MySuperInterface
MySpecificInterface extends MySuperInterface
MySpecificImplementation implements MySpecificInterface

How does one achieve this with the implements module? Currently PyLint raises an error when I make SpecificInterface extend both SuperInterface and Interface (From the implements package).

I figured it out. I can get it to work by using the following approach:

from implements import Interface

class MySuperInterface(Interface):
    def myExampleFunction(self, arg1, arg2: str):
        pass

And

from .mySuperInterface import MySuperInterface
from implements import implements, Interface

@implements(MySuperInterface)
class MySpecificInterface(Interface):
    pass

And

from .mySpecificInterface import MySpecificInterface
from implements import implements

@implements(MySpecificInterface)
class MySpecificImplementation():
    def myExampleFunction(self, arg1, arg2: str):
        print("yeet")
    def extraFunction(self, arg1):
        print(arg1)

Now it gives me an error that MySpecificInterface does not implement MySuperInterface which is the behaviour I expect.

@ShariJoosten. The two comments actually do different things. I'd like to provide some clarity to you on what is happening here.

Lets start with comment No.2:

Comment No.2 :: Code snippet No.2

@implements(MySuperInterface)      <--- this says: satisfy MySuperInterface
class MyInterface(Interface):      <--- but, MyInterface doesn't have 'myExampleFunction'
    pass

^^^ So, this will raise an error

"Now it gives me an error that MyInterface does not implement MySuperInterface which is the behaviour I expect." ^^^ This error is actually being thrown by code snippet no.2 and not by code snippet no.3

Comment No.2 :: Code snippet No.3

@implements(MyInterface)        <--- MyInterface is actually an empty class, so nothing is being enforced
class MyClass():
    def myExampleFunction(self, arg1, arg2: str):
        print("yeet")
    def extraFunction(self, arg1):
        print(arg1)

I'm assuming that code snippet no.3 comes after snippet no.2 in the code, in which case no.3 has not yet executed.

Snippets in comment no.2 aren't actually representative of the intent expressed in comment no.1. So, below I'll try to explain what I've understood from comment no.1, and how that could be achieved with implements.

Intended Design:

MySuperInterface is the parent interface. It has one method myExampleFunction
MySpecificInterface exends MySuperInterface. It adds a new method extraFunction
MySpecificImplementation implements MySpecificInterface. So, it must implement both myExampleFunction and extraFunction.

Here is how this would look

Example 1: extraFunction defined in the implementation and not enforced by any interface

class MySuperInterface(Interface):
    def myExampleFunction(self, arg1, arg2: str):
        pass

class MySpecificInterface(MySuperInterface):    <--- note the inheritance. No use of @implements here
        pass                                    <--- this class is as good as MySuperInterface

--- implementation ---

@implements(MySpecificInterface)
class MySpecificImplementation:
    def myExampleFunction(self, arg1, arg2: str):
        print("yeet")
    def extraFunction(self, arg1):            <--- extraFunction defined in the implementation. Not enforced
        print(arg1)

Example 2: extraFunction defined in the extended MySpecificInterface and both methods enforced

class MySuperInterface(Interface):
    def myExampleFunction(self, arg1, arg2: str):
        pass

class MySpecificInterface(MySuperInterface):    <--- note the inheritance. No use of @implements here
    def extraFunction(self, arg1):              <--- only define extraFunction here
        pass

--- implementation ---

@implements(MySpecificInterface)
class MySpecificImplementation:
    def myExampleFunction(self, arg1, arg2: str):
        print("yeet")
    def extraFunction(self, arg1):
        print(arg1)

@pshirali Let me first thank you for your excellent explanation.

"This error is actually being thrown by code snippet no.2 and not by code snippet no.3" ^^^ I was aware that the error was thrown from code snippet no.2, but at that time I was confusing how extends works when you're talking about interfaces. I thought when interface B extends A, all methods of A should be repeated in B. But I learnt now that this is wrong. All methods are already present in A, so no need to mention them again in B. In B only the additional methods on top of A should be mentioned.

"Now it gives me an error that MySpecificInterface does not implement MySuperInterface which is the behaviour I expect." ^^^ So what I said here was wrong, this is not the behavior one should expect, see the explanation above.

"Snippets in comment no.2 aren't actually representative of the intent expressed in comment no.1. So, below I'll try to explain what I've understood from comment no.1, and how that could be achieved with implements." ^^^ I agree. What I ended up making was an interface implementing yet another interface, which is not what I intended to make: an interface extending another interface.

About the intended design, I added extraFunction as an extraFunction in the implementation class which I ended up naming MyClass. My example was bad because it makes it seem like the MySpecificInterface adds nothing, but in the application I'm building MySpecificInterface will add multiple new methods on top of MySuperInterface (Which I didn't show in my example), and the implementation will implement all of these methods and add it's own additional methods as well (Which I did show in my example). Next time I'm making an example I'll be consistent with this, giving MySuperInterface one method named myExampleFunction, then MySpecificInterface adds one method to that named myExtraInterfaceFunction and lastly MySpecificImplementation adds another method named myExtraImplementationFunction.

So my intended use was Example 1. Tomorrow I'll be fixing my code so that I indeed extend the interfaces instead of implementing them.

Thank you very much for your help, and I understand everything perfectly clear now. The only thing I've left to add is that in your comment it seems like both Example 1 and Example 2 will fulfil the intended design as described by you, but only Example 2 fulfils the intended design as described by you. Example 1 fulfils the design I desired.

Edited all my comments to use @pshirali 's naming scheme which was simply better. I did this in case if anyone else is wondering how to have one interface extending the other using the implements package and ended up on this Issue Page that they'd have an easier time understanding.

Thanks @ShariJoosten for the additional details on what you intended to build.

There are two ways to go about what you've described. One is by inheriting and combining interfaces. The other is by enforcing segregated interfaces using multiple @implements decorations.

Consider these as the base requirement:

class Reader(Interface):
    def read(self, size: int) -> str:
        pass

class Writer(Interface):
    def write(self, data: str) -> int:
        pass

Combining Interfaces through inheritance

Interface classes are regular python classes. So, the natural mechanisms of inheritance hold true, and these can be used to extend interface classes. You can override signatures, or create mixins to combine multiple interfaces.

class ReadWriter(Reader, Writer):  <--- inherits both 'read' and 'write' methods
    pass

@implements(ReadWriter)
class MyIOImplementation:     <--- this must implement both 'read' and 'write' methods from the combined ReadWriter
        ...

Enforcing multiple segregated interfaces (preferred)

An implementation class can have multiple interfaces enforced on it. The interfaces themselves could be small and segregated (with just a few methods). For example Reader and Writer have only one method each.

@implements(Reader)
@implements(Writer)
class MyIOImplementation:
    pass

You can use either of the above methods, or a combination of both. They all work fine.

However combining interfaces through inheritance will result in large classes, and you may end up creating far too many derived/extended classes for each combination; each of which represents all methods that an implementation must implement.

This may be prone to more human errors:

Accidental overrides with MRO or having a deep inheritance tree
Very long combinatorial interface class names: MySuperSpecialisedAwesomeClassThatRocks
Less visibility when an interface in a base class changes. The implementation has only one decorator, but, one needs to know the entire interface class-hierarchy to mentally grasp what methods exist in it.

Making multiple decorator calls and having short segregated interfaces is better.

You won't have to build specialised classes for every combination.
Less human errors as you aren't dealing with a deep tree of inherited classes, or long combinatory class names.
The declaration of multiple interfaces that each class implements is explicit and very readable close the the implementation.

I fully understand your concerns, yet I've decided to combine interfaces through inheritance. I'm building a data access layer for a large application project, and the exact interfaces I'll end up using are:

IRepository (An interface containing three methods: exists, save & delete) I<Entity>Repository, as example, if my entity is an car, it will be ICarRepository (Contains additional specific methods like getCarById) CarRepository (located in the folder of my DB implementation, if I have multiple DB implementation, this file will have the same name, but be located multiple times in different folders with different implementations. This file structure is not shown in the example below.)

I fully agree that if as a programmer, when looking at CarRepository, it's frustrating to follow the entire interface hierarchy up to see all the methods I have to implement. I'd prefer just looking at ICarRepository and seeing all required methods there already (Including the three from IRepository).

I personally don't want CarRepository to implement both IRepository and ICarRepository because I find that ugly, I can't really explain it better, sorry about that.

But I thought I could solve this by defining ICarRepository as followed:

from .IRepository import IRepository
from .entity import Entity
from implements import implements, Interface

class ICarRepository(IRepository):    
    def exists(self, entity: Entity):
        pass
    def getOwner(self, entity: Entity):
        pass

I did realize that in this case, I get to completely overwrite the exists function in ICarRepository (It doesn't have to match the exists method in IRepository anymore), which I find a shame. Because in this case having IRepository is useless unless ICarRepository doesn't copy the exists method, but then you have the issue again which I was trying to solve: Having to look at both IRepository and ICarRepository when implementing CarRepository to see the required methods. So so far I see that the only options I have are either accepting that programmers have to look up the entire interface hierarchy, or do what @pshirali said, having CarRepository implement both IRepository and ICarRepository.

Lastly, I noticed another issue with using this module. Lets suppose I have a class named Car which is my entity, and it extends the general class Entity. In IRepository I'd specify that each repository should contain an exists method with as arguments self and entity: Entity. But in CarRepository I'd prefer to specify which entity I'm talking about, I'd prefer to implement the exists method with the arguments self and car: Car. In Java this would have worked, however I now get the error that I didn't implement the exists method with the exact same arguments, because I'm using a more specific class named Car instead of the base entity class named Entity. When in IRepository I define the arguments of the exists method only as self and entity, I still get the same error because in CarRepository the arguments self and entity: Entity don't match one to one. Is this something done by design or will this be improved upon in the future?

@ShariJoosten :)

First, let me thank you for spending your valuable time engaging in a fruitful conversation around how you use implements, and sharing scenarios which have been challenging. It is through conversations like this that we are able to improve the library and address limitations, best-practices etc.

Please allow me some time to respond to your queries with some examples.

@ShariJoosten, I'll express what I've understood by building upon the classes you've defined:

Problem definition

IRepository is a base interface to many I<Entity>Repository interfaces. Methods in IRepository define signatures with Entity objects in their type-hints (purely for reference). Entity is again a base for many specific entities like Car, Van etc.
Lets have two extended I<Entity>Repository interfaces: ICarInterface and IVanInterface. Each of these are expected to operate with Car and Van objects (extensions of the base Entity themselves)

And what you are looking for is a mechanism to define extension boundaries. Not every method in the base IRepository is expected to be overridden. Two cases for methods arise here:

IRepository defines methods which are meant to be inherited and used as is. Signature does not change. Extended interfaces aren't expected to override them (but overriding is possible, as Python allows it)
IRepository defines methods, but sets an expectation that there MUST be an extension to IRepository where the extended interfaces MUST override the base method, respecting some rules set by the base interface IRepository.

Example of rules for case (2): The base IRepository defines meta-method exists(self, entity:Entity) where the expectation is that:

Extended interfaces must implement a method called exists, overriding that of the parent IRepository
The signature from IRepository must be respected, except specific parts of the args must be mandated for change: Example: Entity must be replaced by Car, Van etc, in respective ICarInterface and IVanInterface

Could implements solve for this?

What is defined above is a very complex interface extension mechanism. implements does not support his, and instead sticks to very readable method+signature based enforcement. It is what-you-see-is-what-you-get, with no magic.

Supporting mechanisms to provide rich interface extension increases the complexity of interface design by a large factor, and will make implements a very complex library (with a lot of tests!). But I'll leave this idea on the table for others to ponder over this problem as well.

Thinking aloud; in order to support something like this, implements will have to support interface/method definition in a form that caters to additional metadata & rules. The rules and metadata could end up supporting/defining:

A partial (/incomplete) interface. Where, the interface MUST be extended in order to be used. (Example: IRepository would serve as a partial interface, defining rules, but must be used in conjunction with I<Entity>Repository interfaces). @implements(IRepository) will raise an error as it is a partial interface.
Methods allow rules for how some arg types can/must be overridden by extensions. Example: For the above scenario: arg-types (in case of classes) can only be overridden by subclasses. (Example: Entity must be replaced by subclass of Entity, like Car, Van)

def partial_interface():
    '''
    Decorator which wraps an interface and tags it as a partial interface. A partial interface
    can't be used with '@implements' decorator. A subclass of this which overrides methods
    having rule conditions will serve as the actual interface. 
    '''

def must_satisfy(arg_dict):
    '''
    Decorator which maps objects (type-hints) found in args against a callable which decides
    whether the expected override rule is satisfied. The signature for this callable must be (this, other),
    where 'this' is the object found in the arg as a type-hint, and 'other' is what it is compared
    against in the extended interface.
    '''

def is_subclassed(this, other):
    '''
    An example of a callable that verifies forced subclass on arg-types.
    Returns True iff 'other' is a derived class of 'this'
    Note: This is different from issubclass(this, other) which can return True for issubclass(this, this)
    '''
    return <bool>

@partial_interface                    # with this decorator, @implements(IRepository) will raise errors
class IRepository(Interface):
    @must_satisfy({
        Entity: is_subclassed
    })
    def exists(self, e: Entity):
        pass

class ICarRepository(IRepository):    # this is a usable interface with @implements
    def exists(self, e: Car):
        pass

----

@implements(ICarRepository)           # magically ensures that expected rules are followed (*see below)
class MyCarRepository:
    ...

# *How will _implements_ decorator check this?
# 1. Go through interface class hierarchy to find the immediate partial interface (if any)
# 2. Fetch methods decorated with @must_satisfy. Enforce the conditions on the actual interface

PS: The stuff above is a half-baked representation of what it might take to do something like this. It has not been thought through fully. There are (and will be) many corner cases. (Example: How to handle rules between multiple partial interfaces or ones inheriting others). The code above is just for future reference and food-for-thought. :)

Important factors to consider:

Complexity for the user: implements, at the moment has small API footprint (just two objects!) and no magic. The code is easy to grasp.
Proactive handling of corner cases: to prevent unintentional behaviour by early detection of faulty use
Testing: can get complex (both short term and maintenance)
Documentation: More literature to educate the user on how to use implements

Support from other libraries

I had a quick look at other libraries:

zope.interface does not support this (ref: Interface Inheritance, foot note no.3)
interface does not support this (ref: Interface subclassing > Warning)

The onus is on the developer to ensure that method signatures across extended interfaces do not break implementations. The common case here would be extensions which are mutually exclusive, and serve to add new methods.

What could be done with implements in its current state (.. and by no means a full solution to the problem)

In the case of IRepository defining exists, save, delete, I'd assume that the signatures for these methods will involve the use of extended entities like Car, Van in their extended interfaces. So, there'd be value in keeping IRepository light, with only those methods (if any) which don't have a signature change in I<Entity>Repository. The I<Entity>Repository classes could be richer in the actual methods & signatures that they locally define, which are enforced across multiple DB implementations. Ensuring commonality in pattern between I<Entity>Repository interfaces is not supported by implements.

With respect to using multiple @implements decorations (for those who are still ok with this as an option), one way to reduce ugliness is to name interfaces based on the utility/methods they provide and keep them mutually exclusive.

Interfaces in Go serve as a good reference on how to go about this. Ref: Effective Go and this talk

Interface for single methods could be defined; such as Exister, Saver, Deleter, and combined to a single interface if they are typically used together often (say IO). An unrelated method like getOwner could be in an OwnerGetter interface.

Example:

Implementing IRepository and ICarRepository could be less readable because:

@implements(IRepository)       # we know this is a baseclass: but we don't know what methods are defined
@implements(ICarRepository)    # name seems related to IRepository, but we don't know what this adds
class CarRepository:
    ...

Instead, a (subjectively) less uglier version focuses on interfaces built on methods/operations are being implemented v/s macro-interface classes.

class ICarRepositoryIO(ICarExister, ICarSaver, ICarDeleter):
    pass

class ICarOwnerGetter(Interface):
    def getOwner(self, owner: Car):
        ...

@implements(ICarRepositoryIO)    # hints that CarRepository does IO: exists, save, delete
@implements(ICarOwnerGetter).   # hints that CarRepository has a getOwner
class CarRepository:
    ...

You had the problem definition spot on. I understand that what I requested is difficult to build and will make implements indeed a very complex library. I just expected something like this to exist in Python since it already exists in so many other languages, like C# and Java.

The possible solution you described with the is_subclassed callable in combination with the decorators must_satisfy and partial_interface sounds plausible to me, but keep in mind that Python isn't exactly my expertise. I will however say that this does sound like a lot of hassle: In C#, or Java, all of these things work out of the box by simply using the implements and extends keywords. Personally as an end-user of the implements project, if I'd see that that amount of hassle is required to get what I desire up and running, I'd seriously consider just accepting the fact that other programmers can make mistakes and that we should prevent this by code reviewing and proper documentation.

Thanks for pointing out that none of the other libraries meet my requirements.

The solution I've chosen is to simply not use the IRepository interface at all. I will have an ICarInterface as example, which will then declare the methods exists, save and delete. I can then immediately declare their arguments as car: Car, which would then force all the implementations to implement these methods thanks to the implements module. I haven't chosen to create one interface for every method because I'd end up with hundreds of interfaces, and each <Entity>Repository implementation class would have dozens of implements decorators, which I personally am not a fan of, especially because a programmer can then simply remove one of these implements decorators and solve his problem that way. Using my solution there will be a single implements decorator pointing to the I<Entity>Repository interface, and then the programmer will have to check himself which methods are present in this interface to learn what he has to implement. When a programmer creates a new entity and an matching I<Entity>Repository, he should remember to add the exists, save and delete methods himself because this will be written in the documentation of the project. If he doesn't remember this, hopefully the programmer code reviewing will notice his mistake. It's not ideal, but it's the best I can do.

Thank you for @pshirali for being the most helpful dude I've met on any Git Repository so far.

ksindi / implements