Allow locking down the top level of a directory in Linksharing

[wthorp]

If we did a .nolisting file in a directory, we'd need to check it for every directory, which seems expensive, even if we do cache it. My gut is that we should add this to the [already cached] access credentials.

[stefanbenten]

This is very short minded and only works if we control the linksharing services globally. Anyone could simply uncomment that check and start listing things. This needs a proper macaroon definition in my mind.

[wthorp]

Rephrasing Stefan, we could create EG a noListPaths caveat for macaroons. Customers could continue to use their own unrestricted macaroon, but distribute a restricted macaroon for use with semi-trusted Linksharing services.

[wthorp]

JT agrees with Stefan and there are other interesting things in the roadmap surrounding macaroons. I think it's probably worth doing the slightly harder way here, even if the real use case doesn't exist today. I hope to argue that we need the shared macaroon stories that would support Stefan's use case.

[shaupt131]

@egonelbre to gather some thoughts that @wthorp can get in front of the customer.

[egonelbre]

There are a few ideas I can come up with.

Separate service

Instead of using linksharing directly, there can be a separate service that creates macaroons that have a low expiration time.

The benefit for this is that there's no need for Satellite side changes. Similarly the service could have many other rules that would be difficult to handle on the satellite. e.g. per user permissions.

The main con is that this requires a separate service that manages the root macaroons.

If the macaroons aren't too long (https://stackoverflow.com/a/812962/192220) it could be implemented as a redirect service.

The rough idea of the server implementation is:

rootAccess, err := uplink.ParseAccess(...)

func ServeHTTP(w http.ResponseWriter, r *http.Request) {
    access, err := rootAccess.Share(uplink.Permission{
        AllowList: true,
        AllowDownload: true,
        NotAfter: time.Now().Add(5*time.Minute),
    }, uplink.SharePrefix{
        Bucket: "bucketName",
        Prefix: r.URL.Path, // transform this as necessary
    })
    // handle error

    //note: this is not the correct url, just a proof of concept
    http.Redirect(w, r, "https://linksharing:123/" +  access.Serialize() + "/xyz", http.StatusTemporaryRedirect)
}

Tag matching on the satellite

An alternative approach would be to introduce a new macaroon mechanism where satellite verifies whether a "tag" on object matches. These tags would be an INT4[], where user can choose what they do.

For example when a object has a tag:1 it could mean that it's public. Then the macaroon could limit with:

rootAccess.Share(uplink.ReadOnly(), uplink.SharePrefix{
    Bucket: "bucketName",
    Prefix: r.URL.Path,
    Tag: 1,
})

Then uplink library can be used to update the respective tags as needed.

One drawback with this "tag" approach is that it wouldn't mesh well with different libraries using uplink, since the meaning of a "tag" could mean different things.

It might make sense to rename that to "user" or "group" to make it clear, that it can be used to implement user/group like permission system.

Obviously one major drawback is that this information won't be encrypted.

E-Tag like matching on linksharing

Instead of storing that information in a custom field, it would be possible to add this into custom metadata. e.g. "hidden" and then the linksharing can hide and not allow access to those objects that have that specific metadata.

Of course, updating custom metadata requires more operations and ends up needing more data. Both gateway and linksharing probably should respect that metadata.

Path matching on the satellite

To suppress the most top level information it would be possible to implement a new custom prefix, but this doesn't seem like it would be that widely useful.

[shaupt131]

Discussion surrounding this topic: https://storj.slack.com/archives/C02U4NTP59N/p1644604242448919