lbryio / reflector.go

MIT License
14 stars 13 forks source link

Blobs upload proposal #48

Open nikooo777 opened 3 years ago

nikooo777 commented 3 years ago

1) send sdblob in this format:

type request struct {
    hash string
    sdblob []byte
}

2) server checks the blocklist to see if it wants the stream 3) server checks if the sdblob is already in our system 3.1) if the sd blob exists, respond with a list of blobs it needs 3.2) if the sd blob doesn't exist, store the sd blob, save the blobs in the DB with is_stored=0, respond with a list of the blobs it needs (all of them) 4) client sends one blob at a time with the following format

type request struct {
    hash string
    blob []byte
    size int    
}

5) server checks that the uploaded blob has the correct hash and size, saves the blob to S3, sets is_stored=1 in the database, responds with a success 6) optionally we should add an endpoint that accepts a list of sdblob hashes and returns whether the sdblob is needed or not and if the sdblob is already stored, a list of blobs that are missing the server thus has to implement the following endpoints: /sdblob POST accepts:

type request struct {
    hash string
    sdblob []byte
}

returns:

type response struct{
    success bool
    error *error
    type data struct {
        required_blobs []string
    }
}

/blob POST accepts:

type request struct {
    hash string
    blob []byte
    size int    
}

returns:

type response struct{
    success bool
    error *error
}

/sdblobs POST accepts:

type request struct {
    hashes []string
}

returns:

type response struct{
    success bool
    error *error
    type data struct {
        required_sdblobs []string
        required_blobs []struct{
            sdhash string
            missing_blobs []string
        }
    }
}