search

SDShare / Specification

SDShare - Protocol for the syndication and synchronisation of RDF
Other
10 stars 1 forks source link

Snapshots are rubbish #8

Closed gra-moore closed 12 years ago

gra-moore commented 12 years ago

Snapshots for anything other than a small data set just doesn't work very well. Getting and streaming GB's of data over a single http request is not scalable.

We could say that the snapshot links to another feed where each entry links to a subset of the snapshot. A client could perhaps request the number of triples in each segment.

larsga commented 12 years ago

We've been thinking about paging them. It seems a bit tricky to get right, but might work.

larsga commented 12 years ago

On the snapshot we put a link with a new relation type to a feed that has entries for pieces of the snapshot. Write verbiage for this.

Best practice: server: break up big snapshots, but not small ones. Client: if there's a broken-up version, use it, otherwise use the single-link one.

gra-moore commented 12 years ago

done now.