This pr was created to fix the broken update() method ( #37 ).
The main problem with the bug was the extensive use of global
state which made updating a torrent impossible. By storing
the old state in snapshots, snapshots can be swapped out
when an updated one is available.
A snapshot stores all the state and information for a torrent.
When update() is called and the website has changed, a new
snapshot is created. A snapshot contains the torrentMeta,
manifest, and state related objects like the torrents chunk store
and priority store. Be moving all torrent related global state into
snapshots, multiple snapshots can be used by multiple instances
without interfering with each other. This also allows for a given
instance to update to a new snapshot by simply swapping out the
underlying snapshot with the updated one.
./lib/seeder.js
This file was originally the downloader but the name was changed
to reflect what it actually does since it does more than just
download, and seeding implies downloading.
All of the module's global state was moved into the Seeder
class, and the seeder supports seeding multiple snapshots
simultaneously. Seeding multiple snapshots was a necessity
for update() to work properly since multiple instances can
be one different snapshots at the same time.
Tests
To be able to test the update() function required generating
two torrents so update() could switch between the two. These
two torrents are in test/www/v1 and test/www/v2.
Future work
There is currently no cleanup of stale torrents. The
main problem with this is there is no straight forward way for
determining when a torrent is no longer in use.
When updating to a new snapshot, the new snapshot is downloaded
from scratch despite the high likelyhood that the old snapshot
already contains most of the data.
closes #37
This pr looks more daunting than it really is. Most of the code changes were moving files/functions around
This pr was created to fix the broken update() method ( #37 ). The main problem with the bug was the extensive use of global state which made updating a torrent impossible. By storing the old state in snapshots, snapshots can be swapped out when an updated one is available.
A snapshot stores all the state and information for a torrent. When update() is called and the website has changed, a new snapshot is created. A snapshot contains the torrentMeta, manifest, and state related objects like the torrents chunk store and priority store. Be moving all torrent related global state into snapshots, multiple snapshots can be used by multiple instances without interfering with each other. This also allows for a given instance to update to a new snapshot by simply swapping out the underlying snapshot with the updated one.
./lib/seeder.js
This file was originally the downloader but the name was changed to reflect what it actually does since it does more than just download, and seeding implies downloading.
All of the module's global state was moved into the Seeder class, and the seeder supports seeding multiple snapshots simultaneously. Seeding multiple snapshots was a necessity for update() to work properly since multiple instances can be one different snapshots at the same time.
Tests
To be able to test the update() function required generating two torrents so update() could switch between the two. These two torrents are in test/www/v1 and test/www/v2.
Future work
There is currently no cleanup of stale torrents. The main problem with this is there is no straight forward way for determining when a torrent is no longer in use.
When updating to a new snapshot, the new snapshot is downloaded from scratch despite the high likelyhood that the old snapshot already contains most of the data.
closes #37
This pr looks more daunting than it really is. Most of the code changes were moving files/functions around