Feature/history phase1

[mikekucera]

General information

Associated issues: #122, #123

Checklist

Author:

• [x] One or more reviewers have been assigned.
• [x] Automated tests have been included in this pull request, if possible, for the new feature(s) or bug fix.
• [x] The associated GitHub issues are included (above).
• [x] Notes have been included (below).

Reviewers:

• [ ] All automated checks are passing (green check next to latest commit).
• [ ] At least one reviewer has signed off on the pull request. Reviewers have two business days to review the pull request, after which the author may merge in the pull request unilaterally.

Notes

• Usage Details:
  • Adds a History Panel that allows the user to take a snapshot of the network. Each snapshot saves the complete state of the network at the moment the button is pressed.
  • Snapshots can be deleted or restored. Restoring a snapshot overwrites the network with the contents of the snapshot.
  • Restoring a snapshot automatically takes a new snapshot before the restore.
  • Restoring a snapshot calls fit content. This is because the positions of the nodes could be offscreen.
• Implementation details:
  • Each snapshot is saved as a new document in couchDB. The contents of the document is just a copy of the network document, with two extra fields { snapshot: true, timestamp: 1234556677, ...data }.
  • When the couchDB database is created for the network, a design document is also created. For now the design document just has one "view" which is used to query for a list of all the snapshots. Additional views and filters can be added for other purposes in the future.
  • The CytoscapeSyncher was modified as follows:
  • The replication calls use the network document ID as a filter, that is the only document that is replicated. This avoids having all the snapshot documents replicated to the client for no reason.
  • Note, I could not get PouchDB to accept a proper filter from the design document. I followed the documentation to a T, and tried a bunch of things, and I couldn't get it to work. I'm fairly certain that the problem is not on the CouchDB side, because I was able to call the filter from the http API and it worked fine. I assume there is something wrong with PouchDB.
  • Added a listener to the server side change feed. This detects new/deleted shapshot documents. It then fires an event through the emitter. This is used to update the History Panel API if snapshots are taken in another tab.
  • Added an API endpoint for the history panel. It has APIs for
  • Get a list of snapshots
  • Create a snapshot
  • Delete a snapshot
  • Restore a snapshot
  • Added API endpoint for getting a thumbnail image of a network.
  • This works for both the network document and for snapshots.
  • Uses cytosnap to create the image
    • Note: I had to manually calculate all the styles for the entire network and create a big stylesheet where each selector is a node/edge ID. This is because for technical reasons the functions we use on the client side to allow for dynamic styles does not work with cytosnap.
    • Requesting a thumbnail for a snapshot also saves the image as an attachement to the snapshot document. Subsequent requests for the thumbnail cause it to be retrieved from the database.
  • Moved the express server initialization for the test suite to a global root.js file. Had to do this because the syncher tests were failing because they came after the history tests and the history tests were shutting down express.

[mikekucera]

I don't know why these checks are failing. The build and test suite run clean on my machine.

[maxkfranz]

Some initial comments:

Note, I could not get PouchDB to accept a proper filter from the design document. I followed the documentation to a T, and tried a bunch of things, and I couldn't get it to work. I'm fairly certain that the problem is not on the CouchDB side, because I was able to call the filter from the http API and it worked fine. I assume there is something wrong with PouchDB.

Did you use a filter on an existing Pouch object, or on a new, separate one? What kind of filter were you applying?

I don't know why these checks are failing. The build and test suite run clean on my machine.

The package-lock.json file isn't in-sync with the package.json file. The simplest way to solve this problem is to delete package-lock.json, run npm i again, and then commit the fresh lock file. This may update some minor/patch versions of some dependencies, though usually that's OK.

[maxkfranz]

Aside: I think that bug with the extra node in the snapshot images would be fixed by upgrading to the latest version of edgehandles: https://github.com/cytoscape/cytoscape.js-edgehandles

[mikekucera]

Aside: I think that bug with the extra node in the snapshot images would be fixed by upgrading to the latest version of edgehandles: https://github.com/cytoscape/cytoscape.js-edgehandles

Did you run into this bug? I made some changes to how the snapshot doc was created, then I didn't see this bug anymore, so I was hoping it was fixed.

[maxkfranz]

Aside: I think that bug with the extra node in the snapshot images would be fixed by upgrading to the latest version of edgehandles: https://github.com/cytoscape/cytoscape.js-edgehandles

Did you run into this bug? I made some changes to how the snapshot doc was created, then I didn't see this bug anymore, so I was hoping it was fixed.

I didn't run into it now. Not sure how to reproduce it. It just came to mind

[mikekucera]

Did you use a filter on an existing Pouch object, or on a new, separate one? What kind of filter were you applying?

When calling sync() or replicate() on a pouch object, you can pass the name of a filter in the options object. The filter is a javascript function that is part of the design doc. I could not get this to work on the client side, it would just filter out everything.

Here's the documentation I was following... https://pouchdb.com/2015/04/05/filtered-replication.html https://pouchdb.com/api.html#replication (section on Filtering Options)

[mikekucera]

I didn't run into it now. Not sure how to reproduce it. It just came to mind

Yeah, it was very intermittent for me, happening maybe 1/50 snapshots. I'm not sure what was causing it, but I refactored some of my code and then the bug went away. Let me know if you ever see it again.

[maxkfranz]

I didn't run into it now. Not sure how to reproduce it. It just came to mind

Yeah, it was very intermittent for me, happening maybe 1/50 snapshots. I'm not sure what was causing it, but I refactored some of my code and then the bug went away. Let me know if you ever see it again.

I think updating edgehandles should definitely remove the possibility of it happening, in any case. I think it should just be an npm i cytoscape-edgehandles@4, in which case it may be worthwhile including the upgrade in this PR.

Edit: edgehandles, not cytosnap

[maxkfranz]

Did you use a filter on an existing Pouch object, or on a new, separate one? What kind of filter were you applying?

When calling sync() or replicate() on a pouch object, you can pass the name of a filter in the options object. The filter is a javascript function that is part of the design doc. I could not get this to work on the client side, it would just filter out everything.

Here's the documentation I was following... https://pouchdb.com/2015/04/05/filtered-replication.html https://pouchdb.com/api.html#replication (section on Filtering Options)

Good to know that the functions don't work (maybe a serialisation issue?). The doc IDs approach would probably be faster / less expensive anyway, right?

[maxkfranz]

Changed PR to merge onto new dev branch, as discussed.

[mikekucera]

I think updating edgehandles should definitely remove the possibility of it happening, in any case. I think it should just be an npm i cytoscape-edgehandles@4, in which case it may be worthwhile including the upgrade in this PR.

The extra node was actually showing up in the snapshot document in couchDB! There was an extra node, with only position info and no other attributes, at the bottom of the document below all the edges.

I don't know how edgehandles might be related to this issue, but I'm glad to update the version if that might be the cause.

[maxkfranz]

I don't know how edgehandles might be related to this issue, but I'm glad to update the version if that might be the cause.

The extension used to use elements for the handle (red dot) and preview edges so that previews would be drawn properly like the real, resultant elements. The update is better in that it doesn't use a node for the handle. The preview edge still exists, but that shouldn't be a problem. We only use the draw mode feature in CE (without the handle), so I don't think there should be any issues with the removal of some of the edgehandles API in the update