Determine if we can use javascript instead of python

[johnbradley]

See changes https://observablehq.com/@hanliuxxx/andromeda_js/2

[johnbradley]

The andromeda_js notebook doesn't implement MDS dimension reduction. Instead it uses

D3 force layout to simulate the MDS dimension reduction.

Is this change in algorithm acceptable?

[johnbradley]

I used the code export option from the observablehq notebook. This exported the code with all the code relying upon the UI observablehq library instead of just the javascript code. A copy of the javascript code is viewable here: https://github.com/Imageomics/Andromeda/blob/pure-js/andromeda-ui/backend/andromeda.js.raw

[johnbradley]

The code exported wraps every piece of code in a placeholder function. I assume this is for the observablehq library, but this means each piece of code may require some modifications to re-use. For example https://github.com/Imageomics/Andromeda/blob/f14783311041c01b8c4a21a87b6d1466c6d2a571/andromeda-ui/backend/andromeda.js.raw#L622-L648

[johnbradley]

Chris:

Don't need regular drag. MDS uses MSE and D3 vector uses hooks law (linear error). Why are we using MDS anyway? High level intuition of preserving distances.

Leanna:

Is inverse MDS or MDS force? - inverse is the distance function

Dan

Need some reduction from multiple dimension to 2D. Need to be able to explain it to teachers.

Chris:

Difference loss function is linear vs square, but not something to tell teachers.

Hilmar:

Multiple distance metrics and projection metrics cross reference able in other projects. t-distributed stochastic neighbor embedding (t-SNE) would be another good option.

Dan:

Need to be able to explain why using linear vs square. Need to document when to use which.

Chris:

There are many dimension reduction methods. Each have difference emphasis. Example where user can pick the algorithm - https://github.com/infovis-vt/AndromedaJupyter-2.0

Dan:

Two levels here. Scientists knowledgable about their data they can choose. For a workshop we need to choose for them and justify how to use the option.

Elizabeth:

Squared magnifies differences.

Hilmar:

Need to decide between javascript and python for backend. Javascript solves scaling, but limits DR algorithms. Python backend requires a server to be running, but has larger ecosystem of DR algorithms.

Dan:

Can we have both?

Hilmar:

Double prong strategy viable IF javascript is strongly limited. No feature parity.

Dan:

Flexibility for research, but standard too does what if thinking.

DECISION: We will use javascript for and dimension reduction logic for datacamp and QUEST.

[johnbradley]

In our meeting today the decision was made to use MDS(python) for now.

[johnbradley]

Moving this issue back to Todo for now.

[thompsonmj]

I think we are settled on this decision now, correct? Closing for now unless others feel it needs to be revisited.