Open florian6973 opened 4 months ago
@florian6973,
we will take a look at this soon. We are in the middle of a spikeinterface hackathon, but super curious about this. It is a little hard for me to read the code (without having a nice diff view). Could you also post the same code with comments on the lines you changed to make comparison a bit easier. If we haven't responded by next week please ping us again!
Thanks for your reply!
Sure, here are some more details:
$$\begin{array}{c|ccc} & b_1 & b_2 & b_3 \ \hline a_1 & 0.4 & \textbf{0.6} & 0.6 \ a_2 & \textbf{0.6} & 0.6 & 0.6 \ a_3 & 0.6 & 0.6 & \textbf{1} \end{array}$$
$$\begin{array}{c|ccc} & c_1 & c_2 & c_3 \ \hline b_1 & \textbf{1} & 0.1 & 0.1 \ b_2 & 0.1 & \textbf{1} & 0.1 \ b_3 & 0.1 & 0.1 & \textbf{1} \end{array}$$
$$\begin{array}{c|ccc} & c_1 & c_2 & c_3 \ \hline a_1 & \textbf{1} & 0.1 & 0.1 \ a_2 & 0.1 & \textbf{1} & 0.1 \ a_3 & 0.1 & 0.1 & \textbf{1} \end{array}$$
BaseMultiComparison
to reflect this particular situation, and check if we obtain in the end the true optimal matching $(a_1, b_1, c_1)$, $(a_2, b_2, c_2)$, $(a_3, b_3, c_3)$. Please find the diff below. Note that I do not need the whole comparison matrix given the way the graph is built, and I assume the match_score is low enough:
def __init__(self):
self.name_list = ['a', 'b', 'c']
self.object_list = ['1', '2', '3']
# def _compare_ij(self, i, j):
# raise NotImplementedError
# def _populate_nodes(self):
# raise NotImplementedError
def _populate_nodes(self):
for name in self.name_list:
for unit_id in self.object_list:
self.graph.add_node((name, unit_id))
# def _do_comparison(
# self,
# ):
# # do pairwise matching
# if self._verbose:
# print("Multicomparison step 1: pairwise comparison")
# self.comparisons = {}
# for i in range(len(self.object_list)):
# for j in range(i + 1, len(self.object_list)):
# if self.name_list is not None:
# name_i = self.name_list[i]
# name_j = self.name_list[j]
# else:
# name_i = "object i"
# name_j = "object j"
# if self._verbose:
# print(f" Comparing: {name_i} and {name_j}")
# comp = self._compare_ij(i, j)
# self.comparisons[(name_i, name_j)] = comp
def _do_comparison(
self,
):
# do pairwise matching
if self._verbose:
print("Multicomparison step 1: pairwise comparison")
self.comparisons = {
('a', 'b'): {
'1': ('2', 0.6), '2': ('1', 0.6), '3': ('3', 1.0)
},
('b', 'c'): {
'1': ('1', 1.0), '2': ('2', 1.0), '3': ('3', 1.0)
},
('a', 'c'): {
'1': ('1', 1.0), '2': ('2', 1.0), '3': ('3', 1.0)
}
}
def _do_graph(self):
# ...
# for comp_name, comp in self.comparisons.items():
# for u1 in comp.hungarian_match_12.index.values:
# u2 = comp.hungarian_match_12[u1]
# if u2 != -1:
# name_1, name_2 = comp_name
# node1 = name_1, u1
# node2 = name_2, u2
# score = comp.agreement_scores.loc[u1, u2]
# self.graph.add_edge(node1, node2, weight=score)
for comp_name, comp in self.comparisons.items():
for u1 in comp.keys():
u2 = comp[u1][0]
if u2 != -1:
name_1, name_2 = comp_name
node1 = name_1, u1
node2 = name_2, u2
score = comp[u1][1]
self.graph.add_edge(node1, node2, weight=score)
I hope this is clearer. I am not sure if I am fully correct, but I was trying to properly understand the multiple comparison module, so that's why I am asking.
Have a good hackathon :)
By the way, if you are in Boston at some point we could discuss it in person if needed :)
Hey @florian6973,
thanks for the well wishes. We could definitely meet at some point. If you're on the slack just send me a message. But I think @alejoe91 is better for looking over this one. I didn't work on the initial code so he would know it way better.
Thank a lot for this @florian6973, super interesting and detailed investigation. Will definitely look into this while working on #2626, please feel free to give any feedback and thoughts on the plan I posted there.
Hi!
I have been looking at the literature on Multidimensional Assignment Problems / Entity Matching to understand multiple sorting or template assignments, and I realized that the current method does not seem to always return the optimal matching.
To check my hypothesis, I modified the
BaseMultiComparison
class to create a minimal working example with the potential issue (example from https://arxiv.org/pdf/2112.03346 page 3), it can be run as a script:Therefore, according to you, is my MWE correctly adapted from the literature to the spikeinterface framework? If so, have you envisioned other methods so far or should we think more about it to solve this issue please?
Thanks!
Florent