Open DevinBayly opened 4 weeks ago
ok, can count on what the suggestion of the vis images project is. They trained a bounding box detection system that can be pointed at a multi image figure and pull out regions and types. This is probably what we want to use https://github.com/VisImages/visimages-detection,
especially look at https://github.com/open-mmlab/mmdetection/blob/main/demo/demo.jpg and the result https://github.com/VisImages/visimages-detection/blob/main/result.jpg
it turns out that mmdet can help us get bounding boxes as well as chart types when using the visimages detection. Then setup will be handled in a different card #document vis images detection
The main goal was to take the 30 element long "chart type" estimate variable and embed that in 2 dimensions.
the result at the moment doesn't immediately convey that the technique is working, but it does mean that we are getting closer.
I'll close with the comment that the visimages paper is definitely what we needed, but that Iqbal is recommending that we still use a different similarity method for exploring the images
I'm using zotero, but I think https://ieeexplore.ieee.org/abstract/document/9729541?casa_token=cqOpWFrPXUcAAAAA:cTAZC5WGiU6_VpypUQjsprGsL1q20jN6NEkS_PHPeQm6cFKU--Cjqiu84VLx1LKkyx_1532yKQ is a great read since its quite recent and aims to provide a labeled database of figures.
we might be able to have a deep learning system trained on this and then apply it to the figures we have from princeton 2022