unitaryfund / metriq-app

Web app for Metriq
https://metriq.info
Apache License 2.0
27 stars 19 forks source link

Coherence gain chart not visible on homepage #914

Closed WrathfulSpatula closed 3 months ago

WrathfulSpatula commented 4 months ago

There are three featured tasks on the homepage: quantum volume, coherence time (T2), and coherence gain. Currently, the coherence gain chart thumbnail is not visible. This is a live Charts.js chart that plots the data from the database for the task "coherence gain." There is possibly a problem with results data points that are "invalid domain," for example, negative numbers taken to a logarithmic scale, though this might not be the exact issue.

Please, diagnose the specific problem with the coherence gain thumbnail chart on the homepage, and get the chart to display again.

carrington-115 commented 4 months ago

Hi @WrathfulSpatula, This seem like an interesting task to diagnose, correct and fix the bug on the chart. I am ready to take this task. Please can you assig this issue to me?

vprusso commented 4 months ago

Hi @carrington-115

Thank you for your interest in Metriq! The way we are doing this is to assign to the hacker once the issue has been completed. By all means, please do put your hat into the ring though!

And of course, let us know if you have any questions that we can offer any direction on. Happy Hacking!

carrington-115 commented 4 months ago

Okay. Thank you @vprusso . I am currently working on it.

ljcamargo commented 4 months ago

hello @WrathfulSpatula , I'm inclined to believe that the database dump in metriq-postgre is not updated because it seems that https://metriq.info/Task/164 doesn't exists on the dump, this is needed to reproduce this error locally

WingCode commented 3 months ago

Following up the discussion here, I checked via wayback machine to find the original coherence gain chart. image

I wanted to find the chart and text description so that I can search through the commit diffs and commit logs in the metriq-postgres and find the corresponding entry which was used to populate the DB.

I searched through all branches, commits and diffs in metriq-postgres repository but I couldn't find matches for the search term Coherence gain. I assume it should have been available in DB backup file or subsequently in different versions of it.

@WrathfulSpatula / @vprusso If you could provide me any hints or pointers towards finding the original DB entry, I would be happy to take this issue to the completion.

vprusso commented 3 months ago

@WrathfulSpatula is it possible that the DB on metriq-postgres is out-of-sync with the data currently on Metriq (and hence why @WingCode isn't able to see "Coherence Gain" present in the SQL file?).

WrathfulSpatula commented 3 months ago

I'm sorry that I missed this before the end of the event. Coherence Gain is task ID 164; it's true that 164 is not in the QA database image. The reason the data are "out-of-sync" is that, well... they are "supposed to be." A QA database should usually have "mock" data rather than a live image of the database. This helps protect data privacy and integrity. It might be possible to recreate the issue with "mock" data, and this would have been preferable. Note the data in our QA image are at least "anonymized," not to expose hidden details about our users' activity and accounts in the Metriq app.

Apologies, I could have provided a sufficiently anonymized database, like what's on the repository, containing the Coherence Gain data, but I would have done so by "spinning up" the QA database image locally, manually adding the same data as task 164 on production environment Metriq, and updating the image. To be clear, we cannot publicly provide a true backup of the live database: this is by design and intent, for a normal development workflow, not "lapse" in putting live production data in the QA database.

ljcamargo commented 3 months ago

hello @WrathfulSpatula yes, I figured as much, that the db from the website was in fact production and was not the aim to share it in the repo, unluckily I couldn't figure out a way to reproduce the error mimicking the data to something similar. As the error produces a total unrender or failure of the chart component I guess is something like an out of bounds exception very specific of that data series. As I was unfamiliar with this project I did not have enough time to recreate the data from an article to see if the error reproduces. Thanks anyway. While contributing to this repo I saw much room for improvements. Let me know if you need a hand ;)

WrathfulSpatula commented 3 months ago

FWIW, I'm digging into this issue again, now, and it turns out that many different task IDs, but not every Task ID, passed to this same <FeaturedTask> slot, similarly do not display. I've poured over the various results data, and it's not even the case that we're trying to display 0 on a logarithmic axis, for example, for any of the Task IDs that don't work.

We've had significant issues and ("weird") bugs in the past few months with Charts.js, hence we've moved to transition off it completely, in the end. I'm beginning to think this could be another edge case in the Charts.js user code API that isn't even necessarily a mistake in our use of it.

But, it's worth clarifying, as a result: I seem to be able to recreate the bug just with "mock" data, and the problem isn't limited to the coherence gain task. Thanks to everyone for your willingness to try to diagnose, but I might even be glad that no hackathon contributor sunk much of their time into an issue that might turn out to be more systemic and ultimately require dropping a set of NPM dependencies.