search

loculus-project / loculus

An open-source software package to power microbial genomic databases
https://loculus.org
GNU Affero General Public License v3.0
36 stars 2 forks source link

Add database debug dump endpoint to backend #1925

Open corneliusroemer opened 5 months ago

corneliusroemer commented 5 months ago

Right now to see what happened to a preprocessing pipeline upgrade requires connecting to the database. That works for now while it's easy to access the database, but in production it will be laborious to go into the database.

So it would be nice to have an endpoint that summarizes the current status of preprocessing pipeline.

In fact, we could just have a dump of relevant database tables (read only) and without personally identifiable data - this would make debugging easy while still upholding privacy.

This endpoint would be restricted to the "superuser" (for instance) but not limited to debug builds, as this endpoint would be very useful in production as well.

Originally posted by @corneliusroemer in https://github.com/loculus-project/loculus/issues/1922#issuecomment-2112868424

chaoran-chen commented 5 months ago

A "dump" endpoint sounds too broad. I could imagine making some specific information available such as the pipeline version but the inclusion of each piece of information should be a conscious act. I.e., if a table gets a new column, it should not be surfaced by default but only if we explicitly declare it somewhere in the code.

corneliusroemer commented 5 months ago

Sure, I just mean export things that are useful for debugging, only to developers - the thing that one could get by logging in to the database (in a complicated way).

I wouldn't be too worried about a new column appearing as that dump or whatever we call it would only be accessed by privileged users - the same group more or less that can anyways log into the database.