Open radusuciu opened 3 years ago
Yes, those column constraints were based upon approximations, but SQLite doesn't really care about data types or sizes, and it's not strict about foreign keys. They're merely hints about what the user expects. I haven't experimented with loading it into any other RBMS, but it's fixable. I can investigate the dependency graph and adapt the type modifications from you if that's okay.
What's your use-case for loading this all into Postgres?
I'm already using Postgres for other data and would prefer to keep everything under one umbrella to enable joins from the Unimod table to others, but this isn't a strict requirement by any means as it's pretty easy to go from SQLite to Postgres or to accomplish what I want in application code.
I guess if you're not interested in supporting Postgres, maybe you could indicate so in the docs?
Hi, thanks for the great project. I'm attempting to load the latest Unimod XML file into Postgres using pyteomics and have encountered a few errors that I've attempted to solve along the way. I've documented the process I've undertaken as best I could but please let me know if you'd like more information, or if I've made a mistake along the way.
I'm attempting to load data into a PostgreSQL 12 database like so:
This database lives in a docker container, brought up with docker-compose:
I initially encountered this error:
I sort of blindly tried to fix this by adding a
unique
constraint onBrick.brick
: https://github.com/radusuciu/pyteomics/commit/364413d361177ff7747ab858b8cdc63b399db6a3#diff-1ade65743552f19d20d8bcc9845a4166aefd0fe563acc01c5716402ff7cbaee7L303-R303Similar errors then crop up on the
Specificity
andElement
tables: https://github.com/radusuciu/pyteomics/commit/364413d361177ff7747ab858b8cdc63b399db6a3After adding
unique
constraints for those other tables, I hit a different error.My best guess at this point was that maybe
Brick
did not exist. I wiped the slate clean (stop container, remove database folder, and then re-create). I hit another issue with someCrossreference
URLs being longer than 128 chars, so I swapped to aTEXT
type column from the originalVARYING
: https://github.com/radusuciu/pyteomics/commit/35ff6dc007f70b83d81772df1d03c2d6fd739b28Then, I deleted all the postgres data and re-created the container once more, only to hit a new error:
Wipe the slate clean once more, and a new error that I think also hints at a table creation order issue:
I will continue to investigate a bit, but wanted to report this issue to see if you have any thoughts on whether or not I've identified a real bug, or made a mistake somewhere along the way. I am able to successfully load into a SQLite database, without any of the modifications that are present on my fork, so I might just import those tables into Postgres, or maybe query the SQLite database directly.
Thanks!