search

vindelorme / PlateEditor

Repository for PlateEditor, the web application allowing management of plate layout, results visualization and analysis
MIT License
20 stars 1 forks source link

database backend to unlock persistent metadata and downstream analysis #2

Open bgriffen opened 3 years ago

bgriffen commented 3 years ago

Hi, nice work! I was curious though, was there any thoughts as to using a database backend e.g. SQL, postgres, mongoDB to hold the plate info and results. You could then just have a view of all historic plates and enable downstream data analysis in Python using an ORM like peewee. You could even just use peewee for storing all of the session metadata that could then be accessed via port X on a lab's local server. It would also remove file handling! =)

vindelorme commented 3 years ago

Hi there, Thanks for your comment! The idea of database storage is still on the table, but I pushed it back due to lack of training. I will need a thorough refresher on database security and maintenance first! I like the idea of accessing the files from the lab's server. I was also thinking about accessing files from the Cloud, but again I need some training here first. I you have an interest in this and time to dedicate, please feel free to contribute!

bgriffen commented 3 years ago

If you have an example of the relevant fields you need to record in one file, I could make a model in peewee that can store them potentially. The results one would be fairly straight forward (well --> value)

vindelorme commented 3 years ago

The fields in PlateEditor are selected by the user and are dependent on the parsing properties selected for each file. The parsing allows the file header or some leading columns to be ignored for example. Based on this, the mapping then allows to assign roles to the parsed columns. You can check the wiki on Sourceforge for more details about this: https://sourceforge.net/p/plateeditor/wiki/Data%20Import/ Regarding the database storage, I guess dumping the parser and mapper configs attached to the file as a json string would be a relatively simple approach, combining ease of use with robustness towards any future changes. The database model should also be simple then. What I need to check in more details are the procedures to attack the database through JavaScript, in a secured fashion. Any tips are welcomed!