search

cytomining / cytominer-database

[DEPRECATED] A package for storing morphological profiling data.
Other
10 stars 12 forks source link

Other ways of creating db backends or obviating the need for a db #124

Open shntnu opened 4 years ago

shntnu commented 4 years ago

For our notes, there are two alternate paths that either replicate cytominer-database functionality or obviate its need

  1. @gwaygenomics works directly with CSVs in pooled Cell Painting experiments link
  2. @bethac07 has a script for ingesting CSVs into a SQLIte file, which can be loaded in CellProfiler Analyst link

@bethac07 @gwaygenomics could you please edit this comment to add the links / details above? 🙏

bethac07 commented 4 years ago

Where would you like mine to live? My generic repo, one of our profiling repos?

shntnu commented 4 years ago

Where would you like mine to live? My generic repo, one of our profiling repos?

~I think the google docs is perfectly fine, or at least, it doesn't seem worth the effort of transferring 80+ pages into a markdown document (unless you find that easier to maintain in the longer run)~ sorry I misread the topic :D

bethac07 commented 4 years ago

Sorry, clarifying- my python script that you can use in lieu of cytominer-database. It's just been living in my random Broad project repo, but if you think it should live somewhere slightly more official now is a good time for it!

shntnu commented 4 years ago

but if you think it should live somewhere slightly more official now is a good time for it!

A repo that has frequently-used scripts by assay devs would be great, but this might be an overhead for you to maintain, so its your call.

Either way no harm in creating a repo image-analysis-scripts or similar in github.com/broadinstitute, even if this is the only script that lives in it.

diskontinuum commented 4 years ago

For our notes, there are two alternate paths that either replicate cytominer-database functionality or obviate its need

  1. @gwaygenomics works directly with CSVs in pooled Cell Painting experiments [link]
  2. @bethac07 has a script for ingesting CSVs into a SQLIte file, which can be loaded in CellProfiler Analyst link

@bethac07 @gwaygenomics could you please edit this comment to add the links / details above? 🙏

The link is broken?

bethac07 commented 4 years ago

The link is not broken, just only available inside the BroadInstitute, sorry!

gwaybio commented 4 years ago

This completely slipped through my TODOs - I updated my link above (also here it is). Beth's link is in a private repo that it looks like you don't have access to.

One of the reasons I decided to sidestep the database step is b/c in the pooled cell painting project we will likely work with single cell data. It is a pain to manipulate/extract single cell files from the standard sqlite format (especially since we need to merge across tables each and every time) - these processes take way too long.