search

RGLab / CytoML

A GatingML Interface for Cross Platform Cytometry Data Sharing
GNU Affero General Public License v3.0
29 stars 14 forks source link

gs-to-flowjo parse error #134

Closed alexheubeck closed 3 years ago

alexheubeck commented 3 years ago

Hi @mikejiang,

We recently updated our CytoML version, and we're getting this error when trying to use gatingset_to_flowjo:

./gs-to-flowjo --src /data/output/PB1/gating-set --dest /data/output/PB1/B052_PB1_PB00641-01_QC.wsp [libprotobuf ERROR google/protobuf/message_lite.cc:122] Can't parse message of type "pb.GatingHierarchy" because it is missing required fields: (cannot determine missing fields for lite message) terminate called after throwing an instance of 'std::domain_error' what(): Failed to parse GatingHierarchy B052_PB1_PB00641-01_QC.fcs ls /data/output/PB1 Aborted (core dumped)

This looks similar to #133.

Let me know if you have any ideas, or if you think reverting the version will fix this. Not sure if it's CytoML or the associated Docker.

Thanks,

Alex Heubeck

[1] "SESSION INFO" R version 4.0.3 (2020-10-10) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Debian GNU/Linux 10 (buster) Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.7.so locale: [1] C attached base packages: [1] parallel stats graphics grDevices utils datasets methods [8] base other attached packages: [1] gridExtra_2.3 colorRamps_2.3 scales_1.1.1 [4] CytoML_2.3.4 optparse_1.6.6 tibble_3.1.1 [7] flowDensity_1.22.0 data.table_1.14.0 ggcyto_1.19.1 [10] ggplot2_3.3.3 ncdfFlow_2.36.0 BH_1.75.0-0 [13] RcppArmadillo_0.10.4.0.0 flowCore_2.3.2 flowWorkspace_4.3.8 [16] openCyto_2.3.0

nicolehoward commented 3 years ago

Hi there @mikejiang,

So, to add onto this as Alex and I are both working on this, I recently discovered that you are now maintaining your images at this location. I switched to this image to run ./gs-to-wsp, and it is giving the same error as before, but now it is identifying what required fields are missing, similar to this ticket: #133 .

Here is the error message:

./gs-to-wsp --src /data/output/PS1/gating-set --dest /data/output/PS1/B046_PS1_IMM19_418_QC.wsp [libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/message_lite.cc:133] Can't parse message of type "pb.GatingHierarchy" because it is missing required fields: tree.node[0].node.hidden, tree.node[1].node.hidden, tree.node[1].node.indices.indtype, tree.node[1].node.thisGate.neg, tree.node[1].node.thisGate.isGained, tree.node[2].node.hidden, tree.node[2].node.indices.indtype, tree.node[2].node.thisGate.neg, tree.node[2].node.thisGate.isGained, tree.node[3].node.hidden, tree.node[3].node.indices.indtype, tree.node[3].node.thisGate.neg, tree.node[3].node.thisGate.isGained, tree.node[4].node.thisGate.neg, tree.node[4].node.thisGate.isGained, tree.node[5].node.indices.indtype, tree.node[5].node.thisGate.neg, tree.node[5].node.thisGate.isGained, tree.node[6].node.hidden, tree.node[6].node.indices.indtype, tree.node[6].node.thisGate.neg, tree.node[6].node.thisGate.isGained, tree.node[7].node.hidden, tree.node[7].node.indices.indtype, tree.node[7].node.thisGate.neg, tree.node[7].node.thisGate.isGained, tree.node[8].node.hidden, tree.node[8].node.indices.indtype, tree.node[8].node.thisGate.neg, tree.node[8].node.thisGate.isGained, tree.node[9].node.hidden, tree.node[9].node.indices.indtype, tree.node[9].node.thisGate.neg, tree.node[9].node.thisGate.isGained, tree.node[10].node.indices.indtype, tree.node[10].node.thisGate.neg, tree.node[10].node.thisGate.isGained, tree.node[11].node.thisGate.neg, tree.node[11].node.thisGate.isGained, tree.node[12].node.hidden, tree.node[12].node.thisGate.neg, tree.node[12].node.thisGate.isGained, tree.node[13].node.hidden, tree.node[13].node.thisGate.neg, tree.node[13].node.thisGate.isGained, tree.node[14].node.indices.indtype, tree.node[14].node.thisGate.neg, tree.node[14].node.thisGate.isGained, tree.node[15].node.indices.indtype, tree.node[15].node.thisGate.neg, tree.node[15].node.thisGate.isGained, tree.node[16].node.hidden, tree.node[16].node.indices.indtype, tree.node[16].node.thisGate.neg, tree.node[16].node.thisGate.isGained, tree.node[17].node.indices.indtype, tree.node[17].node.thisGate.neg, tree.node[17].node.thisGate.isGained, tree.node[18].node.indices.indtype, tree.node[18].node.thisGate.neg, tree.node[18].node.thisGate.isGained, tree.node[19].node.indices.indtype, tree.node[19].node.thisGate.neg, tree.node[19].node.thisGate.isGained, tree.node[20].node.hidden, tree.node[20].node.indices.indtype, tree.node[20].node.thisGate.neg, tree.node[20].node.thisGate.isGained, tree.node[21].node.hidden, tree.node[21].node.indices.indtype, tree.node[21].node.thisGate.neg, tree.node[21].node.thisGate.isGained, tree.node[22].node.hidden, tree.node[22].node.thisGate.neg, tree.node[22].node.thisGate.isGained, tree.node[23].node.hidden, tree.node[23].node.thisGate.neg, tree.node[23].node.thisGate.isGained, tree.node[24].node.hidden, tree.node[24].node.indices.indtype, tree.node[24].node.thisGate.neg, tree.node[24].node.thisGate.isGained, tree.node[25].node.hidden, tree.node[25].node.thisGate.neg, tree.node[25].node.thisGate.isGained, tree.node[26].node.hidden, tree.node[26].node.indices.indtype, tree.node[26].node.thisGate.neg, tree.node[26].node.thisGate.isGained, tree.node[27].node.thisGate.neg, tree.node[27].node.thisGate.isGained, tree.node[28].node.indices.indtype, tree.node[28].node.thisGate.neg, tree.node[28].node.thisGate.isGained, tree.node[29].node.hidden, tree.node[29].node.indices.indtype, tree.node[29].node.thisGate.neg, tree.node[29].node.thisGate.isGained, tree.node[30].node.hidden, tree.node[30].node.thisGate.neg, tree.node[30].node.thisGate.isGained, tree.node[31].node.thisGate.neg, tree.node[31].node.thisGate.isGained, tree.node[32].node.thisGate.neg, tree.node[32].node.thisGate.isGained, tree.node[33].node.hidden, tree.node[33].node.thisGate.neg, tree.node[33].node.thisGate.isGained, tree.node[34].node.hidden, tree.node[34].node.thisGate.neg, tree.node[34].node.thisGate.isGained, tree.node[35].node.hidden, tree.node[35].node.thisGate.neg, tree.node[35].node.thisGate.isGained, tree.node[36].node.thisGate.neg, tree.node[36].node.thisGate.isGained, tree.node[37].node.hidden, tree.node[37].node.thisGate.neg, tree.node[37].node.thisGate.isGained, tree.node[38].node.thisGate.neg, tree.node[38].node.thisGate.isGained, tree.node[39].node.thisGate.neg, tree.node[39].node.thisGate.isGained, tree.node[40].node.hidden, tree.node[40].node.thisGate.neg, tree.node[40].node.thisGate.isGained, tree.node[41].node.hidden, tree.node[41].node.thisGate.neg, tree.node[41].node.thisGate.isGained, tree.node[42].node.hidden, tree.node[42].node.indices.indtype, tree.node[42].node.thisGate.neg, tree.node[42].node.thisGate.isGained, tree.node[43].node.hidden, tree.node[43].node.thisGate.neg, tree.node[43].node.thisGate.isGained, tree.node[44].node.hidden, tree.node[44].node.thisGate.neg, tree.node[44].node.thisGate.isGained, tree.node[45].node.hidden, tree.node[45].node.thisGate.neg, tree.node[45].node.thisGate.isGained, tree.node[46].node.thisGate.neg, tree.node[46].node.thisGate.isGained, tree.node[47].node.thisGate.neg, tree.node[47].node.thisGate.isGained, tree.node[48].node.hidden, tree.node[48].node.thisGate.neg, tree.node[48].node.thisGate.isGained, tree.node[49].node.hidden, tree.node[49].node.indices.indtype, tree.node[49].node.thisGate.neg, tree.node[49].node.thisGate.isGained, tree.node[50].node.thisGate.neg, tree.node[50].node.thisGate.isGained, tree.node[51].node.thisGate.neg, tree.node[51].node.thisGate.isGained, tree.node[52].node.hidden, tree.node[52].node.indices.indtype, tree.node[52].node.thisGate.neg, tree.node[52].node.thisGate.isGained, tree.node[53].node.thisGate.neg, tree.node[53].node.thisGate.isGained, tree.node[54].node.thisGate.neg, tree.node[54].node.thisGate.isGained, tree.node[55].node.thisGate.neg, tree.node[55].node.thisGate.isGained, tree.node[56].node.hidden, tree.node[56].node.thisGate.neg, tree.node[56].node.thisGate.isGained, tree.node[57].node.indices.indtype, tree.node[57].node.thisGate.neg, tree.node[57].node.thisGate.isGained, tree.node[58].node.hidden, tree.node[58].node.indices.indtype, tree.node[58].node.thisGate.neg, tree.node[58].node.thisGate.isGained, tree.node[59].node.hidden, tree.node[59].node.indices.indtype, tree.node[59].node.thisGate.isGained, tree.node[60].node.indices.indtype, tree.node[60].node.thisGate.neg, tree.node[60].node.thisGate.isGained, tree.node[61].node.hidden, tree.node[61].node.thisGate.neg, tree.node[61].node.thisGate.isGained, tree.node[62].node.thisGate.neg, tree.node[62].node.thisGate.isGained, tree.node[63].node.hidden, tree.node[63].node.thisGate.neg, tree.node[63].node.thisGate.isGained, tree.node[64].node.indices.indtype, tree.node[64].node.thisGate.neg, tree.node[64].node.thisGate.isGained, tree.node[65].node.hidden, tree.node[65].node.thisGate.neg, tree.node[65].node.thisGate.isGained, tree.node[66].node.hidden, tree.node[66].node.indices.indtype, tree.node[66].node.thisGate.neg, tree.node[66].node.thisGate.isGained, tree.node[67].node.hidden, tree.node[67].node.thisGate.neg, tree.node[67].node.thisGate.isGained, tree.node[68].node.hidden, tree.node[68].node.thisGate.neg, tree.node[68].node.thisGate.isGained, tree.node[69].node.indices.indtype, tree.node[69].node.thisGate.neg, tree.node[69].node.thisGate.isGained, tree.node[70].node.hidden, tree.node[70].node.indices.indtype, tree.node[70].node.thisGate.neg, tree.node[70].node.thisGate.isGained, tree.node[71].node.indices.indtype, tree.node[71].node.thisGate.neg, tree.node[71].node.thisGate.isGained, tree.node[72].node.indices.indtype, tree.node[72].node.thisGate.neg, tree.node[72].node.thisGate.isGained, tree.node[73].node.hidden, tree.node[73].node.thisGate.neg, tree.node[73].node.thisGate.isGained, tree.node[74].node.hidden, tree.node[74].node.indices.indtype, tree.node[74].node.thisGate.neg, tree.node[74].node.thisGate.isGained, tree.node[75].node.indices.indtype, tree.node[75].node.thisGate.neg, tree.node[75].node.thisGate.isGained, tree.node[76].node.indices.indtype, tree.node[76].node.thisGate.neg, tree.node[76].node.thisGate.isGained, tree.node[77].node.hidden, tree.node[77].node.thisGate.neg, tree.node[77].node.thisGate.isGained, tree.node[78].node.hidden, tree.node[78].node.thisGate.neg, tree.node[78].node.thisGate.isGained, tree.node[79].node.hidden, tree.node[79].node.thisGate.neg, tree.node[79].node.thisGate.isGained, tree.node[80].node.hidden, tree.node[80].node.indices.indtype, tree.node[80].node.thisGate.neg, tree.node[80].node.thisGate.isGained, tree.node[81].node.indices.indtype, tree.node[81].node.thisGate.isGained, tree.node[82].node.hidden, tree.node[82].node.indices.indtype, tree.node[82].node.thisGate.neg, tree.node[82].node.thisGate.isGained, tree.node[83].node.indices.indtype, tree.node[83].node.thisGate.neg, tree.node[83].node.thisGate.isGained, tree.node[84].node.indices.indtype, tree.node[84].node.thisGate.neg, tree.node[84].node.thisGate.isGained, tree.node[85].node.hidden, tree.node[85].node.thisGate.neg, tree.node[85].node.thisGate.isGained, tree.node[86].node.hidden, tree.node[86].node.thisGate.neg, tree.node[86].node.thisGate.isGained, tree.node[87].node.hidden, tree.node[87].node.thisGate.neg, tree.node[87].node.thisGate.isGained, tree.node[88].node.hidden, tree.node[88].node.thisGate.neg, tree.node[88].node.thisGate.isGained, tree.node[89].node.hidden, tree.node[89].node.thisGate.neg, tree.node[89].node.thisGate.isGained, tree.node[90].node.hidden, tree.node[90].node.thisGate.neg, tree.node[90].node.thisGate.isGained, tree.node[91].node.thisGate.neg, tree.node[91].node.thisGate.isGained, tree.node[92].node.hidden, tree.node[92].node.thisGate.neg, tree.node[92].node.thisGate.isGained, tree.node[93].node.hidden, tree.node[93].node.indices.indtype, tree.node[93].node.thisGate.neg, tree.node[93].node.thisGate.isGained, tree.node[94].node.hidden, tree.node[94].node.thisGate.neg, tree.node[94].node.thisGate.isGained, tree.node[95].node.hidden, tree.node[95].node.indices.indtype, tree.node[95].node.thisGate.neg, tree.node[95].node.thisGate.isGained, tree.node[96].node.hidden, tree.node[96].node.indices.indtype, tree.node[96].node.thisGate.neg, tree.node[96].node.thisGate.isGained, tree.node[97].node.hidden, tree.node[97].node.thisGate.neg, tree.node[97].node.thisGate.isGained, tree.node[98].node.hidden, tree.node[98].node.thisGate.neg, tree.node[98].node.thisGate.isGained, tree.node[99].node.hidden, tree.node[99].node.thisGate.neg, tree.node[99].node.thisGate.isGained terminate called after throwing an instance of 'std::domain_error' what(): Failed to parse GatingHierarchy B046_PS1_IMM19_418_QC.fcs Aborted (core dumped)

We are using RProtoBufLib version 2.3.5. I noticed this commit happened recently and am wondering if it has anything to do with it. It looks like proto2 and proto3 have significant differences and one of those differences is required fields are no longer available. This makes me question why this error is happening if they got rid of that functionality. Are we missing some dependency?

Thanks for looking into this.

mikejiang commented 3 years ago

I've updated public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp, pull and try again

alexheubeck commented 3 years ago

Thanks @mikejiang, that fixed our issue.

Alex