search

pelkmanslab / TissueMAPS-OLD

Old TissueMAPS code (until 2016); please use pelkmanslab/TissueMAPS instead.
http://github.com/pelkmanslba/TissueMAPS
0 stars 0 forks source link

Metadata assignment issue with 3D yokogawa images #31

Closed scottberry closed 7 years ago

scottberry commented 7 years ago

I recently processed a 3D dataset from the Yokogawa on a test instance build from the TissueMAPS master branch on 20/2/16 (a.m.). I have the following strange bug with assignment of metadata resulting in the two channels being confused. Some images (certain z-positions only) are assigned to the wrong channel.

Here is a zoom out of the pyramid:

screen shot 2017-02-20 at 16 09 39

My data has only two channels, but clearly DAPI and cell trace are being interchanged in certain z-positions. The effect seems to be random.

I have added ssh keys for @HackerMD @alecrimi @SaadiaIftikhar to have access to the instance (172.23.93.94). There is only one experiment there "/storage/experiments/..."

screen shot 2017-02-20 at 16 10 09

hackermd commented 7 years ago

I cannot access the instance.

scottberry commented 7 years ago

@HackerMD I now added both your ssh keys from github, please try again

On 20 Feb 2017, at 16:31, Markus D. Herrmann notifications@github.com wrote:

I cannot access the instance.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pelkmanslab/TissueMAPS/issues/31#issuecomment-281108828, or mute the thread https://github.com/notifications/unsubscribe-auth/AFKEaISbOR2JAwTAQuHCzmj9I1JKktKcks5rebHLgaJpZM4MGT6L.

hackermd commented 7 years ago

The problem becomes apparent when inspecting image_file_mappings:

SELECT wavelength, map->'files' FROM image_file_mappings;
2          | ["AssayPlate_Greiner_655090_D03_T0001F001L01A01Z01C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z02C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z03C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z04C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z05C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z06C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z07C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z08C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z09C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z10C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z11C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z12C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z13C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z14C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z15C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z16C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z17C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z18C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z19C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z20C02.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z21C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z22C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z23C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z24C02.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z25C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z26C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z27C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z28C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z29C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z30C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z31C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z32C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z33C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z34C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z35C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z36C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z37C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z38C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z39C02.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z40C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z41C01.tif"]

There are files with C01 and C02 in the same row. The problem is independent of the pyramid creation phase, but is due to incorrect grouping of planes in the metaconfig step.

scottberry commented 7 years ago

I did not provide any metadata options in the interface but relied entirely upon the supplied metadata file and image filenames. Note that I had to hack the filenames and the metadata file a bit to remove “#” from the filenames (and each file's reference in the metadata file). This was necessary to prevent failure at "image conversion”.

On 20 Feb 2017, at 16:58, Markus D. Herrmann notifications@github.com wrote:

The problem becomes apparent when inspecting image_file_mappings:

SELECT wavelength, map->'files' FROM channel_image_files; 2 | ["AssayPlate_Greiner_655090_D03_T0001F001L01A01Z01C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z02C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z03C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z04C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z05C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z06C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z07C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z08C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z09C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z10C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z11C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z12C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z13C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z14C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z15C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z16C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z17C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z18C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z19C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z20C02.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z21C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z22C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z23C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z24C02.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z25C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z26C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z27C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z28C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z29C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z30C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z31C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z32C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z33C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z34C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z35C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z36C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z37C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z38C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z39C02.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z40C01.tif", "AssayPlate_Greiner_655090_D03_T0001F001L01A01Z41C01.tif"] There are files with C01 and C02 in the same row. The problem is independent of the pyramid creation phase, but is due to incorrect grouping of planes in the metaconfig step.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pelkmanslab/TissueMAPS/issues/31#issuecomment-281115958, or mute the thread https://github.com/notifications/unsubscribe-auth/AFKEaFoii0Yo3tmuf1DtvcGSW30z01aRks5rebg-gaJpZM4MGT6L.

hackermd commented 7 years ago

I don't think this is related. It rather seems there is something wrong with assigning indexes upon grouping z-planes. I am looking into it.

hackermd commented 7 years ago

I found the problem. In your case channel C02 was acquired before C01 and as a consequence images in the metadata file have a different order when compared to the naturally sorted image files in the directory. I assumed this order would match. The problem thus lies in tmlib.workflow.metaconfig.cellvoyager.CellvoyagerMetadataReader.

hackermd commented 7 years ago

Should be fixed with TissueMAPS/TmLibrary@ee42fbafd6d16e5c6bd1a2febed651a417676043

scottberry commented 7 years ago

Pulled this commit into my local instance and testing now.

scottberry commented 7 years ago

This is now fixed, thanks.

screen shot 2017-02-22 at 09 01 15

However, there is some issues with updating both channels in the viewer when the z-slider is used. I have opened a new issue #32