Reading the header

[richiemort79]

Hi Bruno,

I'm trying to parse Simi biocell files in ImageJ and I came across your code. I wonder could you let me know the interpretation of the headers in the .sbd files? I'm having trouble interpretting them.

Best wishes Richard

[bruvellu]

Hi Richard,

I also had troubles with figuring out the .sbd file, but I finally got the format specification. Although some things are still a bit obscure... :

### Data v4.00 ####################################################

# Header: # # # # # # #
SIMI*BIOCELL
400
---
# # # # # # # # # # # #

SIMI*BIOCELL = magic ID string
400          = file version 4.00
---          = separator

# Cell: # # # # # # # #
<free 3D cells count>
<start frame> <end frame> <x> <y> <level> <comment>     <= *count
---
<start cells count> <start time>
<start generation>                                      <= *count
---
<cells left count> <"right> <active cell left> <"right> <gen.name1>
<gen.time of birth sec.> <g.level> <g.wildtype> <g.color> <g.name2>
<birth frm> <mitosis lvl> <wildtype> <size> <shape> <color> <name>
<coordinates count> <cell comment>
<frame> <x> <y> <level> <size> <shape> <coord.comment>  <= *count
---
# # # # # # # # # # # #

<start ...> values are not used yet (for later implementation)
<color> is a real hexadecimal RGB value (e.g. 00ff00 for green)
<size> and <shape> are internal values of BioCell
<coordinates> are real pixels

###################################################################

What are you using to parse it in ImageJ? I think you can easily use the simi.py running as a Jython script. I'm using the library to convert my cell lineage data to the MaMuT format.

Best, Bruno

[richiemort79]

Hi Bruno,

Thanks for getting back. Currently I am manually reading the file into arrays using ImageJ so I can access the data but I'll try using your python file also. I hadn't realized Mamut was a fiji plugin this might be the solution I require.

I'll try and work on some front end macro code to import .sbd files parse and convert to MaMut using your scripts. It would be very useful for me to be able to do this from within Fiji as my collaborators won't want to sue Python. Maybe you already have such a macro?

Best R

On 19/12/16 14:39, Bruno C. Vellutini wrote:

Hi Richard,

I also had troubles with figuring out the .sbd file, but I finally got the format specification. Although some things are still a bit obscure... :

|### Data v4.00 #################################################### # Header: # # # # # # # SIMIBIOCELL 400 --- # # # # # # # # # # # # SIMIBIOCELL = magic ID string 400 = file version 4.00 --- = separator

Cell: # # # # # # # # <free 3D cells count> <end

frame> <= *count ---

<= *count --- <"right> <"right> <= *count --- # # # # # # # # # # # # values are not used yet (for later implementation) is a real hexadecimal RGB value (e.g. 00ff00 for green) and are internal values of BioCell are real pixels ################################################################### | What are you using to parse it in ImageJ? I think you can easily use the |simi.py| running as a Jython script. I'm using the library to convert my cell lineage data to the MaMuT format. Best, Bruno — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub , or mute the thread .

[bruvellu]

Hey, I have a script to convert to MaMuT .xml. It works, but it's not complete yet, and it's fairly specific to the lineage I'm working on. But you can use the parts that are useful. The script is pure python, but Fiji can have macros in Jython (Python for Java), and I guess most of the code can be simply re-used.

The script loads the simi.py library, reads and store the lineage data, and write the data out formatted as .xml. It's here: simi2mamut.py.

The script works (I have my lineage fully loaded on MaMuT), but take care because it's not well documented or finished yet. There are also some known bugs. I will be working on the code in the next weeks. Let me know if you have issues!

[richiemort79]

Great I'll look into this and let you know how I get on.

Best

R

[richiemort79]

Hi Bruno,

So running your code from the linux cli seems to work (on your test data). It writes the text to the terminal window. When I copy this and save it as an .xml file it looks OK. However I can't seem to open it in Mamut? Are there additional steps required? If you had an .xml that works with Mamut you could send me as an example that might help?

As far as running these scripts in Fiji using jython is concerned there are some problems as I think you used 2.7 and we are stuck at 2.5 in Fiji. So the OrderedDIct module is not available but this work around fixes that:

http://code.activestate.com/recipes/576693-ordered-dictionary-for-py24/

Place in Fiji/Plugins and import like this:

OrderedDict is not available in 2.5 so include the following workaround

from java.lang.System import getProperty
import sys
sys.path.append(getProperty("fiji.dir") + "/plugins")
from ordereddict import OrderedDict
from os.path import splitext

The other problem is that the string.format method does not work in Python 2.5 (it still uses % method). I am not great at Python and it is taking me an age to rewrite your code to take account of this. If its an easy fix for you you could look at that?

One last question. In the simi files there are lots of cells that seem to be redundant. How do you decide whether an entry is valid?

Best wishes

R

[bruvellu]

Hi! I just pushed a quick fix for the MaMuT file. The script was only printing the spots and edges part of the XML and now it outputs the whole file. Make sure the new file mamut_xml_templates.py is on the same folder as simi.py and simi2mamut.py.

For now, you need to manually change the ImageData in the Settings part at the end of the XML (filename, folder, width, height, nslices, nframes). This information is on the .xml file that pairs the .h5 created by the BigDataViewer. I assume you have MaMuT installed and your data on HDF5 format.

Check now if you can open the XML with MaMuT using "Load MaMuT file".

I'll reply for the next issues below.

[bruvellu]

About the OrderedDict and string format, I hadn't realized Jython was so outdated. Before we delve into Python 2.5 compatibilities, do you really need the conversion from Simi to MaMuT to be ran within Fiji? You mentioned that your collaborators won't use python, but how is the workflow that you imagined? If they will fire up Fiji and run the conversion, it is also possible use the system's python installation to run simi.py. You can for example call exec in a ImageJ macro:

exec('sh', '-c', 'cd /directory/to/simi.py/; python simi2mamut.py')

This prints out MaMuT's xml within Fiji, but you can do all sorts of things with the output (e.g. saving it somewhere and directly opening in MaMuT, for instance). Most computers today have Python 2.7 or it's easy to install, so simi.py should run without any additional installation.

If this is not the workflow you need, I just saw that Jython 2.7 exists and should cover these quirks. Just a matter of updating it in Fiji (maybe the devs can do it in a breeze).

The third alternative is to update simi.py code to accommodate py2.5. It's not particularly complicated, but it will be a hassle to maintain the compatibility over time.

Let me know what you think!

[bruvellu]

On my lineage files there are a lot of cells that do not really exist (i.e. haven't been tracked). They come from the default template based on C. elegans lineage.

To decide whether a cell is valid or not, I check if the cell has any spots (i.e. a manually tracked point with xyz coordinates). This is done here. Only valid cells are used to build the MaMuT file.

[richiemort79]

Cheers for your messages. I'll try the new code when i get chance over Xmas. I agree the best option would be for Fiji to upgrade jython. I'll look into this. I'll need to look into calling Python externally. I'm guessing this will be hard to make consistent between platforms. Have a great holiday! R

Sent from BlueMail ​

On 22 Dec 2016, 11:38, at 11:38, "Bruno C. Vellutini" notifications@github.com wrote:

Hi! I just pushed a quick fix for the MaMuT file. The script was only printing the spots and edges part of the XML and now it outputs the whole file. Make sure the new file mamut_xml_templates.py is on the same folder as simi.py and simi2mamut.py.

For now, you need to manually change the ImageData in the Settings part at the end of the XML (filename, folder, width, height, nslices, nframes). This information is on the .xml file that pairs the .h5 created by the BigDataViewer. I assume you have MaMuT installed and your data on HDF5 format.

Check now if you can open the XML with MaMuT using "Load MaMuT file".

I'll reply for the next issues below.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/nelas/simi.py/issues/1#issuecomment-268782713

[richiemort79]

Hi Bruno,

Happy new year! I'm just catching up on the emails you sent before the break. When I run your new code I get the following error opening the .xml with mamut:

(Fiji Is Just) ImageJ 2.0.0-rc-54/1.51h; Java 1.8.0_66 [64-bit]; Linux 4.4.0-57-generic; 154MB of 10805MB (1%)   java.lang.NullPointerException at fiji.plugin.trackmate.io.TmXmlReader.readTracks(TmXmlReader.java:1011) at fiji.plugin.trackmate.io.TmXmlReader.getModel(TmXmlReader.java:338) at fiji.plugin.mamut.LoadMamutAnnotationPlugin.load(LoadMamutAnnotationPlu gin.java:74) at fiji.plugin.mamut.LoadMamutAnnotationPlugin.run(LoadMamutAnnotationPlug in.java:61) at ij.IJ.runUserPlugIn(IJ.java:217) at ij.IJ.runPlugIn(IJ.java:181) at ij.Executer.runCommand(Executer.java:137) at ij.Executer.run(Executer.java:66) at java.lang.Thread.run(Thread.java:745)

Best R

On Thu, 2016-12-22 at 03:38 -0800, Bruno C. Vellutini wrote:

Hi! I just pushed a quick fix for the MaMuT file. The script was only printing the spots and edges part of the XML and now it outputs the whole file. Make sure the new file mamut_xml_templates.py is on the same folder as simi.py and simi2mamut.py. For now, you need to manually change the ImageData in the Settings part at the end of the XML (filename, folder, width, height, nslices, nframes). This information is on the .xml file that pairs the .h5 created by the BigDataViewer. I assume you have MaMuT installed and your data on HDF5 format. Check now if you can open the XML with MaMuT using "Load MaMuT file". I'll reply for the next issues below. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[bruvellu]

Hi Richard, Did you manage in the end to run simi.py within Fiji? Best!

[richiemort79]

Hi Bruno, In the end (thanks to your explanation of the header) I wrote me own code to read the Simi files and draw the lineage trees. It also extracts the interdivision times.

https://github.com/richiemort79/simi_reader

Thanks for your help. And cheers for the colourblind luts also I've been using those recently.

Best wishes R

Sent from Alto

[bruvellu]

Nice! Glad it worked out. I'll close the issue, but feel free to re-open if you need.

Cheers