heitzmann / gdstk

Gdstk (GDSII Tool Kit) is a C++/Python library for creation and manipulation of GDSII and OASIS files.
https://heitzmann.github.io/gdstk/
Boost Software License 1.0
324 stars 78 forks source link

Add Raith MBMS Paths #252

Closed MatthewMckee4 closed 1 week ago

MatthewMckee4 commented 1 month ago

Raith MBMS paths are normal paths but store some additional data. image Base element refers to another element in the gds file.

Below is a normal gds file with 2 cells. Cell "1" has a box in it, cell "2" has a normal path in it. image This is the gds file after converting the path in cell "2" into a Raith MBMS path referencing cell "1". image The cut off data is: RaithPXXData: 0 0 0 0 0 0 0 0 0 bd ff 40 0 0 0 0 0 6a d8 40 0 0 0 0 0 0 f0 3f 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0

This raith pxx data is structured the following way: image

The path in cell 2 now has the following attributes: image

Please let me know if you are able to do this or if you need any more information from me. Thanks

heitzmann commented 1 month ago

@MatthewMckee4 I have no time available to implement such a feature in the foreseeable future, sorry.

tvt173 commented 1 month ago

hi @MatthewMckee4, you should be able to do this with gdsfactory version 8

import gdsfactory as gf

@gf.cell
def my_path() -> gf.Component:
    c = gf.Component()
    # add your path here
    c._kdb_cell.add_meta_info(gf.kdb.LayoutMetaInfo("RaithPXXData", "Your Data", persisted=True))
    return c

if __name__ == "__main__":
    c = my_path()
    c.show()

image

this new major version of gdsfactory should be released within a few days, but you can try it now by pulling and installing from the main branch on github. if this functionality is critical to you, i suggest you give it a try!

@sebastian-goeldi

sebastian-goeldi commented 1 month ago

Yes, this is possible. But if it should be restored on read, c.info["my_info"] is thebetter way. This gets serialized and deserialized.other metainfos than settings/infos/ports are currently ignored by kfactory and consequently gdsfactory (on load).

Also, I belive the initial example is in user properties. I believe gdstk can also write those?

tvt173 commented 1 month ago

@sebastian-goeldi , I was going to suggest that at first, but in that case it seems like the attribute name gets prefixed by kfactory:... and I was assuming that the application here needs the attribute exactly named to be processed by the tool correctly. there's no way to avoid the prefix when saving into the info, right?

sebastian-goeldi commented 1 month ago

Ah, I see. No I don't think so. So if this matters, I think the MetaInfo is the wrong place then, assuming these are user infos.

The thing is the metainfos are not a gds standard but rather a KLayout addition. So if it's an external tool, it would be important to know the target format. If it's gds it becomes tricky as user properties only can have a dict[int, str] format. Oasis can do dict[str|int,Any] format. Ofc shouls the external too support meta infos (probably not as it's a rather new additon for KLayout), then if prefix matters, your way works of course.

Rory-Lambert commented 1 month ago

Hi guys, colleague of @MatthewMckee4 here. I run the Raith tool which requires these unusual paths. The file type required by the machine is .gds. As far as I understand from our discussions with the tool manufacturer, the Pxx data is simply appended to the end of a normal Path instance. This certainly breaks the normal GDSII specification, and caused files generated by this tool to fail to open in Klayout. We managed to convince the developer of LayoutEditor to implement these structures for us. Firstly, to handle file I/O such that any of these custom paths were just handled as normal paths, rather than crashing the editor. Then, eventually we gained the ability to add, remove and edit the metadata, including via scripting. I want to do the same thing here with the GDStk/ GDSfactory toolchain. I think from Matthew's example provided above, they are defined as their own record type ('MBMPath') and the metadata stored in the RaithPxxdata attribute of this record. I've made an example GDS file which I will attach here, perhaps it's helpful? Note that I had to append the .txt suffix to workaround GitHub's limits on file types that can be uploaded. mbms_demo.gds.txt. Below I also paste the output of our rudimentary GDSII record reader, in case its any use.

If you guys (or @MatthewMckee4) can generate a file that you think might work, I can very quickly throw it in the Raith software and see if it behaves as expected. Many, many thanks all!

HEADER: 3
BGNLIB: 2024 6 4 10 6 47 2024 6 4 10 6 47
LIBNAM:mbms_demo.gds
UNITS: 0.0001 1e-10
BGNSTR: 2024 6 4 10 3 36 2024 6 4 10 3 36
STRNAM:this_is_the_base_cell
BONDRY:
LAYER: 1
DTATYP: 1000
XY: -5000 -5000 5000 -5000 5000 5000 -5000 5000 -5000 -5000
ENDEL:
ENDSTR:
BGNSTR: 2024 6 4 10 5 4 2024 6 4 10 5 4
STRNAM:this_cell_contains_an_MBMS_path
RaithMBMPath:
LAYER: 2
DTATYP: 1000
WIDTH: 60000
XY: 0 0 0 1000000
SNAME:this_is_the_base_cell
RaithPXXData: 0 0 0 0 0 0 0 0 0 4c dd 40 0 0 0 0 0 6a e8 40 0 0 0 0 0 0 f0 3f 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
ENDEL:
ENDSTR:
BGNSTR: 2024 6 4 10 6 47 2024 6 4 10 6 47
STRNAM:this_is_what_the_tool_woul_write
AREF:
SNAME:this_is_the_base_cell
STRANS: 0
COLROW: 2 33
XY: 0 0 100000 0 0 990000
ENDEL:
ENDSTR:
ENDLIB:
tvt173 commented 1 month ago

IMHO that's a pretty bad limitation... what about trying to convince the vendor to do it in a compatible way that will open in Klayout (and other tools)? you can try the output of the sample script i put above. note that for demonstration purposes, it only has the attribute with "Your Data" as the data value, and no actual paths or shapes. and i send as a .gds.gz because github will not let me send a .gds, but you should be able to open in klayout as-is or decompress to use otherwise raith_attributes.gds.gz

Rory-Lambert commented 1 month ago

IMHO that's a pretty bad limitation... what about trying to convince the vendor to do it in a compatible way that will open in Klayout (and other tools)?

Yes mate, its an awful limitation! It my biggest headache with this machine by far, and it's been really difficult to do proper scripted GDSII layouts while working around this issue. Its one of the biggest reasons we are pushing to get this implemented in these packages. I have spoken with the tool vendor at length, and it was difficult enough to get them to share enough about their file structure so as we could get this far. I don't think its possible that they will make a change to it at this point unfortunately.

you can try the output of the sample script i put above.

I think that your script adds meta data to the cell? I don't believe that this would be appropriate - one cell can have many MBMS paths, each of which containing its own metadata, so as I understood it the challenge is to pack these properly into the GDS record.

it only has the attribute with "Your Data" as the data value

Apologies, I don't use KLayout often, where can I find this info?

tvt173 commented 1 month ago

you can view the info in klayout like this: image image

you should also be able to see the attributes in the binary of the file (i.e. if you open it in notepad)

to implement the functionality in a way that is not compatible with klayout is a tall order... the new version of gdsfactory is based on klayout. i think you would need to make a PR to klayout for such a request.

but personally, i think this is bad design on the part of the vendor. there are other, more compatible ways of annotating this data. I would push them towards a better solution