MetaInfo with (gigantic) size makes klayout write a faulty gds

[sebastian-goeldi]

When writing large lists or dictionaries (I am very much not a fan of this practice, but alas) as MetaInfos, klayout will happily write the gds, but fail on reading it.

Example:

import klayout.db as kdb

ly = kdb.Layout()
c = ly.create_cell("TEST")

# first entries are just to illustrate that the total size of the records is not the problem, the 8190 line is enough to trigger the error
for i in (
    1,
    10,
    100,
    1000,
    3000,
    8188,
    # 8190, # does not work if uncommented
):
    print(i)
    c.add_meta_info(kdb.LayoutMetaInfo(str(i), [1 for _ in range(i)], persisted=True))

c.write("test.oas")

I suspect MetaInfos have a maximum size. when opening this file directly from a terminal I get

ERROR: ../../../src/tl/tl/tlDeflate.cc,248,n < sizeof (m_buffer) / 2
ERROR: Internal error: ../../../src/tl/tl/tlDeflate.cc:248 n < sizeof (m_buffer) / 2 was not true

When doing the same but through klive (i.e. it will just tell the layout view to reload), I get:

Caught the following exception:
ENDEL, PROPATTR or PROPVALUE record expected (position=31499, record number=22, cell=$$$CONTEXT_INFO$$$) in LayoutViewBase.reload_layout (Class RuntimeError)

tl;dr

I am pretty sure that klayout expects the entry to have a maximum size in bytes, but on write will happily write larger entries.

It would be nice if klayout could either

• throw an error if record written is too large
• read unlimited size of record

Whichever is easier to implement. (First one would allow me to not entertain ridiculous ideas though :stuck_out_tongue_closed_eyes:)

P.S.: Is there an absolute maximum of number of records the $$$CONTEXT_INFO$$$ cell can hold? We had it that in one tapeout klayout stopped writing metainfos at a seemingly random number of metainfos, but I couldn't confirm or deny yet.

[klayoutmatthias]

Hi @sebastian-goeldi,

I think that is due to record size limit in GDS. OASIS should not have this limitation. I will check what I can do. An error is easy, unlimited size is possible maybe. It needs some stitching of pieces however.

The number of metainfo items should not be limited, but I will check that.

Matthias

[sebastian-goeldi]

Thank you Matthias!

I am happy with either solution. Whatever you deem is better.

I am pretty sure though that this also affects oasis, I will recheck.

[klayoutmatthias]

I think I found a solution that fixes both the "too many" and "too big" problem in a backward-compatible way.

The solution is to split the strings used to serializing the meta info and to stop using property attribute numbers for meta info entries exceeding the GDS limit of 32768 entries.

Old versions will read files written with the new version, but skip entries using the new string notation.

[sebastian-goeldi]

Thank you, that sounds like a great solution!