Pes files don't work.

[tatarize]

inkstitch/inkstitch#273

Finds inkstitch pes files to be in error.

[tatarize]

Pes version 1, pyembroidery-only convert. -- Works. No preview. Pes version 6, pyembroidery-only convert. -- Does not show. Pes version 1, truncated pes-block convert. -- Works. No preview.

It seems that the PEC block itself must be flawed in some way to prevent previews. Further the version 6 header block must be flawed in some way to prevent showing up as a pes file.

Other outputs; Wilcom, embird, and brother all output version 6 with both headers and pec bodies that function.

Brother V6 blank header with pasted in brother PEC body shows preview. Brother V6 blank header with pasted in pyembroidery PEC body, does not show up.

---

This may entirely be a flaw in PEC blocks. Namely anything with a pyembroidery pec block, in V1 shows up with no preview and V6 does not show up at all.

[tatarize]

pec-checks.zip

Should contain raw brother, raw pyembroidery. Then switch each of the three main sections from pyembroidery's pec into brother's pec.

[tatarize]

@pivi66 @wwderw can you try these files?

[pivi66]

In your zip file are only pes files. Is this correct?

[pivi66]

@tatarize In the first quick test only three files are displayed. I have to test it one by one tomorrow. There was different thread information displayed. Due to the fact that the brother V3 dose not show the filename i have to retest and will send you the results.

[tatarize]

Sounds great, yeah, 1 that works is going to be the the raw brother. The raw pyembroidery will fail. But, it means 2 of the three mixed work. Which means it will quite likely narrow down the exact section that has the point of failure.

[pivi66]

1. Bbh6-br-pyg.pes -> no display
2. Bbh6-br-pyh.pes -> is displayed and thread info is:
   • 1801 Madeira Poly
   • 1637 Madeira Poly
   • 1800 Madeira Poly
3. Bbh6-br-raw.pes -> is displayed and thread info is : 29, 5, 20
4. Bbh6-br-pys.pes -> is displayed and thread info is : 29, 5, 20
5. Bbh6-py-raw.pes -> no displayed

[tatarize]

So the flaw is in pyg or the pyembroidery graphics section of the pec block. The difference in the pyh file is that the colors are located in the header and that file switched them out. The expectation there should be that that would cause functional changes the listed colors. But, the rest of it works fine, so I'm not concerned there. The lack of display is problematic but is apparently caused by something in the graphics block. So I'll see if I can find something functionally and potentially catastrophically different in that section between the versions that work and the ones that don't.

[tatarize]

These should be fixed versions of the V1 files. Namely convert-v1 and convertv1t both files should work. They should be totally displayed rather than simply displayed by a single dot.

The error with the single dot was due to what turned out to python v2 v3 error where division was changed from integer division to floating point native. Which means if I didn't specify the floating point division for the v2 it caused an integer division that became 0 rather than something like 0.2948... which caused the scaling on the image render everything in the graphic into the centermost pixel. That seems to be the bug with reference to the pec graphic not displaying. These 2 files should prove that fact.

pesv1fixed.zip

Do note if the above files work properly, that it should be possible from here to convert the default mode pes version 1 and fix the floating point error with the graphic rendering namely PecGraphics.py lines 97 and 98 to:

    scale_x = (graphic_width - buffer) / float(diagram_width)
    scale_y = (graphic_height - buffer) / float(diagram_height)

That would suffice for the most part as a fix.

---

The part still in error in the version 6 of the file turns out to likely be the result of a post-pec block section within the version 6 of the file, that I hadn't clocked previously.

This should put a fake one of those on it, and maybe work.

convert6-A-suf.zip

[pivi66]

Nice work! convert6-A-suf.pes -> the file is usable but the preview is still only one dot convertv1.pes & convertvt1.pes -> both files are ok and show the madeira threads

[tatarize]

The convert6 is the original file that didn't work. I pasted on the suffix bit from a working file. I changed nothing else, so it should have started working like the V1 from convertv1 which is which had the broken 1-dot preview too.

I'm not really sure what the suffix bit actually means, but giving it a working copy from a version6 to the broken pyembroidery version 6 fixes it. So it's just an unmapped version 6 element.

The bugs appear to be: 1) Division bug causes PEC graphics to become a single dot. 2) Unmapped section in version 6 is missing causing version 6 to fail.

Version 1 is fixed. Version 6 has the preview fixed, but the display is broken because of the missing element. Gives me a solid path to fixing it. Namely, figure out what that post-pec-block data is and reproduce it.

[tatarize]

The current workable workaround is switch to version 1 of the filetype, and fix the PEC graphics bug. It will properly fix the bug for now. Really the V6 of the file needs some additional mapping. There's apparently some additional code information after the graphics file that contains various colors and something that looks like addendum bitmap information. I'll figure it out and update @danielkschneider wiki. ( https://edutechwiki.unige.ch/en/Embroidery_format_PES )

@pivi66 I wrote all of the original code and know all of the formats. There's really nobody else outside of perhaps Josh (of Embroidermodder) with the current knowhow. I tried to make it as easy as I could and document the formats as much as I could, but I don't think there anybody other than me whose poised to fix the data.

I doubt Lex's ego would preclude him from updating his forked version of my code. Sure, he'll permaban me for being right about a technical matter that hurt his feelings, but I doubt he'd let the project suffer greatly beyond that.

So in theory he'll update whatever backend he's using to my corrected code in due course. For now I've updated the default PES version to V1 and fixed the python2 PEC graphics bug as highlighted above. I updated the PyPI version but it might require him manually the stuff himself but it should get fixed.

Here's the current workaround version: pyemb-A.zip

If this will sew, then it's clear the workaround will function, while I figure out the missing pec-addendum in version 6.

[tatarize]

The PEC addendum is 128 bytes duplicated from PEC [30-A0] namely, a number that determines the number of entries, then padding of spaces 0x20 to pad out the characters. Then there are n+1 entries for the color changes of 0x93 bytes (147). Much of this data seems duplicated between each element for each layer. Then version 6 ends with two extra 0 bytes. Versions 4 and 5 don't have that.

Factors of 147: 1, 3, 7, 21, 49, 147.

Might be positioning data or something.

Took a raw pyembroidery version 6 and fed it blank addendum data. Not sure what this would do, but it might oddly fix things or break them in different and interesting ways. blank-post-block-A.zip

---

Update: The only time it changes is when the thread color changes, so it must be thread data. The final triplet appears to be color, and you previously mentioned it had thread data available. Might not all be from the header. Though the header seems like the thing that changed that information.

[tatarize]

It's apparently actually 128 bytes, color byte and padding. Then 0x90 bytes of something data. Wilcom has 0x5A (15 6 byte lines) bytes of 0s followed by 6 lines of heavy data, brother has light data followed by zeroed data, and embird has light data and then heavy data. At the end is a list of straight RBG values. So each proper info block is 0x90 which is 144 so likely 24 * 6 bytes. or 12 x 12.

To that end, if they can be blank is some and not blank in others maybe they can be blank in all. So rather than blank header, here's the code with an empty postblock. If you can find a difference in this code it would actually instantly imply what that block does.

empty-pb-A.zip

[pivi66]

Hi all three files are working on the innovis v3, great work. Sorry for asking the question, how the update we are working on will flow back to inkstitch. Inkstitch was the entry point for me to test what is possible with open source and embroidery. I think without any software to create your own designs a sewing machine is only an expensive toy. So thanks for your great work. If there is something more to test let me know.

[tatarize]

Hm. If they all work, then I have little idea what they do. They are apparently needed, but then have some color information and 144 bytes of something that seems like they are 3 byte sections. And somehow don't matter. I found a color in brother's own software that produced pure zeros, so it must be valid to do so. But I'll keep the default PES version to 1, it's max compatibility though loses some color definition.

No need to apologize for asking whether the time you're spending is well spent. In fact, it's likely one of the best questions you could ask. And admittedly there's a bunch of seeming added complexity to the code flow. Inkstitch used to point to my PyPI repository directly but after Lex permabanned me for making him feel bad by correcting a conceptual error, he spun off the code into a different code base. He permabanned me again for pointing out that permabanning people because they make you feel bad is entirely bad form and not helpful to the project. But that does leave the project in a weird place since I'm the only one with ample expertise in embroidery formats. It means critical bugs here need to be integrated back into his code back manually. And might mean that other bug fixes won't be reflected back if they aren't important enough, or things like correcting the PHC and PHB or MIT reading and perhaps some of the important breaking changes to include specific needles in formats or other tie-off modes.

Though by my count most of the code is verified as sewn. People have sewn VP3, EXP, JEF, DST. I haven't gotten any data back on anybody sewing out .U01 or .PEC and even with PES corrected, it's not necessarily sewn which is the gold standard. I do have reports of .PMV being sewn but that's a stitch format rather than an embroidery format. So in theory it the project is almost complete. Though I did stumble on some small bugs when I made EmbroidePy (python based low level embroidery editor) start injecting sequins.

---

But, having somewhat mapped that addendum data or knowing what is needed,

I can do some neat things with the extra information. That A or Flower test was pretty easy to fix. A-or-flower2.zip It should show one thing but sew another. It lacked that end bit, which totally explains why it failed. It was intended to figure out if it was using the PEC stitches or the PEC graphics for the data display.

This should be the current exported version of PES V6, with code-created addendum blocks: py-new-a.zip If this works the version 6 code works.

And here's the truncated version of V6 which is much smaller but should still sew (it omits all data for brother software). truncate6-a.zip I am not sure if this would display or sew, but it's a weird sort of hacked file omitting the stuff that only brother's computer software reads and including just the bits that the embroidery machine should read.

Unless that added block has something that changes on the display screen for you. I'm not really sure what that 144 bytes of data should do. But, apparently it need only exist, it doesn't actually need to say anything, so it's a bunch of zeros and then the RGB colors.

[pivi66]

All three files are displayed one is a flower and two times "A". I also tested the inkstitck fix and the PES works.

[tatarize]

If you wanted to go above and beyond, you could try sewing something from inkstitch fix and report back to them if it works, or me if it doesn't. It's a bit above and beyond to invest cloth and thread other than time but really it is the gold standard of knowing it actually functions. It's also the only way to some of the other errors it might do like double stitching weirdly or cutting where it shouldn't.

Though I'd imagine it's your intent to sew something, but if you do, it might be nice if you told them that. Let's the issue get closed with confidence.

[pivi66]

I will give sewing using inkstitch a try. So far i sewed yesterday a simple design and it worked. The selected thread colour was displayed correct. I am quite new to github and how to do stuff. But i have a different request what one of your project could do. there is a special file format for characters on the brother sewing machines. This allows to use the on screen keyboard to create your text. I have lots of letters ( Alphabets ) and it would be handy to convert such a file (i think the file extension is PEF). An what is the right way to communicate or crate an idea.

Thanks for you great effort.

[tatarize]

Usually you'd start up a new issue and post it in there. So each request bug thing gets its own topic. You could email me a copy of some of them to my gmail account tatarize. I'd be happy to take a look. I did recently to through the trouble of hacking together a .pmv reader and writer and editor. Which is a stitch format for the sewing machines. I've not seen any embroidery fonts, directly but I grasp a lot of the ideas behind them. Namely fonts are glyph objects that have very specific abilities and render scheme and encodings, but embroidery fonts are different as they require to have specific pathing. So much of the embroidery font sort of formats is going to be largely predicated what amounts to clipart like embroidery files that you sort of insert on to screen.

I don't have the Brother's font editor so I don't really have example files, and can't make them. But, I'd be happy to take a look, if sent a few. For Copyright reasons I would avoid posting them here. Assuming they reused similar code to their other formats it would be pretty straight forward, but it might be alien.