search

treveradams / libpes

A library to read and write valid PES embroidery files
20 stars 11 forks source link

Contribute to implementation #1

Open JoshVarga opened 10 years ago

JoshVarga commented 10 years ago

I am the developer primarily in charge of formats on the Embroidermodder project. We have a simple implementation of writing PES files. Recently I worked to understand the entire PES format primarily version 1 and 7. I have an application written in C that parses nearly all of the data properly. I plan to make it open source, currently embroidermodder is not prepared to handle all the design information so I wanted to understand your plans for implementation. Does it make sense to you for me to contribute to this project or just spin up a new repository?

Thanks, JoshVarga

tatarize commented 7 years ago

PES0001ƒ............... ..LA:

This oddly enough loads as valid in PE-DESIGN and Wilcom and both can well show the shapes. Between the 00 00 00 00 where the PES ends and the seek value where the PES starts you can put anything in there. I wouldn't be shocked if you could stack PEC objects and get different results simply by changing the seek value. The PES and PEC parts have nothing to do with each other. You can trash one and the other still works. All the programs just go to the seek file and read the PEC data. You could likely make a few different valid file formats that are also valid PES/PEC formats. You just need the header and to seek to the value. And PES just skips that seek and reads configuration information for PES.

You could make your own format and just hijack the data, and still very likely make it work on any Brother embroidery machine. The same thing Brother did to make their design format and their machine embroidery format work in the same file would still be valid. I'd bet the machines don't even check the PES version. You could start out #PES001(valid pec seek value) and then have your own format with anything you want.

Like this, valid.zip

Inserted a bunch of text in it, and so long as I kept the seek valid, and terminated the PES file it loaded with designs in Wilcom and just as a blank thing in PE-DESIGN.

JoshVarga commented 7 years ago

This bit you should be able to find in my version of libpes.... take a look if you are missing more.

https://github.com/JoshVarga/libpes/blob/master/PesLib/pes-header.c

On Mon, Apr 17, 2017 at 5:36 PM, tatarize notifications@github.com wrote:

And looked for the setting that 0x0e bit messed with and found it. [image: toggle-0x0e] https://cloud.githubusercontent.com/assets/3302478/25107341/95570506-2383-11e7-832c-4db7f818e8ba.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/treveradams/libpes/issues/1#issuecomment-294615829, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYcnBGchJZGn15tzrZfZds4Hq6wTcFCks5rw-lpgaJpZM4BgWJO .

frno7 commented 7 years ago

Many thanks, @tatarize! These PES files are a great start for validating the format description as well as any library implementation. I’ve collected them in a samples directory.

2 bytes: 1 (we are going to write only one stitchblock object).

This integer of the version 1 header still spooks, as it apparently can be 44 (0x2c) as well:

@JoshVarga, would you be able to explain the section count variable in your implementation? The value appears to be 44 (0x2c) in complex-v1.pes which only contains 2 sections (CEmbOne and CSewSeg). Similarly, it appears to be 18 (0x12) in complex-v6.pes which only contains 7 sections (CEmbCirc, CEmbRect, CEmbLine, CEmbPunch, CSewFigSeg, CEmbNText and CLettert). Surely the value must be related to something else than the section count? Both files were created by @tatarize using Brother PE-Design.

tatarize commented 7 years ago

Oh, no. That's from my explanation of how to write pes. I choose to write a 1 in that value. It can be any number it's the count of the number of CEmbSewSeg objects appended that block.

tatarize commented 7 years ago

The CEmbOne and the SewSeg objects are the same thing. You can just put the CEmbOne object in front of the SewSeg object. This bunch of data can be parsed as usual. But, in it can also be the case that there are other objects appended within the SewSeg block, these will be prefaced with exactly the data of the CEmbOne block and then the stitch data of the SewSeg bit. It's simple to make this 1 then put the description information in CEmbOne and the stitches in SewSeg. But, if you have more than one, these are mashed together and put on the end of the SewSeg block.

lines.zip

Take special note at the lines512.pes I made sure to expand it such that there's a value in the 0x11

tatarize commented 7 years ago

It occurs to me that I'm not sure whether that section value is the number of SewSeg blocks or would include the shapes as well. So I made different versions of them. The cv1 is circle added to version 1, and cv6 is circle added to version 6. (I don't know the elements of v6 header as I only mapped out v1).

For reference is included the v2-v6 versions of the same lines512 pes, because maybe something weird happens there. But, I'm not actually sure if the value for that count is really the value of the SewSeg blocks in total or all the blocks including circles and other shapes. Hence the inclusion.

line512versions.zip

Note for Line512cv1 the circle object is 2 objects, the fill and the edge. That's why the value goes to 0x0202.

frno7 commented 7 years ago

@tatarize, if I understand your latest test files for PES version 1 correctly, the PES header integer at offset 16 is related to the number of PEC stitch “groups”, which is the same as the number of TRIM+JUMP commands in the PEC plus one. So, for instance in the file line8.pes we have a PEC section like

PEC stitch_count 15
PEC stitch    0    4.6    0.0 NORMAL
PEC stitch    1   -1.1  -13.8 TRIM
PEC stitch    2    3.5  -13.8 NORMAL
PEC stitch    3   -1.2  -25.8 TRIM
PEC stitch    4    3.4  -25.8 NORMAL
PEC stitch    5   -0.3  -40.9 TRIM
PEC stitch    6    4.3  -40.9 NORMAL
PEC stitch    7   12.3    0.8 TRIM
PEC stitch    8   16.9    0.8 NORMAL
PEC stitch    9   11.2  -13.0 TRIM
PEC stitch   10   15.8  -13.0 NORMAL
PEC stitch   11   11.1  -25.0 TRIM
PEC stitch   12   15.7  -25.0 NORMAL
PEC stitch   13   12.0  -40.1 TRIM
PEC stitch   14   16.6  -40.1 NORMAL

with 7 TRIMs thus 7+1=8 as expected. This also holds for line512cv1.pes which abbreviated looks like

PEC stitch_count 9834
...
PEC stitch 1019   -0.5   13.6 TRIM
PEC stitch 1020    4.1   13.6 NORMAL
PEC stitch 1021    0.4   -1.5 TRIM
PEC stitch 1022    5.0   -1.5 NORMAL
PEC stitch 1023    5.0   -1.5 STOP
PEC stitch 1024    3.6   44.1 JUMP
PEC stitch 1025    3.6   44.1 NORMAL
PEC stitch 1026    3.5   43.8 NORMAL
PEC stitch 1027    3.6   44.1 NORMAL
PEC stitch 1028    2.8   42.3 NORMAL
PEC stitch 1029    2.0   40.4 NORMAL
PEC stitch 1030    1.3   38.6 NORMAL
...
PEC stitch 8533    3.7   44.1 NORMAL
PEC stitch 8534    3.6   44.1 NORMAL
PEC stitch 8535    3.8   43.9 NORMAL
PEC stitch 8536    3.6   44.1 NORMAL
PEC stitch 8537    3.6   45.1 TRIM
PEC stitch 8538    3.6   45.1 NORMAL
PEC stitch 8539    3.3   45.0 NORMAL
PEC stitch 8540    3.6   45.1 NORMAL
PEC stitch 8541    3.4   43.1 NORMAL
PEC stitch 8542    3.2   45.1 NORMAL
...

with 512 TRIMs and 1 JUMP thus 512+1+1=514 as expected. However, the relation does not appear to hold for the collection “Embroidery Design Boutique 2” of about 126 PES version 1 files supplied by Brothers. The integer is 1 for all files with no relation to the number of TRIMs and JUMPs.

Perhaps it is best to assume and document that Brother PE-Design is an authoritative source for this value and go with the relation found in your test files?

tatarize commented 7 years ago

They are the same thing. Within the SewSeg blocks there can be multiple SewSegs blocks. The EmbOne block is the first header for the SewSegs. Each segments chunk has a header, the EmbOne block being the first of these. The remaining headers occur inline with the SewSeg block rather than in the EmbOne block. Within the EmbOne block there is a number which indicates the how many segment lists occur. But in the header we have the number of segment chunks in total. Both are valid and load fine in PE-Design.

There are groups of groups. If you have 40 segment lists. You can say you have 1 block of 40 lists, as Wilcom and most saveouts will say. Or you can say you have 40 blocks of 1 list. You could equally write the file to say it has 20 blocks of 2 lists. The only difference is that for each added block you need an additional header, appended to the CSewSeg block to store those lines. So it's much easier to say you have 1 block and n items than n blocks of 1 item. Because you don't have to write (n-1) addition headers within the CSewSeg block.

When you write the lists as a single chunk of lists, you must break them with trims. This isn't a requirement if the end of the chunk is the end of the list. This is a bit of a pain but easier than inserting additional headers. The headers within PE-Design are the selection boxes themselves. They indicate the box we are selecting and grabbing etc. So dividing them up does make a change. But if you take a design and load it then save it with PE it'll have broken them into subparts. But, it loaded the original fine, as a single object.

tatarize commented 7 years ago

My fiddle ( https://jsfiddle.net/Tatarize/4hyx4e06/ ) properly shows the distinction. Reading Block CSewSeg Expected number of segment-blocks 514 ... Segments Expected In This Block 1 ... Expecting 2 X/Y 977, 1000 X/Y 1023, 1000 Proper Segment End -- END SEGMENT BLOCK ... Segments Expected In This Block 1 ... Expecting 2 X/Y 966, 862 X/Y 1012, 862 Proper Segment End -- END SEGMENT BLOCK

Whereas, if I take the same file with the 514 blocks, load it in wilcom and save it back.

line512cv1-w.zip

SewSeg Blocks 1 ... Segments Expected In This Block 1028 ... Proper Segment End -- END SEGMENT BLOCK

So rather than have 514 different objects of 1 segment. It has 1 and tosses in cuts between all of those (or should have), so that 1 object of 1028 objects (increased because it has to coax the trim with added objects).

tatarize commented 7 years ago

I tweaked your wiki to make this clear and correct some of the errors, as well as fully described most of the blocks, given that Josh had mapped them out. The Rect etc objects are pretty easy, they are just filled with amazing amounts of trash. I mean specific values about how to fill the shapes (even when that's not the type of fill used).

PES-CSewSeg-section

frno7 commented 7 years ago

Great, @tatarize! I’ll continue the discussion in the wiki Improvements issue you opened.

tatarize commented 4 years ago

@treveradams The documentation here and elsewhere contributed to pyembroidery ( https://github.com/EmbroidePy/pyembroidery ). As well as to the Embroidery Formats project on Edutech's Wiki ( https://edutechwiki.unige.ch/en/Embroidery_format_PES ) which now largely houses PES and several of the other formats technical documentations.

pyembroidery can read and write PES v1 v6. However, I found no examples of the shapes existing in the wild nor did I have a pressing need to convert them. I've debated decoding them, since the work was done, but haven't. It's pretty trivial now to convert PES to SVG, pyemb -i file.pes -o file.svg. But, for the early work this was fantastic. Thank you.