If any field includes a new line, KiField recreates it without location information.

[scandey]

Feature request: warning/catch when schematic includes fields with new lines

This is absolutely on me, but I finally identified the cause of KiField corrupting my KiCad project. At some point while editing in the internal parts spreadsheet of KiCad, I added new lines to the end of some of the values of footprint fields of some, but not all, parts. (Probably copying and pasting values too fast and not being careful)

When KiField read out the parts it all worked fine, but upon returning new values to those parts in the schematic file, KiField deleted the location information for that field and corrupted the schematic so it wouldn't open properly.

While this was a mistake that seems unlikely to be a common issue, I thought I would put it out there so that if anyone else has this issue there is some way to move forward. In the end I removed the new lines from the schematic files before editing with KiFields and that fixed the issue. find . -name '*.sch' -print0 | xargs -0 sed -i .bak -e ':a' -e 'N' -e '$!ba' -e 's/\"\r\"/\"\"/g' was the command I ended up using (on Mac OS)

Am happy to put together an example if needed!

[xesscorp]

Thanks for finding this bug and providing a clear explanation!

I made a change to kifield so it scans all field values and removes trailing newlines and replaces any internal newlines with spaces. Please try it out and see if it fixes the problem.

If it doesn't fix the problem, please send me some test files and I'll work on it some more.

[scandey]

Hello! Sorry it took me too long to get back to this, I am happily using KiField for the project without any new issues so I forgot to come and check on this issue after I found a workaround.

I don't think the .17 update has fixed the issue unfortunately, though I appreciate your quick changes. Maybe a confusion is that I wasn't including newlines in the .csv/.xlsx input to Kifields, but had somehow inserted newlines into the values of the schematic. (EESchema doesn't like this if you try to do it through the normal UI, but I suspect copying and pasting into the internal spreadsheet of components was how I managed it.) This form of corrupted schematic may be beyond the spec of KiFields, but in the interest of completeness I've included example files.

I've attached:

• A working example file, baseline.sch
• Same file with a newline in a value (somehow, still not sure how I accidentally did it originally), newline.sch
• And the file after being modified by kifield and reinserted, newline_corrupted.sch

newline_schematics.zip

I started with baseline.sch and modified the file directly to include a newline in the value of the capacitor (line 43) which is similar to what I accidentally managed to do. Kicad doesn't seem to have a problem with this, though the formatting is a little funky.

Relevant capacitor portion of newline.sch from line 38 to line 49:

$Comp
L Device:C C1
U 1 1 60517F99
P 4600 3400
F 0 "C1" V 4348 3400 50  0000 C CNN
F 1 "C
" V 4439 3400 50  0000 C CNN
F 2 "" H 4638 3250 50  0001 C CNN
F 3 "~" H 4600 3400 50  0001 C CNN
    1    4600 3400
    0    1    1    0   
$EndComp

Used KiField to make a simple change:

Ran kifield -x ~/Desktop/newline.sch -i ~/Desktop/newline.xlsx -g -d 10 Opened newline.xlsx, changed the value of the capacitor from "C" to "NewC", and saved. Ran kifield -i ~/Desktop/newline.sch -x ~/Desktop/newline.xlsx -g -d 10

When attempting to open the newly modified schematic, EESchema throws a warning about an unexpected EOL on Line 43 and opens a blank schematic. The resulting file is newline_corrupted.sch

Error loading schematic file “/Users/scottcandey/Desktop/newline_corrupted.sch”.
unexpected end of line in input/source
"/Users/scottcandey/Desktop/newline_corrupted.sch"
line 43, offset 11

Checking on the line that EESchema had an issue with, the capacitor portion of newline_corrupted.sch from line 38 to line 48:

$Comp
L Device:C C1
U 1 1 60517F99
P 4600 3400
F 0 "C1" V 4348 3400 50 0000 C CNN
F 1 "NewC"
F 2 "" H 4638 3250 50 0001 C CNN
F 3 "~" H 4600 3400 50 0001 C CNN
    1    4600 3400
    0    1    1    0   
$EndComp

And sure enough, there appears to be no positional information for the value field.

Debugging for the KiField command to extract from newline.sch, there's something funky going on.

Exploding C1 => ['C1'].
Updating C1 field value from None to C
Updating C1 field footprint from None to 
Updating C1 field datasheet from None to ~

The KiField command to re-insert doesn't seem to show anything terribly useful, though I do notice that the update to the capacitor value field is missing quotes:Updating C1 field 1 from C to "NewC" compared to the other un-updated components Updating R1 field 1 from "R" to "R".

Please let me know if there are more tests that would be at all useful.

Thanks!

[xesscorp]

Thanks for the detailed explanation and the example files!

I think the problem was in sch.py. That's some code I grabbed 5 years ago to parse schematic files. It works on a line-by-line basis, so having a line-break in a quoted string causes it to process an incomplete statement. I added some logic to detect lines which are broken within a quoted string and reconnect them. With that change, kifield correctly processes your example files.

The changed code is in the development branch. Please give it a try and let me know if it works for you.

[scandey]

That works for me, thank you!