DivineOmega / uxdm

🔀 UXDM helps developers migrate data from one system or format to another.
GNU Lesser General Public License v3.0
167 stars 9 forks source link

Converting from XML to CSV Missing Blanks #24

Open labatt opened 5 years ago

labatt commented 5 years ago

I might be doing something completely wrong...

<?php

require 'vendor/autoload.php';
use DivineOmega\uxdm\Objects\Migrator;
use DivineOmega\uxdm\Objects\Sources\XMLSource;
use DivineOmega\uxdm\Objects\DataItem;
use DivineOmega\uxdm\Objects\DataRow;
use DivineOmega\uxdm\Objects\Destinations\CSVDestination;

$xmlFile = __DIR__.'/play.xml';
$xPathQuery = '/ACES/App';
$xmlSource = new XMLSource($xmlFile, $xPathQuery);

$csvFile = __DIR__.'/output.csv';
$csvDestination = new csvDestination($csvFile,['Part','BaseVehicle','DriveType']);

$migrator = new Migrator;

$migrator->setSource($xmlSource)
         ->setDestination($csvDestination)
         ->setFieldsToMigrate(['BaseVehicle','DriveType','Part'])
         ->setKeyFields(['Part'])
         ->withProgressBar()
         ->migrate();

If the XML doesn't have DriveType defined for some of the "rows", the CSV file should still have the same number of columns with one blank. Instead, it's just missing the column.

For example, input:

<App action="A" id="1" validate="yes">
        <BaseVehicle id="5257">1995, Ford, Explorer</BaseVehicle>
        <DriveType id="8">4WD</DriveType>
        <Note>Position Rear</Note>
        <Qty>1</Qty>
        <PartType id="12809">Axle Shaft Damper</PartType>
        <MfrLabel>Axle Damper</MfrLabel>
        <Part>14077</Part>
    </App>
    <App action="A" id="105917" validate="yes">
        <BaseVehicle id="143120">2017, Honda, Odyssey</BaseVehicle>
        <Note>From Chassis/VIN CB088499</Note>
        <Qty>2</Qty>
        <PartType id="7600">Suspension Strut Mount</PartType>
        <MfrLabel>Suspension Mounts</MfrLabel>
        <Position id="22">Front</Position>
        <Part>143305</Part>
    </App>

You'll notice the first item has a DriveType but the second one doesn't. Here's the resulting CSV:

BaseVehicle,DriveType,Part
"1995, Ford, Explorer",4WD,14077
"2017, Honda, Odyssey",143305

The second line should actually be:

"2017, Honda, Odyssey",,143305
DivineOmega commented 5 years ago

Definitely looks like an issue. I'll look into this as soon as I can.

For the time being, a work around would be to modify the XML to always include the DriveType but leave it blank if there is not content, if possible.

labatt commented 5 years ago

Thanks - unfortunately I have no control over the source file. It is sent to me from a manufacturer, and "DriveType" is just one of several tags that may or may not be blank unfortunately.

I can, for the moment, work around like this:

            $DriveType = $dataRow->getDataItemByFieldName('DriveType');
             if ($DriveType->value=='') {$actual=' ';} else {$actual=$DriveType->value;$dataRow->removeDataItem($DriveType);}
             $dataRow->addDataItem(new DataItem('DriveType',$actual));

But obviously it's preferable to have it auto-recognize null fields and account for them. Nice tool though!