Use Pentaho's open source data integration tool (Kettle) to create Extract-Transform-Load (ETL) processes to update a Socrata open data portal. Documentation is available at http://open-data-etl-utility-kit.readthedocs.io/en/stable
We may want to include a working JAR in DataSync/ directory from the DataSync release pages. Right now, a slug is placed that places the onus to install DataSync on the user.
The objective is to include the highest version of DataSync that is compatible with the commands that assemble an ETL job for the portal. We're starting to experience issues with newer versions of DataSync that are not compatible with flags or control files that are included in this repo.
This would not mean this repo is not compatible with higher versions, but provide some mechanism to verify compatibility.
We may want to include a working JAR in
DataSync/directory from the DataSync release pages. Right now, a slug is placed that places the onus to install DataSync on the user.The objective is to include the highest version of DataSync that is compatible with the commands that assemble an ETL job for the portal. We're starting to experience issues with newer versions of DataSync that are not compatible with flags or control files that are included in this repo.
This would not mean this repo is not compatible with higher versions, but provide some mechanism to verify compatibility.