search

socrata / datasync

Desktop / Console application for updating Socrata datasets automatically.
http://socrata.github.io/datasync/
MIT License
79 stars 33 forks source link

Dataset template for logging unavailable #171

Open shua123 opened 7 years ago

shua123 commented 7 years ago

Both examples of the dataset template for logging that are referenced in http://socrata.github.io/datasync/resources/preferences-config.html#setting-up-logging-using-a-dataset are no longer available.

  1. Source Domain: https://adrian.demo.socrata.com Source Dataset ID: aywp-657c
  2. https://docs.google.com/file/d/0B-VEFikh3T6dX2VyWTZXZW55SFU/edit?usp=sharing
BrienDeveloper commented 7 years ago

Agreed, I cannot access these either, and it would be extremely helpful to be able to. In the comments of the article, there is mention of 'unlikely, but just in case'.. YIKES! Here I go trying to do this completely manual/guessing.: "Alternatively to using a Port Job, if you wish to create the log dataset manually (unlikely, but just in case), download the following CSV file and upload it to your Socrata datasite using the manual upload process in a web browser: https://docs.google.com/file/d/0B-VEFikh3T6dX2VyWTZXZW55SFU/edit?usp=sharing"

If I come up with a usable replacement I'll post it here to share with you @shua123

BrienDeveloper commented 7 years ago

So this was actually not difficult at all.

DatasetLoggingSetup.txt I am not allowed to attach a CSV file, so use the TXT file I have attached then change the file extension to CSV once downloaded. Then import it as a new dataset using the web UI.

Then, using the instructions on the article, setup the column types as listed: Be sure that you set the column data types to match those listed below:

Date: Date & Time DatasetID: Plain Text FileToPublish: Plain Text PublishMethod: Plain Text JobFile: Plain Text RowsUpdated: Number RowsCreated: Number RowsDeleted: Number Success: Checkbox Errors: Plain Text

Then in your actual dataset that you have data uploaded to, link this logging dataset in the configuration settings within the Datasync UI tool. If you have a config file for these settings, then instead just replace the empty string with your logging datasetID: "logDatasetID": "xxxx-xxxx"

I ran an update to my main datasetID, and then a few seconds later there was a new row in my logging dataset. Hope that helps!!!

levyj commented 7 years ago

There is actually at least one more column that was added at some point. (I vaguely remember but would have to hunt down the specifics.)

DataSyncVersion: Plain Text

I am not sure if it is mandatory or optional. We added it but possibly by choice.

With apologies to Socrata, this log is not the best-maintained feature. Some things do not even log properly and I think the direction is towards using the Jobs section of the Admin module. I am not necessarily against that but it could use a better transition and some gap analysis to make sure we do not lose functionality.