search

srophe / Linked-Data

2 stars 1 forks source link

Write transform #1

Closed wsalesky closed 8 years ago

wsalesky commented 8 years ago

Will use XSLT. It processes faster then xquery in my experience with eXist.

@davidamichelson We should discuss if we want this run as a cron job and stored in the db? Stored on github?

wsalesky commented 8 years ago

@davidamichelson @suellenstringer-hye First pass at Gazetteer data transform. There is a record for each, and also a record that for all data, was not sure what format was best. Please let me know how it looks. I did some spot checking but I'm sure we will need to do some tweaking.

suellenstringer-hye commented 8 years ago

Thanks Winona,

Much appreciated! ---Suellen

From: Winona Salesky [mailto:notifications@github.com] Sent: Friday, March 04, 2016 10:17 AM To: srophe/Linked-Data Cc: Stringer-Hye, Suellen Subject: Re: [Linked-Data] Write transform (#1)

@davidamichelsonhttps://github.com/davidamichelson @suellenstringer-hyehttps://github.com/suellenstringer-hye First pass at Gazetteer data transform. There is a record for each, and also a record that for all data, was not sure what format was best. Please let me know how it looks. I did some spot checking but I'm sure we will need to do some tweaking.

— Reply to this email directly or view it on GitHubhttps://github.com/srophe/Linked-Data/issues/1#issuecomment-192340466.

wsalesky commented 8 years ago

Great! Let me know if/when we need to make adjustments.

davidamichelson commented 8 years ago

Look forward to looking at this!

suellenstringer-hye commented 8 years ago

Hi Winona,

We are running into validation errors on all-places.ttl and many of the individual place .ttl files.Some of them seem to be Xquery errors (according to Cliff) but others are maybe errors with the data itself. Do you have a validator you can use to correct these errors? We started trying to do it ourselves but maybe there is an automated method? Plus we wouldn't want to have to validate each time we regenerate. Stardog won't let us upload unvalidated files.

Thanks much, Suellen and Alex

wsalesky commented 8 years ago

Sorry! I will take a look. What do you use to validate .ttl? -Winona

suellenstringer-hye commented 8 years ago

We used

https://www.npmjs.com/package/turtle-validator

(pretty sure...Cliff took us there through a back door but we are 99% sure that's what we used)

---Suellen

wsalesky commented 8 years ago

Great thanks. Sorry about the errors, was trying to push the data before I left for vacation!

suellenstringer-hye commented 8 years ago

No, we're glad you did...some of the individual files were fine so we are happy to have had a chance to upload them as proof of concept.

wsalesky commented 8 years ago

Okay, here is the second version. If I have time later tonight I may try loading them into a triple store. I corrected the XQuery errors, and handled all the validation errors reported by the above turtle validator. (Mostly quotes within labels.)

wsalesky commented 8 years ago

Seems to load okay in my version of StarDog.

CliffordAnderson commented 8 years ago

Thanks, @wsalesky. We look forward to trying it out.