Create air quality dataset

jimjonesbr commented 11 years ago

Create air quality dataset (NOAA) for the demonstration at INSPIRE.

mpfeil commented 11 years ago

Simon mentioned PEGELONLINE. Not air quality but maybe interesting. http://www.pegelonline.wsv.de/gast/start

albhasan commented 11 years ago

I found a WMS, WFS but not a SOS. Can you help me finding the website's SOS? My German is not that good.

mpfeil commented 11 years ago

For PEGELONLINE SOS i found this link: http://pegelonline.wsv.de/webservices/gis/sos?request=GetCapabilities&service=SOS

albhasan commented 11 years ago

@mpfeil : The triple store has more than 1 million triples now, all of them comming from 52 North's demo SOS service. Can you check how they look in the viewer? We need to select a good-looking time-window to make a video or to make a live demo.

Cheers!

mpfeil commented 11 years ago

@albhasan : Thanks. I am testing right now. What looks a bit strange to me is if I query our SPARQL Endpoint 1) getting the graphs is very very slow 2) if I select the watertemperature property the "listFeaturesOfInterest" function of the facade returns 22 times the same FOI

mpfeil commented 11 years ago

3) the "getFOILastObservation" of the facade is not responding anymore :(

albhasan commented 11 years ago

Hi: Jim fix the getting graphs, it's way faster now

jimjonesbr commented 11 years ago

Listing graphs fixed and repository updated. Now the listing graphs is quick enough :+1:

jimjonesbr commented 11 years ago

@mpfeil I just tested the listFeatureOfInterest method and it works perfectly. Could it be you're not iterating through the list properly? The query's resultset does not repeat any value

mpfeil commented 11 years ago

Okay. I will have a look at it. So the watertemperature property has one foi right?

jimjonesbr commented 11 years ago

@mpfeil my apologies, it was my mistake. We never had a property with more than one foi, that's why I never noticed the a variable was outside the iteration loop.

It now should bring this result:

I'm going to commit the change now.

jimjonesbr commented 11 years ago

Commited. :+1:

mpfeil commented 11 years ago

okay i will check it out and will look at it

jimjonesbr commented 11 years ago

great! let me know when you can see them :)

robarto commented 11 years ago

I big that from Wuppertal Oberbarmen

On Tue, Jun 11, 2013 at 9:51 PM, Jim Jones notifications@github.com wrote:

great! let me know when you can see them :)

— Reply to this email directly or view it on GitHubhttps://github.com/jimjonesbr/de.ifgi.europa/issues/2#issuecomment-19287815 .

Thomas Bartoschek ifgi - Institute for Geoinformatics University of Münster Weselerstr. 253 48151 Münster, Germany

Phone: +49 (0)251 83 30011 Mobile: +49 (0)173 511 6094 Fax: +49 (0)251 83 39763

Email: bartoschek@uni-muenster.de http://ifgi.uni-muenster.de/ http://www.gi-at-school.de/ http://www.uni-muenster.de/MExLab/ http://vespucci.org/

jimjonesbr commented 11 years ago

@mpfeil , the last query (actually the most imporant one :D) is still taking too long, but @albhasan is working on it. If we don't solve it quickly, I would really suggest to drastically reduce the amount of triples for the sake of the presentation.

@robarto I hope you're not the one who is driving :) I big that from the empty ifgi :+1: Have a nice trip, greetz to uncle Sam !

mpfeil commented 11 years ago

Okay. I got it run. With the last one you mean getFOILastObservation? The getGraph is really fast :+1:

jimjonesbr commented 11 years ago

With the last one you mean getFOILastObservation?

Exactly. Also the getObservationTimeInterval is probably affected.

albhasan commented 11 years ago

I just deleted the triple store (it was broken anyway). This time I uploaded some properties just for a week instead of a month. The new triples agree with out new ontology interpretation but so far I'm not sure if they're "faster" to retrieve. Only the new queries will tell

jimjonesbr commented 11 years ago

Nice! :+1:
Can you adapt the queries to the new triples? I can then immediately update the factory!

albhasan commented 11 years ago

I'll do it later, I'll let you know. I'm not done with all the variables yet. I just uploaded some partial results for you to be able to keep working without breaking the server (I wish!)

It's never that easy

albhasan commented 11 years ago

@jimjonesbr : Can you please run some XSLTs in you computed tonight while you sleep? The new modifications implied a lot more of processing

jimjonesbr commented 11 years ago

absolutely! Just give me the instructions and I will let it running :-)

albhasan commented 11 years ago

Hi.

0 - Make sure your computer will not suspend or hybernate. 1 - Close other applications 2 - In dropbox, there is a file called "oneweek.7z". Download it and unzip it. 3 - In the "oneWeek" folder, there is a file called "europaXSLT_JIM.txt". Please open it, change the path accordingly and make it a bash. 4 - Run it from the terminal 5 - Wait for 10 minutes, if you got an error please give me a call

I was thinking that maybe we can make a bash from the whole process, I mean, go to the SOS, store the observations, transform them and upload them to the triple store.

Thanks, cheers!

2013/6/13 Jim Jones notifications@github.com

absolutely! Just give me the instructions and I will let it running :-)

— Reply to this email directly or view it on GitHubhttps://github.com/jimjonesbr/de.ifgi.europa/issues/2#issuecomment-19407378 .

albhasan commented 11 years ago

Hi: I updated the query "geFOILastObservation" and it's improved a lot.

@jimjonesbr You were right, the new ontology implementation made a huge difference. Can you please update the remainng query? I don't remember its name

jimjonesbr commented 11 years ago

Awesome! I just tested the query and the application no longer freezes :-) However, the globe does not show anything.

@mpfeil can you take a look? This is the error:

Exception in thread "AWT-EventQueue-0" java.lang.ArrayIndexOutOfBoundsException: 0 >= 0

Thanks, @albhasan !!

mpfeil commented 11 years ago

Wow it is really quick. @jimjonesbr this is my error: Exception in thread "AWT-EventQueue-0" com.hp.hpl.jena.query.QueryParseException: Encountered "" at line 1, column 2502. Was expecting one of:

... ... ... ... ... ... "select" ... "values" ... "graph" ... "optional" ... "minus" ... "bind" ... "service" ... "filter" ... "true" ... "false" ... ... ... ... ... ... ... ... ... ... ... ... ... ... "(" ... ... "{" ... "}" ... "[" ... ... This happens if I select a FOI and the error is called when I try to get the last FOI observation via the facade.

jimjonesbr commented 11 years ago

mm... it seems there is a problem in the query :( @albhasan do you have any idea?

jimjonesbr commented 11 years ago

@mpfeil I just corrected the query and committed the bug fix, now the getFOILastObservation works.

albhasan commented 11 years ago

Sorry guys, I guess I missed something in the query. Anyway, I was wondering about the units of measurement, I mean, the query for the latest observation retrieves just a number, should we show the user the units as well? Cheers!

jimjonesbr commented 11 years ago

@albhasan good idea! I didn't notice it was missing :) Let's add it..

By the way, the XSLT transformation is still running. I guess it started about 10:00 ago... shall I still wait a bit longer?

jimjonesbr commented 11 years ago

@albhasan I had to stop the process, I ran the script in console mode (no GUI) for 13:00h straight and it seemed to still be running. Sorry.

albhasan commented 11 years ago

No problem! We need to do what we can with what we have

2013/6/14 Jim Jones notifications@github.com

@albhasan https://github.com/albhasan I had to stop the process, I ran the script in console mode (no GUI) for 13:00h straight and it seemed to still be running. Sorry.

— Reply to this email directly or view it on GitHubhttps://github.com/jimjonesbr/de.ifgi.europa/issues/2#issuecomment-19449603 .

jimjonesbr commented 11 years ago

It'd be good if you could take a look on the files.. the folder with the output files is now ~300MB

albhasan commented 11 years ago

What do you mean?

albhasan commented 11 years ago

@mpfeil : The test dataset we have now goes from the 2013-06-06 to 2013-06-12. So, the data covers partially the flooding, can you see it in the app?

mpfeil commented 11 years ago

i will check it when i am back at ifgi

mpfeil commented 11 years ago

@albhasan i looked through our datasets and right it does not contain the waterlevel set which Arne mentioned today. Can you add it to our triplestore so that we can show the flooding?

here is the link to the 52N platform from today´s morning: http://sensorweb.demo.52north.org/sensorwebclient-webapp-stable/

albhasan commented 11 years ago

Hi:

I'm worried because I uploaded all the dataset we were able to transform, so probably that dataset was between the big ones we couldn't handle, even using Jim's machine.

albhasan commented 11 years ago

Hi:

@mpfeil Can you please test the app with this graph http://ifgi.uni-muenster.de/testNOAA/? - Thanks

mpfeil commented 11 years ago

i tried it and i get for every available property fois. :+1:

mpfeil commented 11 years ago

both opengis.net properties should not work right?

mpfeil commented 11 years ago

and for the mmisw foi is no SOSvalue available and the foi has no defaultgeometry

mpfeil commented 11 years ago

hey while i was testing i found a strange thing. i was selecting the property lufttemperatur and got 33 fois for it. i selected the foi Bramsche_xxxxxx. And nothing happend.

By debugging I found out that Bramsche has no observations for lufttemperatur. It only has Chlorid and Windrichtung and Windgeschwingkeit if I try to get the last observation.

ifgi / europa-browser

Create air quality dataset #2