search

52North / sensorweb-server-helgoland

Thin Web binding API to access series data.
GNU General Public License v2.0
24 stars 21 forks source link

SOS instances management #158

Open zbodrosk opened 8 years ago

zbodrosk commented 8 years ago

(this issue could be also related to restful-timeseries-proxy project)

Is there a feature that would allow dynamic SOS instances adding? I'm aware that similar process could manually be done by editing sos-instances.data.xml in the restful-timeseries-proxy, but what I want is to do the same by the service call without the need to redeploy and to cache all the other SOS instances again.

If there is an easier or better way to manage custom SOS instances, I'm open for suggestions.

It seems that similar problem has already been addressed here #111

ridoo commented 8 years ago

@zbodrosk currently there is no external trigger to update the SOS cache, just the continuous cache update job. Unfortunately, it does not read in the sos-instances.data.xml file again so no new SOS instances can be added at runtime. However, this could be added easily by providing a proper method in the ConfigurationContext class.

In what you describe I think @nuest nailed down the simplest solution already: Creating a file watcher which starts a cache reload after some regular time interval. If you want to re-write the config file via remote trigger, then you'll get into user authentication to prevent every user from reloading or adding SOS instances. This can cause very long harvesting times.

If you are willing to dive into this

  1. fork SensorWebClient
  2. create new branch for your development
  3. have a look at ConfigurationContext and SosInstanceHandler how SOS instances are read into memory.
  4. create a rest new Spring MVC endpoint in the SOS REST module.
  5. If this works you should have a look at Spring Security and put an In-Memory Authentication Provider to make sure you can protect against unintended service adding and server loads.
  6. make a pull request so I can review
zbodrosk commented 8 years ago

@ridoo Sorry for late reply. In the meantime, as you suggested, we developed feature that reads SOS URLs from the other sources, validates them, and write changes into the sos-instances.data.xml.

But, there is a problem with this approach. All changes made in the sos-instance.data.xml during a runtime will not be persistent after re-deploy. Of course, it's possible (but not convenient) to write directly into the war file, but that will automatically trigger a re-deploy.

We could suggest a few solutions to overcome this problem, but we are not sure is it a right direction to go. In general, if we want to update metadata list during a runtime, content of the sos-instance.data.xml should be saved separate from deployment structure. Otherwise, if we want to save existing code structure, it is also possible to maintain that data separately from sos-instance.data.xml.

Please, give us your opinion and suggestions.

ridoo commented 8 years ago

@zbodrosk saving it separatly is the best way to go. I would go for %HOME%\.52n\sensorwebclient\sos-instances.data.xml or $HOME/.52n/sensorwebclient/sos-instances.data.xml where the SOS update mechanism searches first. If not present it would read the deployed sos-instances.data.xml.

What do you think?

zbodrosk commented 8 years ago

@ridoo I agree. Maybe just to add a small correction. I think that it is a good idea to have both files. I'll try to explain.

If we want to create new file (in home folder) and after that to use just this one, we definitely need to include content of the old one (from deployment). Otherwise, after redeploy, we lose initial SOS list. But... If someone change initial xml (from initial project), those changes wont appear in runtime after redeploy. That's why a suggest to use both files by simply check the existence of the one from home folder and if it's there to read its content (in the same manner as we do for the one from deploy). Maybe it is not the most elegant solution, but it will separate 'custom' SOS list from initial one.

What do you say?

ridoo commented 8 years ago

well, actually what I said. Both files are allowed. If the first under user's home is found the other is ignore. otherwise the one under local Web context is read.

@zbodrosk could you implement this in a feature branch and create a pull request? I do not see you having signed a 52n-CLA. This would be needed by us to incorporate your changes (and maintain that feature onwards). Just have a look at our CLA guidelines.

zbodrosk commented 8 years ago

@ridoo Yes, we are going to implement this. We haven't signed 52n-CLA yet, but planning to.

ridoo commented 8 years ago

@zbodrosk cool. Let me know if you need more input.