Virtual Sensor, compression

[nicolas-f]

Hi,

A question about virtual sensor over SOS&SPS. Is there any compression of data when transferring records to another SOS (http or websockets) ?

If no, have you an idea to achieve compression of data ?

[nicolas-f]

Maybe this is a jetty configuration.

There is no xml configuration for jetty with opensensorweb ?

https://blog.max.berger.name/2010/01/jetty-7-gzip-filter.html

[nicolas-f]

I will try editing

https://github.com/opensensorhub/osh-core/blob/master/sensorhub-core/src/main/java/org/sensorhub/impl/service/HttpServer.java

using this method:

https://stackoverflow.com/questions/22700026/jetty-gzip-filter

[alexrobin]

Hi Nicolas,

Thanks for your feedback.

Sorry this is not well documented yet, but you can add a jetty.xml for all custom configurations. You can point to the file in the HttpServer config, see https://github.com/opensensorhub/osh-core/blob/master/sensorhub-core/src/main/java/org/sensorhub/impl/service/HttpServerConfig.java#L70

IDs of preconfigured Jetty components are pre-defined so you can append or override their config. See the IDs in https://github.com/opensensorhub/osh-core/blob/master/sensorhub-core/src/main/java/org/sensorhub/impl/service/HttpServer.java#L262

For example, adding DOS filter to all servlets can be done this way:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_0.dtd">
<Configure id="osh-server" class="org.eclipse.jetty.server.Server">
  <Ref refid="osh-servlets">        
    <Call name="addFilter">
      <Arg>org.eclipse.jetty.servlets.DoSFilter</Arg>
      <Arg>/*</Arg>
      <Arg>
        <Call class="java.util.EnumSet" name="of">
          <Arg>
            <Call class="javax.servlet.DispatcherType" name="valueOf">
              <Arg>REQUEST</Arg>
            </Call>
          </Arg>
        </Call>
      </Arg>      
      <Call name="setInitParameter">
        <Arg>maxRequestsPerSec</Arg>
        <Arg>20</Arg>
      </Call>
      <Call name="setInitParameter">
        <Arg>delayMs</Arg>
        <Arg>100</Arg>
      </Call>
      <Call name="setInitParameter">
        <Arg>remotePort</Arg>
        <Arg>true</Arg>
      </Call>
    </Call>
  </Ref>
</Configure>

Let me know if it works for you as this feature hasn't been thoroughly tested.

Thanks,

[nicolas-f]

Hi,

Thanks for the answer.

I've tried this file

<?xml version="1.0" encoding="UTF-8"?>
<Configure id="OSHServer" class="org.eclipse.jetty.server.Server">
  <Call name="insertHandler">
    <Arg>
      <New id="GzipHandler" class="org.eclipse.jetty.server.handler.gzip.GzipHandler">
        <Set name="minGzipSize"><Property name="jetty.gzip.minGzipSize" deprecated="gzip.minGzipSize" default="0"/></Set>
        <Set name="checkGzExists"><Property name="jetty.gzip.checkGzExists" deprecated="gzip.checkGzExists" default="false"/></Set>
        <Set name="compressionLevel"><Property name="jetty.gzip.compressionLevel" deprecated="gzip.compressionLevel" default="-1"/></Set>
        <Set name="inflateBufferSize"><Property name="jetty.gzip.inflateBufferSize" default="0"/></Set>
        <Set name="syncFlush"><Property name="jetty.gzip.syncFlush" default="false" /></Set>

        <Set name="excludedAgentPatterns">
          <Array type="String">
            <Item><Property name="jetty.gzip.excludedUserAgent" deprecated="gzip.excludedUserAgent" default=".*MSIE.6\.0.*"/></Item>
          </Array>
        </Set>

        <Set name="includedMethodList"><Property name="jetty.gzip.includedMethodList" default="GET" /></Set>
        <Set name="excludedMethodList"><Property name="jetty.gzip.excludedMethodList" default="" /></Set>

      </New>
    </Arg>
  </Call>
</Configure>

There is no issue to parse the xml file. But the results are not gziped. I don't know how to debug this.

[alexrobin]

Nicolas, FYI we have successfully enabled GZip compression for one of our projects, with the following Jetty XML config file:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_0.dtd">
<Configure id="osh-server" class="org.eclipse.jetty.server.Server">
  <Set name="handler">
    <New class="org.eclipse.jetty.server.handler.gzip.GzipHandler">
      <Set name="includedMethods">
        <Array type="java.lang.String">
          <Item>GET</Item>
          <Item>POST</Item>
        </Array>
      </Set>
      <Set name="minGzipSize">50</Set>
      <Set name="handler">
        <Ref refid="osh-handlers"/>
      </Set>
    </New>      
  </Set>
</Configure>