Create http/HMI surface

[johnnykv]

Create HMI using HTML.

• Simpel onepage HTML with JS, ActiveX or Java. (Siemens seems to favor ActiveX)
  • Shows gauges
  • Manipulation of controls
  • Need to look ICSish
• Use modbus backend for storing/retrieving values.

[glaslos]

This needs some tests now.

[glaslos]

Minimal test added in 8aff468620c1277145003feb6e26070106be0fcb

[creolis]

A very important step will be to provide the pages as authentic as possible while I'm not talking about the visual aspects but the directory / file structure.

Tools like nmap tend to look at certain patterns ( e.g. does image xyz.png exist at a certain location ) to decide what kind of service version it is dealing with. Further, we have to support extensions that are somewhat different to ".html" and ".htm" - e.g. ".mwsl" files used by Siemens that are basically just HTML files ( - I'm referring to the need for MIME types to avoid browsers trying to download the stuff instead of displaying it ).

I'm not sure, but an option to "proxy" connections to a 3rd party webserver would sound convenient, although this would raise additional questions in regard to logging requirements, configuration etc.

[glaslos]

IMO there are a couple of options or use cases:

• I have a real HMI with it's own web server which I want to connect to Conpot. This should work right now pretty much out of the box. This is obvious the best solution when it comes to providing an authentic HMI. Downsides, as you mentioned already, is the logging. We could add a transparent proxy that captures the HTTP requests, logs them and redirects to the real HMI.
• You have a couple of static HTML pages with some style files and images. You could use a standard HTTP server but again you would use the logging except you use a transparent proxy and in addition there would be no interaction between Conpot and the HMI (not sure if you could get some with JS clients for modbus/snmp).
• You have a static HTML HMI which you populate using our HTTP server and template engine. This would provided you with the most powerful result when it comes to integration (probably not the best when it comes to authenticity) but would also require quite some manual work.

[glaslos]

I had a look at some of the open source HMI's out there. Some run on Windows, some need a JavaRE. I guess it would be possible to integrate them but we would need a logging proxy in between.

[creolis]

Python based minimal proxy that could be used for logging:

    import SocketServer
    import SimpleHTTPServer
    import urllib

    port = 8080

    class Proxy(SimpleHTTPServer.SimpleHTTPRequestHandler):
        def do_GET(self):
            self.copyfile(urllib.urlopen(self.path), self.wfile)

    httpd = SocketServer.ForkingTCPServer(('', port), Proxy)
    print "serving at port", port
    httpd.serve_forever()

[glaslos]

Give it a try by putting it in front of the current HTTP server :)

[creolis]

For the static HMI approach I started to elaborate a way to provide a maximum of authenticity. This includes response headers etc.

Right now, I integrated the following XML subtree to the template, that seems to do the job ( at least it proposes a way to do the configuration ):

    <hmi>
        <headers>
            <entity name="Server">Simatic S7 Embedded Webserver</entity>
        </headers>
        <fs>
            <node name="/Default.mwsl">
                <headers>
                    <entity name="Last-Modified">Tue, 04 Feb 1997 09:22:12 GMT</entity>
                    <entity name="Content-Type">text/html</entity>
                    <entity name="Transfer-Encoding">chunked</entity>
                </headers>
            </node>
            <node name="/favicon.ico">
                <headers>
                    <entity name="Last-Modified">Tue, 04 Feb 1997 09:22:12 GMT</entity>
                    <entity name="Content-Type">binary/octet-stream</entity>
                </headers>
            </node>
            <node name="/CSS/S7Web.css">
                <headers>
                    <entity name="Last-Modified">Tue, 04 Feb 1997 09:22:12 GMT</entity>
                    <entity name="Content-Type">text/css</entity>
                </headers>
            </node>
        </fs>
    </hmi>

It shows how we could deal with filesystem structure etc. As soon as I have a proof of concept implementation for delivering the files securely ( preventing path traversal attacks etc. ) including access to current snmp values etc., I will get back to you with a preview :)

[glaslos]

@johnnykv and @creolis any objections about considering a different format than XML?

[johnnykv]

Nope. Got anything specific in mind?

[creolis]

No objections so far, but I started to implement my proof of concept with XML because I thought you would like to have all configuration stuff at one place and therefore within the central XML file :)

In the end it ( considering conpot to be ... more or less fully functional ) it would be nice to have full "packages" though .. nothing I fully thought through, I just imagine something like a "siemens_plc_1200.pot" you pass over to conpot and it extracts SNMP config, MODBUS config, HMI config + corresponding htdocs/webresources etc.

... but you made me curious .. what different format would you propose?

[johnnykv]

I certainly agree with you @creolis in that it is very usefull to have the entire configuration in a single file.

[creolis]

Just wanted to let you know what I'm working on right now ( to avoid possible redunancy and just keep you up to date a little ) -

Trying to allow the HMI module to be as flexible as possible, I noticed that most implementations ( e.g. WSGI etc. ) are "too intelligent". Example: in order to obey RFCs, they send headers where real HMIs (that have a rather sloppy approach to follow RFCs) would not send them. Some headers can not be modified with reference to RFCs at all. In the end, I realized that - to be able to get highly realistic behaviour - a lower level is needed.

My current approach implements BaseHTTPServer, which allows me to control every single aspect of the HTTP communication. Right now, I'm focused on standard content-length delivery, chunked delivery is on the roadmap since Siemens HMIs seems to prefer that for their dynamic content too.

Dynamic contents through replacing placeholders is not a big deal, but comes with certain limits where holding back a "static" page for every situation ever possible can be considered to be absurd. So I've made a plan on how to deal with real HMIs - you could call it a high-interaction-mode:

Since I already decode HTTP requests and store GET / POST parameters and cookies, spawning a request to a real HMI standing behind the honeypot is not a big deal any more. So if configured that way, requests and answers are relayed (proxied) back and forth, allowing easy logging of what's going on.

Right now, basic communication and error handling ( 404, .. ) are almost done, I will keep you up to date as soon as there's something you can try out :)

[glaslos]

@creolis Sounds pretty good :) Regarding the format: I'm not a big fan of XML and would rather use for example JSON. I also wouldn't mind having a single point of entry for the configuration (bin/conpot) and check there whether we are dealing with XML or JSON and decode accordingly. Or we could also go with ConfigObj: http://www.voidspace.org.uk/python/configobj.html

[creolis]

@glaslos, would you mind if I finish my current approach using XML for now? We could then do a coordinated transition to another solution ( like the ones you mentioned ) for an upcoming release.

[glaslos]

@creolis Not at all! Go ahead with whatever you have.

[creolis]

@glaslos @johnnykv - a sneak preview of my highly configurable HMI module / HTTP Server is available at 0d30c070f5cf66ddc58d43726ffec564b74f1c62. It should allow to replicate interfaces with a high grade of detail.

A running example, featuring a low-interaction copy of a PLC-1500, is available here for your reference: http://91.219.68.113:8000/ - it will be disabled after you had a look at it. Please note that some fields are pulled dynamically from SNMP ( Station Name, Module Uptime ) and evals ( Time and Date on the upper right ). The interfaces values are automatically updated every 10 seconds ( this is a feature of the templated Siemens HMI, not a feature of my module ).

The commit is full of debug outputs for now, of course they are going to be removed. The source code is heavily documented to increase readability.

FEATURES:

• Fully customizable headers ( per-server and per-file configurable )
• Fully customizable HTTP status code responses
• Support for GET method
• Support for POST method
• Support for HEAD method ( can be disabled in configuration )
• Support for TRACE method ( can be disabled in configuration )
• Dynamic header support ( "Date" response header will be updated at every request )
• Dynamic content support ( Tags found in text/html content type can be replaced with dynamic values gathered from SNMP and Python eval() )
• Backproxy support ( Requests to certain files or even status codes (e.g. 404) can be proxied to a remote HTTP server which is provided with original request data. The unmodified response is then delivered to the client ).
• Path traversal security ( Filesystem is fully specified in template - traversal is not possible ).
• Aliases can be configured ( for files and status codes ).

KNOWN ISSUES:

• chunked-encoding are not supported locally.
• chunked-encoding are modified through Backproxy ( it currently converts chunk-encoded answers to simple ( content-length powered ) answers.
• test cases are not available (yet).
• TRACE method currently modifies header capitalization.
• http module currently does not shut down correctly when conpot quits.

I'm looking forward to hear your opinion.

[glaslos]

OMG...

[glaslos]

@creolis why are you not sending a pull request?

[creolis]

I considered it to be way to unpolished for a pull req, because I'd like to distribute some functions to seperate files like it is handled at the SNMP part. Also, the debug loggig is something I'd like to remove before I submit a pull request. But of course I can, if it is of additional use for you..

[glaslos]

For me it's much better to debug and the conversation doesn't get lost. But it's up to you :)

[creolis]

k, I'll send one and resubmit the description too :)

( thanks for taking the time :) )

[creolis]

oh .. I think - except for some polishing - we are done here for the service part :)

[glaslos]

Yes, absolutely. Good job Daniel.