search

project8 / hornet

Hornet is a nearline data processor for the Project 8 experiment
Other
0 stars 0 forks source link

windows compatibility #24

Closed laroque closed 9 years ago

laroque commented 9 years ago

Context

The docker/container based approach was started in hopes of not having to rewrite hornet for it to be compatible with windows.

In testing, it seems that the fact that on windows/mac the docker engine (and container) runs in a virtual machine causes the same break. It is possible to mount a host (mac/windows) directory on the container, but creating a file does not appear to create an inotify event to trigger hornet.

There are two obvious solutions:

  1. hornet could be made to do a filesystem poll and look for new files. This breaks some things that hornet was trying to solve (ie noticing when a new file is complete, not when it shows up but is still being written to, not having to poll the filesystem or the overhead that goes with that, etc.)
  2. the command docker exec <name or id of container> touch <path to data file in container filesystem> will trigger hornet. This eliminates the real-time aspect to hornet and requires either that mantis on windows possibly interact with docker (an extremely ugly and non-general hack), or that some process (possibly the mantis_interface dripline service) do so (much less ugly but certainly not properly real-time).

    Finally, the question:

Is this container-based solution actually useful, given that host-filesystem file creation doesn't trigger container-filesystem notification events? Are there other windows compatibility issues that this helps solve and makes it worth it?

laroque commented 9 years ago

As a followup; does anyone have thoughts about fsnotify (https://gopkg.in/fsnotify.v1)? It appears to be a similar library to inotify, possibly making it into the core of golang in 1.4. API looks similar but my very naive attempt to substitute it in was not fruitful (it was given very few cycles so I don't take this as an indication that it won't work). It appears to work on linux/mac/windows

nsoblath commented 9 years ago

Closing in deference to issue #7.