race condition while auto-restart can result in hub not restarting

[sirloon]

When a new data plugin is registered for instance, and if config.USE_RELOADER is True, the hub restarts to load new code. Problem is if the hub takes a while to stop, the spawn process will fail while starting, giving a "OSError: [Errno 98] Address already in use".

Also, it 2 restart event are gathered almost at the same time, the hub may fail to restart as well.

see: https://github.com/biothings/biothings.api/blob/master/biothings/utils/hub.py#L130

• check for port not being open to make sure hub is stopped before restarting it ?
• dump a file with PID to inform the spawned process to wait until PID is dead ?

[sirloon]

fix (hopefully) in https://github.com/biothings/biothings.api/commit/5fd6c68b52460f6ca69525f710af8c15ab33fecf