Socket error after gp_camera_exit in windows/msys2

[jensanjo]

System OS: Windows 10 + MSYS2 MingW 64-bit Python 3.8 libgphoto2-2.5.27 python-gphoto2 2.3.4 camera: Canon EOS 1200D

The problem

After initializing the camera and then calling cam.exit() an exception occurs when i create a socket. Here is a simple program that illustrates the issue:

import socket
import gphoto2 as gp

# This works
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

cam = gp.Camera()
cam.init()
cfg = cam.get_config()
wid = cfg.get_child_by_name('cameramodel')
model = wid.get_value()
print(f"Detected {model}")
cam.exit()

# after cam init/exit creating a socket fails: 
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

Results in:

$ python camtest.py 
Detected Canon EOS 1200D
Traceback (most recent call last):
  File "camtest.py", line 16, in <module>
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
  File "C:/msys64/mingw64/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [WinError 10093] Either the application has not called WSAStartup, or WSAStartup failed

After calling exit all subsequent socket operations fail. The lines that read and print the camera model are just for illustration, they are not relevant to the issue.

[jim-easterbrook]

I think this must be something internal to libgphoto2. There's no use of socket in the python interface stuff generated by SWIG. Googling WinError 10093 finds loads of apparently unconnected things, so I don't really know where to start.

[jensanjo]

I will see if I can reproduce with a similar C program and libgphoto2.

[jim-easterbrook]

It seems that WSAStartup and WSACleanup must be called in matched pairs, so if Camera.exit() was somehow calling WSACleanup you'd get this problem. I've briefly looked at the source for Camera.exit() and it looks reasonably benign, but there could easily be something hiding.

Another possibility is that python-gphoto2 is somehow corrupting memory it shouldn't be and upsetting the Python socket library. Do you still have the problem with nothing between cam.init() and cam.exit()? (The get_config stuff is probably where I'm most likely to have memory problems...)

[jim-easterbrook]

During my daily (if only!) constitutional walk I thought of something - MSYS2 patches source files as part of its build process, particularly for things that don't support Windows, such as libgphoto2. I've just found this: https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-libgphoto2/libgphoto2-configure-ws232.patch I don't know what it means, but it appears to be removing a check for WSAStartup from the configuration.

[jim-easterbrook]

I wonder if the MSYS2 Python socket package is using Winsock or Winsock2? There seems to be plenty of room for problems in all this stuff...

[jim-easterbrook]

I've updated my old MSYS2 installation (on a Windows 7 virtual machine - it's the only Windows I've got) and installed libgphoto2 and python-gphoto2 but I don't have a working installation yet. It's looking for iolibs in a non-existent directory.

PS It might be worth setting gphoto2 to use Python logging (as in all the examples) to see if it has any useful error messages.

[jensanjo]

Thanks for your helpful suggestions! I turned on logging at DEBUG level in the python program but it did not show anything.

I created a small C program to check if the same problem occurs with just the libgphoto2 library. Basically, it does the same as my python test program. It opens a socket, initializes and exits the camera, and then opens a second socket.

Expected outcome is that this all happens without errrors. When I run it, the same happens as in the python program: creating the first socket and init/exit camera works fine, but creating the second socket fails with the infamous error 10093.

$ ./camtest.exe
Create first socket: OK
Initialize camera
Exit camera
Create second socket: Errro creating socket: 10093

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include <gphoto2/gphoto2-camera.h>
#include <winsock2.h>

static void check_socket(SOCKET sock) {
    if (sock == INVALID_SOCKET)
    {
        printf("Errro creating socket: %ld\n", WSAGetLastError());
        WSACleanup();
        exit(EXIT_FAILURE);
    }
}

int main(int argc, char *argv[])
{
    printf("Hello world");
    WSADATA wsaData;
    int res;

    res = WSAStartup(MAKEWORD(2,2), &wsaData);
    if (res != 0) {
        printf("WSAStartup failed: %d\n", res);
        exit(EXIT_FAILURE);
    }

    SOCKET sock;

    printf("Create first socket: ");
    sock = socket(AF_INET, SOCK_DGRAM, 0);
    check_socket(sock);
    printf("OK\n");

    printf("Initialize camera\n");
    Camera *camera;
    assert(GP_OK == gp_camera_new(&camera));
    assert(GP_OK == gp_camera_init(camera, NULL));
    printf("Exit camera\n");
    assert(GP_OK == gp_camera_exit(camera, NULL));

    printf("Create second socket: ");
    sock = socket(AF_INET, SOCK_DGRAM, 0);
    check_socket(sock);
    printf("OK\n");

    return 0;
}

[jim-easterbrook]

OK, this is definitely a problem with libgphoto2 and/or MSYS2. It might be worth raising on the gphoto2 mailing list.

[jim-easterbrook]

PS My MSYS2 libgphoto2 is on version 2.5.30 - it might be worth checking the problem is still there after updating.

[jensanjo]

Thanks. I will do that. I found a workaround by inserting a WSAStartup after the gp_camera_exit. So that proves our assumption. It has nothing to do with python-gphoto2. I found that WSAStartup and WSACleanup are only used in ptp2/library.c, in the camera_init and camera_exit functions respectively. Perhaps they don't match up somehow.

[jim-easterbrook]

Definitely worth raising with libgphoto2 though. @msmeissn

[jim-easterbrook]

According to the WSAStartup docs:

An application must call the WSACleanup function for every successful time the WSAStartup function is called. This means, for example, that if an application calls WSAStartup three times, it must call WSACleanup three times. The first two calls to WSACleanup do nothing except decrement an internal counter; the final WSACleanup call for the task does all necessary resource deallocation for the task.

In camlibs/ptp2/library.c the init will fail if WSAStartup fails, so it should not call WSACleanup incorrectly. I'm mystified.

[jim-easterbrook]

Another thought: libgphoto2 is requesting Windows sockets version 1.1.

The wHighVersion member of the WSADATA structure indicates the highest version of the Windows Sockets specification that the Winsock DLL supports. The wVersion member of the WSADATA structure indicates the version of the Windows Sockets specification that the Winsock DLL expects the caller to use.

After libgphoto2 calls WSAStartup Windows will be expecting v1.1 calls. Your socket initialisation requests v2.2 - is it possible that libgphoto2 changes the state of Windows sockets? That doesn't explain why it's only after calling gp_camera_exit that there's a problem though.

[jensanjo]

I found the problem in camlibs/ptp2/library.c. The WSAStartup is called in camera_init, but only if the camera port type is GP_PORT_PTPIP:

    case GP_PORT_PTPIP: {
        GPPortInfo  info;
        char        *xpath;

#if defined(HAVE_LIBWS232) && defined(WIN32)
        WORD wsaVersionWanted = MAKEWORD(1, 1);
        WSADATA wsaData;
        if (WSAStartup(wsaVersionWanted, &wsaData)) {
            GP_LOG_E("WSAStartup failed.");
            return GP_ERROR;
        }
#endif

In camera_exit WSACleanup is called, but in this only if the port type is NOT GP_PORT_PTPIP, which is obviously not what was intended. Since my port type is GP_PORT_USB the WSAStartup is not called, but the WSACleanup is.

#if defined(HAVE_LIBWS232) && defined(WIN32)
    else if ((camera->port!=NULL) && camera->port->type != GP_PORT_PTPIP) {
        WSACleanup();
    }
#endif

I will raise this with libgphoto2.

[jim-easterbrook]

Well spotted!