labgrid-project / labgrid

Embedded systems control library for development, testing and installation
https://labgrid.readthedocs.io/
Other
327 stars 164 forks source link

Place cannot be acquired because one of its resources is acquired by itself #748

Open egrigore opened 3 years ago

egrigore commented 3 years ago

Hi,

From time to time I meet this issue: that I can't acquire a place because one of its resources is already acquired by the place itself. I don't know in what situation it happens. Has anyone met the same issue?

Below is the error message I get and how the place resources look like (Matching resource '2' is acquired by the place):

(venv) testlabgrid@client-hostname:~/labgrid-tests/labgrid-config$ labgrid-client -p RO--MX8_evk--1 acquire
Matching resource '2' (exporter-hostname/RO--MX8_evk--1-console-2/NetworkSerialPort/USBSerialPort) already acquired by place 'RO--MX8_evk--1'
labgrid-client: error: failed to acquire place RO--MX8_evk--1
(venv) testlabgrid@client-hostname:~/labgrid-tests/labgrid-config$ labgrid-client -p RO--MX8_evk--1 show
Place 'RO--MX8_evk--1':
  matches:
    exporter-hostname/RO--MX8_evk--1/*
    exporter-hostname/RO--MX8_evk--1-console-0/NetworkSerialPort -> 0
    exporter-hostname/RO--MX8_evk--1-console-1/NetworkSerialPort -> 1
    exporter-hostname/RO--MX8_evk--1-console-2/NetworkSerialPort -> 2
    exporter-hostname/RO--MX8_evk--1-console-3/NetworkSerialPort -> 3
  acquired: None
  acquired resources:
  created: 2021-02-03 14:50:36.272348
  changed: 2021-03-10 00:18:44.230550
Matching resource 'NetworkPowerPort' (exporter-hostname/RO--MX8_evk--1/NetworkPowerPort/NetworkPowerPort):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkPowerPort',
   'params': {'host': '<pdu-service-IP>:8085/192.168.99.104',
              'index': 7,
              'model': 'simplehttp'}}
Matching resource 'NetworkService' (exporter-hostname/RO--MX8_evk--1/NetworkService/NetworkService):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkService',
   'params': {'address': '<pdu-service-IP>',
              'extra': {'proxy': 'exporter-hostname', 'proxy_required': False},
              'username': 'bamboo'}}
Matching resource 'USBResetPort' (exporter-hostname/RO--MX8_evk--1/NetworkUSBResetPort/USBResetPort):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkUSBResetPort',
   'params': {'board': 'imx8dxlevk',
              'busnum': 1,
              'devnum': 90,
              'extra': {'proxy': 'exporter-hostname', 'proxy_required': False},
              'host': 'exporter-hostname.domain.net',
              'model_id': 24593,
              'path': '1-1.1.2',
              'vendor_id': 1027}}
Matching resource 'USBSerialPort' (exporter-hostname/RO--MX8_evk--1/NetworkSerialPort/USBSerialPort):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkSerialPort',
   'params': {'extra': {'path': '/dev/ttyUSB63',
                        'proxy': 'exporter-hostname',
                        'proxy_required': False},
              'host': 'exporter-hostname.domain.net',
              'port': None,
              'speed': 115200}}
Matching resource 'UniversalUpdateUtility' (exporter-hostname/RO--MX8_evk--1/NetworkUniversalUpdateUtility/UniversalUpdateUtility):
  {'acquired': None,
   'avail': False,
   'cls': 'NetworkUniversalUpdateUtility',
   'params': {'busnum': None,
              'devnum': None,
              'extra': {'proxy': 'exporter-hostname', 'proxy_required': False},
              'host': 'exporter-hostname.domain.net',
              'model_id': None,
              'path': None,
              'vendor_id': None}}
Matching resource '0' (exporter-hostname/RO--MX8_evk--1-console-0/NetworkSerialPort/USBSerialPort):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkSerialPort',
   'params': {'extra': {'path': '/dev/ttyUSB52',
                        'proxy': 'exporter-hostname',
                        'proxy_required': False},
              'host': 'exporter-hostname.domain.net',
              'port': None,
              'speed': 115200}}
Matching resource '1' (exporter-hostname/RO--MX8_evk--1-console-1/NetworkSerialPort/USBSerialPort):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkSerialPort',
   'params': {'extra': {'path': '/dev/ttyUSB53',
                        'proxy': 'exporter-hostname',
                        'proxy_required': False},
              'host': 'exporter-hostname.domain.net',
              'port': None,
              'speed': 115200}}
Matching resource '2' (exporter-hostname/RO--MX8_evk--1-console-2/NetworkSerialPort/USBSerialPort):
  {'acquired': 'RO--MX8_evk--1',
   'avail': True,
   'cls': 'NetworkSerialPort',
   'params': {'extra': {'path': '/dev/ttyUSB63',
                        'proxy': 'exporter-hostname',
                        'proxy_required': False},
              'host': 'exporter-hostname.domain.net',
              'port': 42093,
              'speed': 115200}}
Matching resource '3' (exporter-hostname/RO--MX8_evk--1-console-3/NetworkSerialPort/USBSerialPort):
  {'acquired': None,
   'avail': True,
   'cls': 'NetworkSerialPort',
   'params': {'extra': {'path': '/dev/ttyUSB64',
                        'proxy': 'exporter-hostname',
                        'proxy_required': False},
              'host': 'exporter-hostname.domain.net',
              'port': None,
              'speed': 115200}}
Emantor commented 3 years ago

Excuse my late response, we have not seen this before. AFAICS the acquire check in remote/client.py may be to strict, as it requires the resources of a place to not be acquired at all, not even permitting the current place. OTOH this should not happen at all and indicates a synchronization/data race between the coordinator and exporter infrastructure.

Bastian-Krause commented 10 months ago

@egrigore Is this still an issue at your end with recent labgrid versions? We've never seen.

ElenaGrigore commented 10 months ago

Hi @Bastian-Krause , yes , it still appears from time to time (we rebased to latest upstream in the summer) - only for the named-match resources (if this helps).

Bastian-Krause commented 10 months ago

@egrigore Okay. Since you seem to be the only one able to reproduce this, investigation is probably up to you.

ElenaGrigore commented 10 months ago

@Bastian-Krause , yes , I will try to investigate - probably I need to think of a redesign of the current configurations logic. Thank you