search

SpectacularAI / sdk-examples

Spectacular AI SDK examples
Apache License 2.0
214 stars 34 forks source link

Unable to run Mapping_visu with oak-D #35

Closed MasIgor closed 1 year ago

MasIgor commented 2 years ago

Hello!

I am trying to run the dem mapping_visu with a oak-D Pro POE camera.

but when i start the demo I get this as error message:


Starting OAK-D device
Close the window to stop mapping
Spectacular AI SDK: WARN: unrecoginzed OAK board name 'NG9097'
you may need to manually set IMU-to-camera extrinsics (configuration.imuToCameraLeft)
[Open3D WARNING] GLFW Error: WGL: Failed to make context current: Der angeforderte Transformationsvorgang wird nicht unterstützt.
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\in\Anaconda3\envs\DepthAi\lib\threading.py", line 973, in _bootstrap_inner
    self.run()
  File "C:\Users\in\Anaconda3\envs\DepthAi\lib\threading.py", line 910, in run
    self._target(*self._args, **self._kwargs)
  File "E:\Workfolder\DepthAi\SpectacularAI\sdk-examples\python\oak\mapping_visu.py", line 244, in captureLoop
    vioPipeline.startSession(device) as vio_session:
RuntimeError: Spectacular AI SDK:  error: LOW USB SPEED!

############################################################################
Your USB connection speed 'UNKNOWN' is lower than required.
Please check that you have connected the device to an USB 3 port with an
undamaged USB 3 cable (blue interior). Speed SUPER or better is recommended.
Disable this check by setting config.ensureSufficientUsbSpeed to false.
############################################################################

My camera is not a USB camera, its a Power Over Ethernet camera.

Can this be solved?

Thank you!

Best regards

Bercon commented 2 years ago

Do you have latest Spectacular AI SDK version? It should just give a warning for UNKNOWN USB speed i.e. ethernet and not fail on it.

In any case, you can disable the check with config.ensureSufficientUsbSpeed = False.

MasIgor commented 2 years ago

Thank you! Yes it is the latest. I just installed it today. I will test with the usb check disabled tomorrow when inam back at the pc. Thank you

From: Jerry Ylilammi @.> Sent: Tuesday, September 13, 2022 4:38:14 PM To: SpectacularAI/sdk-examples @.> Cc: tanzerlana @.>; Author @.> Subject: Re: [SpectacularAI/sdk-examples] Unable to run Mapping_visu with oak-D (Issue #35)

Do you have latest Spectacular AI SDK version? It should just give a warning for UNKNOWN USB speed i.e. ethernet and not fail on it.

In any case, you can disable the check with config.ensureSufficientUsbSpeed = False.

— Reply to this email directly, view it on GitHubhttps://github.com/SpectacularAI/sdk-examples/issues/35#issuecomment-1245510930, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQGO4ZZOQIDBIVWRX7MWMLTV6CGVNANCNFSM6AAAAAAQLQB6C4. You are receiving this because you authored the thread.Message ID: @.***>

MasIgor commented 2 years ago

Hello @Bercon, I tried as sugested, and this is the output:

` Starting OAK-D device Close the window to stop mapping Spectacular AI SDK: WARN: unrecoginzed OAK board name 'NG9097' you may need to manually set IMU-to-camera extrinsics (configuration.imuToCameraLeft) Spectacular AI SDK: WARN: LOW USB SPEED!

############################################################################ Your USB connection speed 'UNKNOWN' is lower than required. Please check that you have connected the device to an USB 3 port with an undamaged USB 3 cable (blue interior). Speed SUPER or better is recommended. Disable this check by setting config.ensureSufficientUsbSpeed to false. ############################################################################

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: Long sample sync IMU queue 503 SpectacularAI WARN: Long sample sync IMU queue 504 SpectacularAI WARN: Long sample sync IMU queue 505 SpectacularAI WARN: Long sample sync IMU queue 506 SpectacularAI WARN: Long sample sync IMU queue 507 SpectacularAI WARN: Long sample sync IMU queue 508 SpectacularAI WARN: Long sample sync IMU queue 509 SpectacularAI WARN: Long sample sync IMU queue 510 SpectacularAI WARN: Long sample sync IMU queue 511 SpectacularAI WARN: Long sample sync IMU queue 512 SpectacularAI WARN: Long sample sync IMU queue 513 . . . SpectacularAI WARN: Long sample sync IMU queue 22658 `

MasIgor commented 2 years ago

after changing the POE Power Injector with a faster one the new message is this:

(DepthAi) E:\Workfolder\DepthAi\SpectacularAI\sdk-examples\python\oak>python mapping_visu.py
Starting OAK-D device
Close the window to stop mapping
Spectacular AI SDK: WARN: unrecoginzed OAK board name 'NG9097'
you may need to manually set IMU-to-camera extrinsics (configuration.imuToCameraLeft)
Spectacular AI SDK: WARN: LOW USB SPEED!

############################################################################
Your USB connection speed 'UNKNOWN' is lower than required.
Please check that you have connected the device to an USB 3 port with an
undamaged USB 3 cable (blue interior). Speed SUPER or better is recommended.
Disable this check by setting config.ensureSufficientUsbSpeed to false.
############################################################################

[Open3D WARNING] GLFW Error: WGL: Failed to make context current: Der angeforderte Transformationsvorgang wird nicht unterstützt.
[Open3D WARNING] GLFW Error: WGL: Failed to make context current: Das Handle ist ungültig.
MasIgor commented 2 years ago

After changing my GPU settings, I get to this, and it just sits there:

Starting OAK-D device
Close the window to stop mapping
Spectacular AI SDK: WARN: unrecoginzed OAK board name 'NG9097'
you may need to manually set IMU-to-camera extrinsics (configuration.imuToCameraLeft)
Spectacular AI SDK: WARN: LOW USB SPEED!

############################################################################
Your USB connection speed 'UNKNOWN' is lower than required.
Please check that you have connected the device to an USB 3 port with an
undamaged USB 3 cable (blue interior). Speed SUPER or better is recommended.
Disable this check by setting config.ensureSufficientUsbSpeed to false.
############################################################################

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer
Bercon commented 2 years ago

It sounds like that the ethernet cable can't provide sufficient bandwidth or your host machine isn't fast enough to process the data. Does the vio_jsonl.py example work? You can also try disabling stereo, in that mode the required bandwidth and processing power should be minimal.

MasIgor commented 2 years ago

I tried, same problem:


(DepthAi) E:\Workfolder\DepthAi\SpectacularAI\sdk-examples\python\oak>python vio_jsonl.py
Spectacular AI SDK: WARN: unrecoginzed OAK board name 'NG9097'
you may need to manually set IMU-to-camera extrinsics (configuration.imuToCameraLeft)
Spectacular AI SDK: WARN: LOW USB SPEED!

############################################################################
Your USB connection speed 'UNKNOWN' is lower than required.
Please check that you have connected the device to an USB 3 port with an
undamaged USB 3 cable (blue interior). Speed SUPER or better is recommended.
Disable this check by setting config.ensureSufficientUsbSpeed to false.
############################################################################

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

SpectacularAI WARN: SampleSync frame buffer size 10, culling buffer

I also tried setting the disabled stereo flag, but got this error:

TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. spectacularAI.depthai.Pipeline(arg0: object)
    2. spectacularAI.depthai.Pipeline(arg0: object, arg1: spectacularAI.depthai.Configuration)
    3. spectacularAI.depthai.Pipeline(arg0: object, arg1: spectacularAI.depthai.Configuration, arg2: Callable[[spectacularAI.mapping.MapperOutput], None])
    4. spectacularAI.depthai.Pipeline(arg0: object, **kwargs)

Invoked with: <depthai.Pipeline object at 0x000001E630614BF0>, <spectacularAI.depthai.Configuration object at 0x000001E6308D53F0>; kwargs: useStereo=False
Bercon commented 2 years ago

The documentation is slightly outdated there, the correct method to disabling stereo is following:

config = spectacularAI.depthai.Configuration()
config.useStereo = False
vio_pipeline = spectacularAI.depthai.Pipeline(pipeline, config)
MasIgor commented 2 years ago

that seems to work. it loads for about a minute and then shoots out a load of Json packets like this one:


{"acceleration":{"x":-3.562626152189312e-05,"y":-9.360115094463537e-06,"z":-1.7128010347064e-06},"angularVelocity":{"x":-0.03507554531097412,"y":0.014162994921207428,"z":0.011789784766733646},"orientation":{"w":0.5003422102922508,"x":-0.4507721150048008,"y":-0.3266067597656436,"z":-0.6631667945617835},"position":{"x":-0.10315117526698116,"y":-0.18983851679605998,"z":-0.21568761040622178},"status":"TRACKING","time":1325396.971086613,"velocity":{"x":-0.0008178522842683411,"y":0.014236709400918321,"z":0.03406199132179283}}
{"acceleration":{"x":-3.553308287019327e-05,"y":-9.335634195626549e-06,"z":-1.7083212918362151e-06},"angularVelocity":{"x":-0.035030804574489594,"y":0.014144321903586388,"z":0.011774029582738876},"orientation":{"w":0.5003427282860365,"x":-0.4507655874669095,"y":-0.326602634251735,"z":-0.6631728724146703},"position":{"x":-0.10314829385997999,"y":-0.18983794970352166,"z":-0.21569168290992646},"status":"TRACKING","time":1325397.003647613,"velocity":{"x":-0.0008230841523639066,"y":0.014219783133773586,"z":0.03404771681229724}}
{"acceleration":{"x":-3.527495190283181e-05,"y":-9.26781524800939e-06,"z":-1.6959111491746063e-06},"angularVelocity":{"x":-0.034992534667253494,"y":0.014080983586609364,"z":0.011707807891070843},"orientation":{"w":0.5004878728992255,"x":-0.45086389109644665,"y":-0.3267166838153834,"z":-0.6629403059875443},"position":{"x":-0.10290456958986476,"y":-0.1897089354013624,"z":-0.21599308683012583},"status":"TRACKING","time":1325397.036087613,"velocity":{"x":-0.0012909409241483373,"y":0.01269712899984709,"z":0.03387061552690823}}
{"acceleration":{"x":-3.517595698035826e-05,"y":-9.241806235877974e-06,"z":-1.6911517779018397e-06},"angularVelocity":{"x":-0.034944500774145126,"y":0.014061071909964085,"z":0.01169106736779213},"orientation":{"w":0.500488613114199,"x":-0.4508574741846737,"y":-0.32671260106322103,"z":-0.6629461233172317},"position":{"x":-0.10290174262059058,"y":-0.18970826884777178,"z":-0.21599708770928336},"status":"TRACKING","time":1325397.071259613,"velocity":{"x":-0.001295467735396013,"y":0.012681777467588857,"z":0.033856761793625366}}
Bercon commented 2 years ago

You could try some other DepthAI examples to see how well they work. It sounds like there isn't enough bandwidth to transmit the depth images at 30 fps required by the stereo mode. I have previously tested OAK-D PoE with stereo and it seemed to work without issues, so it could be a network issue.

oseiskar commented 1 year ago

Seems to be more related to the PoE camera and not specifically to the mapping_visu example. Similar issues with PoE also discussed here https://github.com/SpectacularAI/sdk-examples/issues/42 so let's continue the discussion there if the issue persists