Open christianPetrucci opened 8 years ago
So the first issue is caused by the SmartGaze CMAKE build not being able to find where you put Halide. It searches in standard system library install paths, not random folders on your system. If you want it to find your install of Halide you could try figuring out a magic variable to pass CMAKE, or I think looking up how the HINTS argument works for find_path
and find_library
cmake functions and editing SmartGaze's src/CmakeLists.txt
file.
For the uvc open issue, that sounds like a problem connecting to the Eye Tribe camera. You'll need to make sure that the Eye Tribe is connected to the VM and the virtual machine and physical port must support USB 3.0.
I forgot to explicitly mention another solution to your Halide issue, which is just to install it to a standard location, possibly with make install
.
Can You write me commands for install halide?
Or new cmakeLists.txt please
i have llvm in /ust/local/include OK
what modifies can add to Halide's Makefile and where?
i don't understand why llvm and clang are in url/local/include but halide not. where i use export command?
now i have: home/ /llvm3.7/...... /Halide/ ....... it's right? help me please
make install in not valid for halide
for the simple version of smartgaze i have uvc status: access denied. if i use command lsusb -v i have:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Couldn't open device, some information will be missing Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 3.00 bDeviceClass 9 Hub bDeviceSubClass 0 Unused bDeviceProtocol 3 bMaxPacketSize0 9 idVendor 0x1d6b Linux Foundation idProduct 0x0003 3.0 root hub bcdDevice 4.02 iManufacturer 3 iProduct 2 iSerial 1 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 31 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0xe0 Self Powered Remote Wakeup MaxPower 0mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 9 Hub bInterfaceSubClass 0 Unused bInterfaceProtocol 0 Full speed (or root) hub iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0004 1x 4 bytes bInterval 12 bMaxBurst 0
Bus 001 Device 008: ID 80ee:0021 VirtualBox USB Tablet Couldn't open device, some information will be missing Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 1.10 bDeviceClass 0 (Defined at Interface level) bDeviceSubClass 0 bDeviceProtocol 0 bMaxPacketSize0 8 idVendor 0x80ee VirtualBox idProduct 0x0021 USB Tablet bcdDevice 1.00 iManufacturer 1 iProduct 3 iSerial 0 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 34 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0x80 (Bus Powered) MaxPower 100mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 3 Human Interface Device bInterfaceSubClass 0 No Subclass bInterfaceProtocol 0 None iInterface 0 HID Device Descriptor: bLength 9 bDescriptorType 33 bcdHID 1.10 bCountryCode 0 Not supported bNumDescriptors 1 bDescriptorType 34 Report wDescriptorLength 85 Report Descriptors: \ UNAVAILABLE ** Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0008 1x 8 bytes bInterval 10
Bus 001 Device 007: ID 29ab:00fb
Couldn't open device, some information will be missing
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.10
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
idVendor 0x29ab
idProduct 0x00fb
bcdDevice 0.00
iManufacturer 1
iProduct 2
iSerial 3
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 265
bNumInterfaces 2
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 500mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 2
bFunctionClass 14 Video
bFunctionSubClass 3 Video Interface Collection
bFunctionProtocol 0
iFunction 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 1 Video Control
bInterfaceProtocol 0
iInterface 0
VideoControl Interface Descriptor:
bLength 13
bDescriptorType 36
bDescriptorSubtype 1 (HEADER)
bcdUVC 1.00
wTotalLength 80
dwClockFrequency 25.187500MHz
bInCollection 1
baInterfaceNr( 0) 1
VideoControl Interface Descriptor:
bLength 18
bDescriptorType 36
bDescriptorSubtype 2 (INPUT_TERMINAL)
bTerminalID 1
wTerminalType 0x0201 Camera Sensor
bAssocTerminal 0
iTerminal 0
wObjectiveFocalLengthMin 1
wObjectiveFocalLengthMax 98
wOcularFocalLength 1
bControlSize 3
bmControls 0x00000a2a
Auto-Exposure Mode
Exposure Time (Absolute)
Focus (Absolute)
Zoom (Absolute)
PanTilt (Absolute)
VideoControl Interface Descriptor:
bLength 12
bDescriptorType 36
bDescriptorSubtype 5 (PROCESSING_UNIT)
Warning: Descriptor too short
bUnitID 2
bSourceID 1
wMaxMultiplier 16384
bControlSize 3
bmControls 0x00001668
Saturation
Gamma
White Balance Temperature
Gain
Power Line Frequency
White Balance Temperature, Auto
iProcessing 0
bmVideoStandards 0x1c
PAL - 625/50
SECAM - 625/50
NTSC - 625/50
VideoControl Interface Descriptor:
bLength 28
bDescriptorType 36
bDescriptorSubtype 6 (EXTENSION_UNIT)
bUnitID 3
guidExtensionCode {e121e378-acc8-a540-8ac9-75a2a02c74fb}
bNumControl 1
bNrPins 1
baSourceID( 0) 2
bControlSize 3
bmControls( 0) 0xff
bmControls( 1) 0xff
bmControls( 2) 0x00
iExtension 0
VideoControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 3 (OUTPUT_TERMINAL)
bTerminalID 4
wTerminalType 0x0101 USB Streaming
bAssocTerminal 0
bSourceID 3
iTerminal 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 8
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
VideoStreaming Interface Descriptor:
bLength 14
bDescriptorType 36
bDescriptorSubtype 1 (INPUT_HEADER)
bNumFormats 1
wTotalLength 131
bEndPointAddress 131
bmInfo 0
bTerminalLink 4
bStillCaptureMethod 1
bTriggerSupport 0
bTriggerUsage 0
bControlSize 1
bmaControls( 0) 27
VideoStreaming Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)
bFormatIndex 1
bNumFrameDescriptors 3
guidFormat {59555932-0000-1000-8000-00aa00389b71}
bBitsPerPixel 16
bDefaultFrameIndex 1
bAspectRatioX 8
bAspectRatioY 6
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 2 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
VideoStreaming Interface Descriptor:
bLength 30
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 1
bmCapabilities 0x03
Still image supported
Fixed frame-rate
wWidth 2304
wHeight 1536
dwMinBitRate 35389440
dwMaxBitRate 35389440
dwMaxVideoFrameBufferSize 7077888
dwDefaultFrameInterval 2000000
bFrameIntervalType 1
dwFrameInterval( 0) 2000000
VideoStreaming Interface Descriptor:
bLength 30
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 2
bmCapabilities 0x03
Still image supported
Fixed frame-rate
wWidth 1344
wHeight 380
dwMinBitRate 20428800
dwMaxBitRate 20428800
dwMaxVideoFrameBufferSize 1021440
dwDefaultFrameInterval 500000
bFrameIntervalType 1
dwFrameInterval( 0) 500000
VideoStreaming Interface Descriptor:
bLength 30
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 3
bmCapabilities 0x03
Still image supported
Fixed frame-rate
wWidth 1024
wHeight 296
dwMinBitRate 12124160
dwMaxBitRate 12124160
dwMaxVideoFrameBufferSize 606208
dwDefaultFrameInterval 500000
bFrameIntervalType 1
dwFrameInterval( 0) 500000
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 1
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Couldn't open device, some information will be missing Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.00 bDeviceClass 9 Hub bDeviceSubClass 0 Unused bDeviceProtocol 1 Single TT bMaxPacketSize0 64 idVendor 0x1d6b Linux Foundation idProduct 0x0002 2.0 root hub bcdDevice 4.02 iManufacturer 3 iProduct 2 iSerial 1 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 25 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0xe0 Self Powered Remote Wakeup MaxPower 0mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 9 Hub bInterfaceSubClass 0 Unused bInterfaceProtocol 0 Full speed (or root) hub iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0004 1x 4 bytes bInterval 12 osboxes@osboxes:~$
if i use command sudo ./SmartGaze i have: init done opengl support available UVC initialized Device found Device opened DEVICE CONFIGURATION (29ab:00fb/0000000001) --- Status: idle VideoControl: bcdUVC: 0x0100 VideoStreaming(1): bEndpointAddress: 131 Formats: UncompressedFormat(1) bits per pixel: 16 GUID: 5955593200001000800000aa00389b71 (YUY2) default frame: 1 aspect ratio: 8x6 interlace flags: 00 copy protect: 00 FrameDescriptor(1) capabilities: 03 size: 2304x1536 bit rate: 35389440-35389440 max frame size: 7077888 default interval: 1/5 interval[0]: 1/5 FrameDescriptor(2) capabilities: 03 size: 1344x380 bit rate: 20428800-20428800 max frame size: 1021440 default interval: 1/20 interval[0]: 1/20 FrameDescriptor(3) capabilities: 03 size: 1024x296 bit rate: 12124160-12124160 max frame size: 606208 default interval: 1/20 interval[0]: 1/20 END DEVICE CONFIGURATION bmHint: 0000 bFormatIndex: 0 bFrameIndex: 0 dwFrameInterval: 0 wKeyFrameRate: 12027 wPFrameRate: 46872 wCompQuality: 46108 wCompWindowSize: 46896 wDelay: 10128 dwMaxVideoFrameSize: 3071814960 dwMaxPayloadTransferSize: 134520283 bInterfaceNumber: 252 get_mode: Invalid mode (-51) Device closed UVC exited osboxes@osboxes:~/SmartGaze/build/
but i don't have images raw, why? it is important to have opened eyetribe server? in this case how install it on linux?
First of all, what is it that you want to use SmartGaze for? I might be able to offer some better advice if I knew what you want to do. I'm also just interested.
A couple things to try for the UVC errors:
src/main.cpp
to what I pasted below, this is a mode that the info dump says it supports, at a really low FPS though, possibly because it doesn't have USB 3 bandwidth. It might get it working a bit though.static const int kCaptureWidth = 2304;
static const int kCaptureHeight = 1536;
static const int kCaptureFPS = 5;
Things to try for compilation issues:
make install
for Halidesrc/CmakeLists.txt
to something likefind_path(LIBHALIDE_INCLUDE_DIR Halide.h HINTS /home/myusername/wherever/Halide/include)
find_library(LIBHALIDE_LIBRARY Halide HINTS /home/myusername/wherever/Halide/lib)
eyeTribe has problem with sunglasses and i would like to solve this problem with file raw. if i write sudo ./SmartGaze the simplest project runs and prints "streaming..." but the windows is black and it is impossible to close else i kill with ctrl+c. main program contains declaration of keys "e, G , q,p etc" but if i have pointer on black window and press some button the situation is the same. how can i view the image in realtime?
with find_path(LIBHALIDE_INCLUDE_DIR Halide.h HINTS /home/myusername/wherever/Halide/include) find_library(LIBHALIDE_LIBRARY Halide HINTS /home/myusername/wherever/Halide/lib) cmake and make are ok but when i run program: ./SmartGaze:error while loading shared libraries: libhalide.so: cannot open file: no such file or directory
Okay so when you manage to get the streaming part working with the black window, did the red lights on the Eye Tribe turn on? If not the black window could have been due to a lack of IR light, try pointing the camera outside and see if the window shows anything. I would also try and get USB 3 working so you might be able to get a better feed.
For the shared library issue, that's because it can't find it in the non-standard location at runtime. There's a hack around that (I'd have to look it up) but for now you should try just putting the libs from your Halide/lib
folder in /usr/local/lib
or some other standard location and the include files in /usr/local/include
.
But really neither of these things are going to solve your underlying problem of wanting the Eye Tribe to work with sunglasses. Glasses often cause major issues for eye trackers because of spurious glints blocking the eye image and confusing the tracker. Your sunglasses might be even worse if they block IR light.
And even if you do get an image that shows what the problem is, that won't help you getting it to work unless you develop a full eye tracking system that avoids the problem (SmartGaze is far from complete) or get the Eye Tribe developers to agree to fix the problem, which they may not be able to do depending on how serious it is.
ok i have solved problem, but programs have black windows and in terminal i have QSettings::value: Empy key passed QSettings::value: Empy key passed
tell me the right parameters please
At this point I have no idea what the issue you are encountering is. Have you made sure you are using a version of VirtualBox that supports USB 3 (above 5.0)?
Also see my notes about SmartGaze likely not helping with your problem.
what are the right parameter for smartgaze? i use ./SmartGaze with zero parameters
There are no parameters.
Are you a programmer? If not SmartGaze won't be very useful to you, best it might do is give you an infrared camera.
At this point I'm pretty sure that even if you get SmartGaze working it likely won't solve your problem with the Eye Tribe.
i have solved problem with cable. now i have: osboxes@osboxes:~/SmartGaze/build/bin$ sudo -b ./SmartGaze osboxes@osboxes:~/SmartGaze/build/bin$ init done opengl support available UVC initialized Device found Device opened DEVICE CONFIGURATION (29ab:00fb/0000000001) --- Status: idle VideoControl: bcdUVC: 0x0100 VideoStreaming(1): bEndpointAddress: 131 Formats: UncompressedFormat(1) bits per pixel: 16 GUID: 5955593200001000800000aa00389b71 (YUY2) default frame: 1 aspect ratio: 16x9 interlace flags: 00 copy protect: 00 FrameDescriptor(1) capabilities: 03 size: 1152x1536 bit rate: 35389440-35389440 max frame size: 7077888 default interval: 1/27 interval[0]: 1/27 FrameDescriptor(2) capabilities: 03 size: 1036x1536 bit rate: 95477760-95477760 max frame size: 3182592 default interval: 1/30 interval[0]: 1/30 FrameDescriptor(3) capabilities: 03 size: 768x1024 bit rate: 94371840-94371840 max frame size: 1572864 default interval: 1/60 interval[0]: 1/60 FrameDescriptor(4) capabilities: 03 size: 768x1536 bit rate: 99090432-99090432 max frame size: 2359296 default interval: 1/42 interval[0]: 1/42 FrameDescriptor(5) capabilities: 03 size: 2304x1536 bit rate: 191102976-191102976 max frame size: 7077888 default interval: 1/42 interval[0]: 1/27 FrameDescriptor(6) capabilities: 03 size: 2072x1536 bit rate: 190955520-190955520 max frame size: 6365184 default interval: 1/27 interval[0]: 1/30 FrameDescriptor(7) capabilities: 03 size: 1536x1024 bit rate: 188743680-188743680 max frame size: 3145728 default interval: 1/30 interval[0]: 1/60 FrameDescriptor(8) capabilities: 03 size: 1536x1536 bit rate: 198180864-198180864 max frame size: 4718592 default interval: 1/60 interval[0]: 1/42 END DEVICE CONFIGURATION bmHint: 0000 bFormatIndex: 1 bFrameIndex: 7 dwFrameInterval: 166666 wKeyFrameRate: 0 wPFrameRate: 0 wCompQuality: 0 wCompWindowSize: 0 wDelay: 0 dwMaxVideoFrameSize: 10077696 dwMaxPayloadTransferSize: 49152 bInterfaceNumber: 1 Streaming...
but the window in black for one second and then closed. IR leds are on now
when i run make i have:
petrucci@christian-petrucci:~/SmartGaze-master/build$ make Scanning dependencies of target SmartGaze [ 16%] Building CXX object src/CMakeFiles/SmartGaze.dir/main.cpp.o [ 33%] Building CXX object src/CMakeFiles/SmartGaze.dir/eyetracking.cpp.o In file included from /usr/include/c++/4.8/chrono:35:0, from /home/petrucci/SmartGaze-master/src/eyetracking.cpp:10: /usr/include/c++/4.8/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
^
/home/petrucci/SmartGaze-master/src/eyetracking.cpp: In function ‘std::vectorcv::Point
help me please, it is very important
Please read my previous comments, I don't think it is important because SmartGaze will probably not help you as much as you think it will. See my previous comment:
SmartGaze will not solve your underlying problem of wanting the Eye Tribe to work with sunglasses. Glasses often cause major issues for eye trackers because of spurious glints blocking the eye image and confusing the tracker. Your sunglasses might be even worse if they block IR light.
And even if you do get an image that shows what the problem is, that won't help you getting it to work unless you develop a full eye tracking system that avoids the problem (SmartGaze is far from complete) or get the Eye Tribe developers to agree to fix the problem, which they may not be able to do depending on how serious it is.
All SmartGaze does is show the raw feed, if you are not a really good programmer who wants to spend a month writing an eye tracking implementation, then this won't solve your problem.
Do you think I'm wrong about this? If so, why?
we'll use eyetribe for try to find problem of algorithms such as active shape model and starburst. can you help me to compile your project with halide? your code has starburst and it is good for me. why have I the previous problem after make?
I'm not sure why you're having that problem. It looks like it is missing c++11 command line flags, but I specify in the CMake file that it should use C++11. It could be that you are using a too old version of CMake and you either have to update it or find some other way to specify to use those flags.
i have solved the previous problem. now when i lunch your program i have
OpenCV Error: Assertion failed (src.type() == CV_8UC1) in adaptiveThreshold, file /home/petrucci/opencv-3.1.0/modules/imgproc/src/thresh.cpp, line 1286 terminate called after throwing an instance of 'cv::Exception' what(): /home/petrucci/opencv-3.1.0/modules/imgproc/src/thresh.cpp:1286: error: (-215) src.type() == CV_8UC1 in function adaptiveThreshold
If I print image's type oh eyetribe I have 8UC3 value and not 8UC1.
Did you change any code? The number of channels passed to adaptiveThreshold should be the same as passed in to trackGlints which should be the same as passed into trackFrame, which I allocate as one channel.
This looks like the kind of simple issue that if you can't debug yourself SmartGaze won't be of much use to you.
SmartGaze is totally unfinished, I might work more on it later, but for now you have to know what you are doing in order to use it.
i have a project smartgaze at https://github.com/trishume/SmartGaze and i have installed all libs. i have used this command for install halide cd llvm3.7 mkdir build cd build cmake -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD="X86;ARM;NVPTX;AArch64;Mips;PowerPC" -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release .. make
in halide dir: export LLVM_CONFIG=../llvm3.7/build/bin/llvm-config export CLANG=../llvm3.7//build/bin/clang
mkdir cmake_build cd cmake_build export LLVM_ROOT=/path/to/llvm3.7/build cmake -DLLVM_BIN=${LLVM_ROOT}/bin -DLLVM_INCLUDE="${LLVM_ROOT}/../include;${LLVM_ROOT}/include" -DLLVM_LIB=${LLVM_ROOT}/lib -DLLVM_VERSION=37 .. make
and for build smartgaze project in it's folder: mkdir build cd build cmake ../
but i have cmake error: the following variable are used in this project, but they are set to NOTFOUND LIBHALIDE_INCLUDE_DIR used as include directory in directory /home/osboxes/SmartGaze-master/src LIBHALIDE_LiBRARY linked by target "SmartGaze" in directory /home/osboxes/SmartGaze-master/src
the build of halide is ok and it is in separate folder of llvm3.7. i have osboxes/Halide, osboxes/llvm3.7 osboxes/SmartGaze-master, is it wrong? help me please
i have builded the simplest version of smartgaze on virtualBox kubuntu but the the program print uvc_open: access denied why?