search

ev3dev / ev3dev-lang-python

Pure python bindings for ev3dev
MIT License
425 stars 144 forks source link

Using Webcam trought USB or Using Pixy Cam from Port #690

Closed MrK-CH closed 4 years ago

MrK-CH commented 4 years ago

Hello every body

I hope anybody can help me. I'm in a School Project for my bachelor's degree. There I have to use a Lego Mindstorm Robot with Brickman. I'm using python as language in the Spyder enviroment.

  1. Now i have to use a Webcam with the robot to drive autonomous. So weth pluged in a cheep webcam from aliexpress in the USB port (port enlargement with 3 Slots pluged in, one for Wifi Dongle other for Webcam one free). But I have no idea how to address the usb port to use the camera. Is that possible with Python at all? I have found Nothing along the internet.

  2. Now i have ordered a pixycam, because it is written on the ev3 page, that the pixy cam is compatible like a sensor. But I can't figure out how to use the pixycam like a normal webcam. I Need to send live pics to my program. Does anybody know which mode i have to use or how this is possible?

I thank you alot for your help and excuse me for my bad english writting XD.

dlech commented 4 years ago

The best way I have found to get images from the PixyCam is to connect it via USB. There is a pixy python module pre-installed in ev3dev, so start with import pixy.

Then see http://cmucam.org/boards/9/topics/5106

It has some examples in C, but the python library is just a wrapper around the C library, so should be quite similar.

MrK-CH commented 4 years ago

Thanks a lot dlech!

Monday the Pixycam should arrive. I will try it out an give you Feedback!

MrK-CH commented 4 years ago

HI dlech

I've found this Video of you on YouTube: https://www.youtube.com/watch?v=b3qE2un9-HQ very funny, but almost this what i need.

I have no idea how to translate the C code from your link into python code. It is more than a year ago that I've written something in C. I'm not a very expirienced programer in C and in Python. Do you have an example code in Python to get a frame from pixy or webcam trought a SSH conection from my robot to my computer?

Is there a documantation about the liberary from the pixy cam, using it trought the USB hub from the ev3?

Foto_EV3_with_Pixy

Perhaps a more detailed description of my project can help.

For my project I need a live image of the robot as it moves through a racetrack to control and accelerate it via SSH with image processing from my PC. The robot must be able to be operated without cables.

dwalton76 commented 4 years ago

You could use OpenCV library in python to process the video feed from your webcam. Example of this here https://github.com/dwalton76/rubiks-cube-tracker/blob/master/rubikscubetracker/__init__.py#L1958

MrK-CH commented 4 years ago

Thanks for helping but we finished the Project yet.

KooshaKayani commented 4 years ago

Hey I am trying to do the same thing but to get video input from pixy to ev3 block but I couldn't find anything. I would be happy if you can help me with that since you have already done it.

MrK-CH commented 4 years ago

Hi koosha18

Unfortunately you can't use the Pixy Cam as a normal camera and run the stream through the EV3. You can only connect it via the sensor inputs of the controller module. The Pixy Cam then needs to be calibrated with the appropriate software. Then you can detect colored objects. The Pixy Cam will only transmit the x and y coordinates of the object to the EV3. This was confirmed to me in a request to Pixy itself. So it is not possible to use the camera stream. In any case I did not make it. We continued to work with an IP camera app and putting the Smartphone on the Robot and thus separated the camera system from the robot hardware and then merged it on the software on the computer. I hope this helped you and forgive my bad English.

Greetings

KooshaKayani commented 4 years ago

Thanks for Your help and your English is really good.

so you used a phone and then connected it to like pc? and then sent the result to your robot?

MrK-CH commented 4 years ago

IMG_8264

MrK-CH commented 4 years ago

Yes we have build a Phone Mount with Lego. Then you can use an IP-Cam App to transfer the Video stream by IP-address to your PC. Then you have your video stream with an IP-adr. which you can bind into your running Python Code with witch you are controlling the robot. There are many examples to bind the IP-adr. to your Python Code. I hope that helps good luck XD!

KooshaKayani commented 4 years ago

yes that is a huge help thank you for your help. :)

Hzhhanna commented 3 years ago

Hi, I can help you with the information of the Pixy streaming, I'm making a project for a competition and made a stream from the Pixy2 by IP to an Android phone. It's very complicated, but not imposible. First I took the pixy camera and connected it to a pc then open the Pixymon app and used obs to streamed to youtube then used the youtube IP and used on my own android app.

Tarcisius-Mak commented 3 years ago

Hellw MrK-CH, I have to do the project for autonomous car and I need to use a camera. But I'm un aware of which camera to be used, I find it online and I saw Pixy camera that are too costfull to afford. May you give me an idea how the phone camera works?

MrK-CH commented 3 years ago

Hi Tracisius-Mak

You can find the solution in the upper side of this issue:

Yes we have build a Phone Mount with Lego. Then you can use an IP-Cam App to transfer the Video stream by IP-address to your PC. Then you have your video stream with an IP-adr. which you can bind into your running Python Code with witch you are controlling the robot. There are many examples to bind the IP-adr. to your Python Code. I hope that helps good luck XD!

hopply it is halpeful for you.

good luck and have fun XD

Tarcisius-Mak commented 3 years ago

Thanks I got your idea, but I have misunderstanding coz the code that control the robot (car) are imported to arduino and the moment a car is moving does not depend on code that are in PC. How do I link it ? Or the process is to be done online?

MrK-CH commented 3 years ago

aaah I got you

Doing this with an Arduino is difficult. Depending on the basic version, it has too little computing power. I would suggest you get a wifi shield for the Arduino and send only the control commands from the computer. I would then do everything else as I have told you. Image processing on one of the common microcontrollers is not possible in my opinion.

Tarcisius-Mak commented 3 years ago

Is there any option for recognizing traffic light apart from using camera as input?

On Sat, Jul 24, 2021, 18:18 Marco K. @.***> wrote:

aaah I got you

Doing this with an Arduino is difficult. Depending on the basic version, it has too little computing power. I would suggest you get a wifi shield for the Arduino and send only the control commands from the computer. I would then do everything else as I have told you. Image processing on one of the common microcontrollers is not possible in my opinion.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ev3dev/ev3dev-lang-python/issues/690#issuecomment-886067324, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU6GCFSV2R5T42PBHGKCUA3TZLKURANCNFSM4JLIHWSQ .

MrK-CH commented 3 years ago

Yes, u can use sveral light intension sensors!

Tarcisius-Mak commented 3 years ago

May you show examples to me? Even images if possible.

On Sun, Jul 25, 2021, 21:55 Marco K. @.***> wrote:

Yes, u can use sveral light intension sensors!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ev3dev/ev3dev-lang-python/issues/690#issuecomment-886243174, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU6GCFVE37JI5I4MQ7IXW7LTZRMY3ANCNFSM4JLIHWSQ .

MrK-CH commented 3 years ago

sorry i haven't any this work you have to do by your own.

KooshaKayani commented 2 years ago

Hey Guys I was going through my history and I came across this issue which I had last year.

Anyways I found out how to use pixy with python. for some examples you can check out my pashmam repository, and look at the part that is for Pixy Cam. but here is a summery.

https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:porting_guide#setlamp-upper-lower

change the configuration to 12c and import it in your main code

from pybricks.iodevices import I2CDevice

Initialize the pixycam.

pixycam = I2CDevice(Port.S4, 0x54) # you can see why these numbers are used in the documentation above.

and then you have to communicate with an array and to do so you have to check the documentation and see what is the right array for what you are trying to do and then read the result.

here is an example lampOn= [174, 193, 22, 2, 1, 0] pixycam.write(0, bytes(lampOn))

this will turn the lamps on

pixycam.read(0,20) this will read the first 20 blocks and you use the documentation to find the block of what you are looking for.

I know it's not exactly what you asked for but it could help someone in some way.