problem in streaming

[varun333]

hamuchiwa I have install the python picam library on my rasp pi and that module is working on my VNC viewer but I dont know whts causing the errror while I am running stream_client_test.py on vnc and same with the server code.

Is there require any kind of configuration in pycharm or something else is causing the error. Following are the errors

While running stream_client_test.py using VNC viewer

Traceback (most recent call last): File "/home/pi/Desktop/raspberryPi/stream_client.py", line 19, in client_socket.connect(('192.168.2.11', 8000)) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 111] Connection refused

While running server test .py on laptop

File "C:/Users/USER/PycharmProjects/AutoRCCar-master/test/stream_server_test.py", line 44, in VideoStreamingTest() File "C:/Users/USER/PycharmProjects/AutoRCCar-master/test/stream_server_test.py", line 12, in init self.server_socket.bind(('192.168.2.11', 8000)) File "C:\Python27\lib\socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 10049] The requested address is not valid in its context

Process finished with exit Please reply soon

[hamuchiwa]

@varun333 Did you run server first? Have you tried changed port?

[varun333]

hello hamuchiwa

thanks for the help but here are some problem which I am currently facing while running these

1)while using rapistill to capture image the preview display opens up but mine is not opening and I had enabled it. Generally in place of image some box opens for the time I am running the code.

2)Now second problem here as you have mentioned Id run the rc control test.py then camera callibration then collect training .py and then stream server .py and then stream client.py , the error mentioned above got resolved.

2)But due to my camera is not showing the preview of the image it throws large sum of errors as of which earlier I was not able to capture any kind of image or video. Any kind of help will be great help.

3) As i had said RC control test.py was working properly but it is again throwing error

File "C:/Users/USER/PycharmProjects/AutoRCCar-master/test/rc_control_test.py", line 73 if name == 'main': ^ IndentationError: unindent does not match any outer indentation level

As i have no idea about working on pycharm so help me here. I had adapted this as my project and now I am stuck, help me out.

[hamuchiwa]

@varun333 Are you trying to display video stream on raspberry pi or pc? It says IndentationError, remove extra spaces before "if name == ''main'" if there are any.

[varun333]

For sure on PC.

Just wanna ask, at first when I run this code RC control test.py was working properly but after running collection_training.py, the car which I was running through keyboard has opted the self steer function.

Is this what causing the problem, when I am again running the RC control test.py and it just showing Traceback (most recent call last): File "C:/Users/USER/PycharmProjects/AutoRCCar/test/rc_control_test.py", line 73, in RCTest() File "C:/Users/USER/PycharmProjects/AutoRCCar/test/rc_control_test.py", line 12, in init self.steer() AttributeError: 'RCTest' object has no attribute 'steer'

So now what If I am initiating my project so I again i have to run this control test.py or it was for just one time as I have run the collection training .py code.

[hamuchiwa]

All test.py files are just for testing purpose. When collecting images, no need to run these test files. Same thing here, start server first, then client.

[varun333]

and these are my camera matrix [[ 2.40406352e+03 0.00000000e+00 1.06956635e+03] [ 0.00000000e+00 2.45713525e+03 8.87405747e+02] [ 0.00000000e+00 0.00000000e+00 1.00000000e+00]] Ay: 2457.13524515 u0: 1069.56635117 v0: 887.405746804

another one included few more images now tell me which matrix should i go with...

[ 2.50404673e+03 0.00000000e+00 1.22267514e+03] [ 0.00000000e+00 2.53166332e+03 8.37163347e+02] [ 0.00000000e+00 0.00000000e+00 1.00000000e+00]] Ay: 2531.66331764 u0: 1222.67513502 v0: 837.163347007

[hamuchiwa]

@varun333 Video streaming is handled by OpenCV. Images are captured by pi camera. If you are using the exact same pi camera module, then calibration is not needed. As OpenCV doc says, it needs "at least 10 images". Go with the second one.

[varun333]

I am streaming through vlc, will that work fine for streaming and any idea about the latency rate.

[varun333]

I have streamed the video using raspivid command that over raspi and I got it streaming over my pc but I coldnt get it by your command

first i run my vlc command on pi camera and started playing the video on vlc player then i run the code of server on pc and of client on pi, the server is showing me the streaming while I am running the client code but after it throws out the error

connection refused

does it has anything to do with pi camera resolution as you have written to the one that of raspivid command that I have passed for vlc streaming

any idea related to pi camera that I should go with seeking out my camera matrix and the frame rate please help me out with streaming step including changes to vlc as well how to run client command

[hamuchiwa]

I didn't use vlc before, but as long as you can turn streaming image into an array, the goal is accomplished. Resolution is set to low to achieve "smooth" video streaming.

[hamuchiwa]

How did you save image array through vlc?

[hamuchiwa]

I think you misunderstood on how to start server and client. First you start the server_test.py on your pc, second run stream_client.py on raspberry pi. There are no vlc and raspivid involved.

[varun333]

First I run the server.py on pc , as I hit enter on server nothing appear but as i run client on raspi error on server side Connection from: ('192.168.2.12', 36988) Streaming... Press 'q' to exit

Traceback (most recent call last): File "C:\Python27\stream_server_test.py", line 41, in VideoStreamingTest() File "C:\Python27\stream_server_test.py", line 13, in init self.streaming() File "C:\Python27\stream_server_test.py", line 31, in streaming image = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_UNCHANGED) NameError: global name 'np' is not defined

and on client site

File "/home/pi/Desktop/raspberryPi/stream_client.py", line 24, in connection.flush() File "/usr/lib/python3.4/socket.py", line 391, in write return self._sock.send(b) ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/pi/Desktop/raspberryPi/stream_client.py", line 33, in connection.close() File "/usr/lib/python3.4/socket.py", line 391, in write return self._sock.send(b) BrokenPipeError: [Errno 32] Broken pipe

[hamuchiwa]

You got an error: "NameError: global name 'np' is not defined" Check Numpy import

[varun333]

@hamuchiwa Still not running,as i am trying to run your codes individually server side : File "C:/Users/USER/PycharmProjects/AutoRCCar/test/stream_server_test.py", line 44, in VideoStreamingTest() File "C:/Users/USER/PycharmProjects/AutoRCCar/test/stream_server_test.py", line 16, in init self.streaming() File "C:/Users/USER/PycharmProjects/AutoRCCar/test/stream_server_test.py", line 34, in streaming image = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_UNCHANGED) AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_UNCHANGED'

client side: File "/home/pi/Desktop/raspberryPi/stream_client.py", line 26, in connection.write(stream.read()) File "/usr/lib/python3.4/socket.py", line 391, in write return self._sock.send(b) ConnectionResetError: [Errno 104] Connection reset by peer

[varun333]

is it necessary to run collection.training.py before the streaming of video, are these both code interrelated and somehow causing these error. And I had opencv install on pc then I dont think there is requirement of it to install on raspi or is there.

And would be nice of you to tell me about the the packages that i need to install on my raspi 3 to run these code of yours.

[varun333]

hamuchiwa thanx for the help, just now my both data are streaming fine but just wanna know in ultrasonic the output comes for 15 to 20 and then it says broken pipe error(here my sensor is in free state)

And now i am heading for next module i.e. mlp training data.py so just guide me early with the module and requirements that I shoud seek to make it work in one go.

Hamuchiwa please reply soon now everything is dependent on you. Just help me to get over this module can I run this mlp individually just for testing

[varun333]

@hamuchiwa I just wanna ask earlier my stream_client was not working but i run the collection training.py and not make any key movement, so now my stream client is working and again I want to start from the begining so how I am gonna do it as my key function or movement were not collected that time and while I perform this it should be on track or random moves count.

[hamuchiwa]

@varun333 Does it throw any errors when using keyboard to control the rc car with the test file?

[varun333]

earlier kind of def self() related error but now when I am pressing my keyboard key the command are getting printed but my car is not moving.

[varun333]

well when i will be running collect training data.py then what kind of output I am gonna be expecting and for how much duration I have to run this module on specified track and after that in mlp training , if you will be able to tell me output ,would be a great help.

[hamuchiwa]

Check this thread @ghost commented on this issue, hope it helps. Well, when running collect_training_data.py, every time you press the key, the rc car moves, one frame is saved, when you exit, all images will be stacked together and saved into a .npz file which you will need for neural network training.

[varun333]

@hamuchiwa to run collecttraining data .py did I have to run stream server.py or not and if yes the address will be same for both, as i just run only collect training data.py and on pi stream_client.py and pygame window open for sort duration and same with cam and yes did I have to change the path in collect training.py for training data wrt of yours

please tell me the procedure fast as I have to submit this very soon thanks

error self.collect_image() File "C:/Users/USER/PycharmProjects/AutoRCCar/computer/collect_training_data.py", line 61, in collect_image temp_array = roi.reshape(1, 38400).astype(np.float32) ValueError: total size of new array must be unchanged

image_array = np.zeros((1, 38400))# i m using so what would be my roi label_array = np.zeros((1, 4), 'float')#and also need changes here or what

[varun333]

@hamuchiwa i have fixed the error as i have increased the camera resolution but how i can come to know about the roi for this 8mp camera and now before collect training.py i have to run stream server or not thats what i wanna ask.

[hamuchiwa]

collect training data.py contains server side code already. Try tilting down the camera angle a bit to take full frame at first (so frame is filled with track only), in this case the initial image array will be np.zeros((1, 76800)) if your resolution is 320x240.

[varun333]

but in your case its 38400, why so?

roi = image[120:240, :] this is for 320 *240,

And what if i set resolution to 640*480 then in that case what will be my initial image array and roi as we are taking lower half image then width gonna be same 480 but what about height how are we gonna decide

And one more thing to save testing data i have to create folder of .npz format or the data generated will be automatically of type .npz

or i just simply run your code

[varun333]

this what i have received as output

Streaming duration: 28.5937477815 (25L, 38400L) (25L, 4L) Total frame: 605 Saved frame: 25 Dropped frame 580

is this correct? Using mlp.py

Loading training data... ['train_labels', 'train'] (25L, 38400L) (25L, 4L) (25L, 38400L) (25L, 4L) Loading image duration: 0.192458062094 Training MLP ... Training duration: 16.4053378938 Ran for 92 iterations Prediction: [1 1 1 1 1 1 1 2 2 0 3 3 3 2 0 2 0 2 0 2 2 0 0 0 3] True labels: [2 2 2 2 2 1 2 2 2 0 0 0 0 2 0 2 0 2 0 2 2 0 0 0 3] Testing... Train rate: 64.000000:

Process finished with exit code 0

[varun333]

@hamuchiwa i am just training and saving in different npz file, so in order to increase train rate what i need to do as you were saying merging the npy file how are we gonna do it or it just keep saving the data in same npz file format.

[hamuchiwa]

Each time you collect, a npz file will be saved. The training can load multiple npz-s at one time as long as they are in the same folder.

[varun333]

@hamuchiwa It like just , I first specified the track and collect the data as test01 and if i want to define the new track and train it then I am gonna overwrite the test01 or i have to save it as test02 npz file in the training_data, and also tell me about the train rate i.e 64, is it workable, how i am gonna increase the train rate.

[hamuchiwa]

You can use one or multiple npz files. You can tell whether it's good or not once you let the car drive itself, 80% accuracy certainly works better than 64%. You could do a straight line track first, and see what happens.

[varun333]

@hamuchiwa yeah i got the accuracy of 80% in the first time of training but as after running mlp training.py ,i ran mlp_predict.test.py and receive the error why so? or I have to run rc_driver.py just to accumulate the testing data.....

File "C:/Users/USER/PycharmProjects/AutoRCCar/computer/mlp_predict_test.py", line 36, in ret, resp = model.predict(test) cv2.error: ........\opencv\modules\ml\src\ann_mlp.cpp:279: error: (-5) Both input and output must be floating-point matrices of the same type and have the same number of rows in func

[varun333]

@hamuchiwa Its urgent plz help.

[hamuchiwa]

I remember you had the same error while you were doing the training, it's caused by OpenCV version difference. If you are always using the same track, you don't even need map_predict_test.py.

[varun333]

@hamuchiwa how do you have obtained the testing data? the test cases of trained data can be used as testing data

[varun333]

As after cut copy paste of a npz file from training data to testing data, I have got this result is it okay or there is other way to get the testing data ['train_labels', 'train'] (53L, 38400L) (53L, 4L) (53L, 38400L) (53L, 4L) Prediction time per frame: 0.00167296483453 Prediction: [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1] True labels: [2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 1 2 1 2 1 1 2 1 2 3 2 2 2 1 1 1 1 2 1 2 2 0 2 2 2 2 2 2 2 1 2 2 2 2 2 2] Testing... 12 Test rate: 22.641509

Process finished with exit code

[hamuchiwa]

You can either have separate files for training and testing, or random split (for example, 7:3) a data set into training and testing. What's your train rate?

[varun333]

@hamuchiwa Train rate: 55.752212:

[varun333]

@hamuchiwa i just only wanna know ,we get the training data from collect training data.py in npz format but how we get the testing data (because when i copy the npz file from training data to testing data file ) then only i was able to get the above prediction as an output.

Is this how its need to be done

[hamuchiwa]

Okay, try this if you haven't done so.

1. collect data using collect_training.py
2. randomly split collected data into training and testing sets (ratio is 7:3 or 8:2)
3. copy each files to its corresponding folder Your training set accuracy is low, make sure the data you collected is good quality.

[varun333]

@hamuchiwa 1) this is just for simple straight track where i first got train rate of 55% and then i again trained over the same track got 69 % train rate and 80% as an prediction

2) after my car moves straight to half track and then always get deviated to right

3) stop sign is getting classified but there is no message as such stop sign ahead & when I place object infront of car it get stopped but after few seconds

[hamuchiwa]

A simple straight line should give you a accuracy at least 90%. Try to tilt down the camera so it only sees the road, and use entire image as train data. It usually gives you a better result than using half of the image.

[varun333]

@hamuchiwa to use entire image as train data what parameter do i need to change? Suppose if i get the accuracy of 82% in first time of collected data then in that case i need to train it again over the same track or is that just sufficient.

[hamuchiwa]

Change array size to reflect resolution. You could just train once and using the same track over and over again without doing any testing, or as I mentioned earlier, you could also do cross validation. I would suggest you to get familiar with the flow of machine learning first.

[varun333]

@hamuchiwa as to work with complete resolution i just double the figure of 38400 to 76800 in image array for 320*240 and for temp array also done the same temp_array = roi.reshape(1, 76800).astype(np.float32) but for roi it will be same or it would be somewhat like this roi = image[320:240, :]

help me with this, then next i am thinking is to make different track and train them and perform cross validation as you said

[hamuchiwa]

Since we are not slicing the image, "roi" is no longer needed, or you could set "roi = image".

[varun333]

@hamuchiwa the stop sign is getting rectangulated but in output its not mentioning stop sign ahead. How do i classify this stop sign with distance beacause i have no idea how to do it please forward me the code for this module, it will be a great help or its just that i have to perform some calculation

Please forward me the code for this.

[hamuchiwa]

The code is in project already. It measures the distance between the camera and stop sign, if the distance is lower the threshold, then the message is displayed.

[varun333]

@hamuchiwa Can you mail me the journal related to object detection Chu, Ji, Guo, Li and Wang (2004).

[varun333]

in rc driver.py what will be the self.alpha as after camera calibration i have different value and u have defined 8*math.pi/180, how did u get this.