Better approach -- rpi zero w

[lbdroid]

Don't think I want to waste too much time on hacking the kernel to make the crappy chinese radio work with UVC devices any more. Its performance stank, and it also messed up the timestamps to such a serious degree that I had to modify ffmpeg to regenerate them. Putting a better computer in between the camera(s) and the crappy chinese car radio seems to fix all the problems. Funny that a single-core ARMv6 with just 512 MB of RAM is actually a "better computer", but in terms of I/O performance, it blows china out of the water. The fact that it is able to handle TWO (or more) cameras simultaneously without so much as a hiccup where china could barely handle ONE speaks volumes.

I think we can set up a raspberry pi camera server on a completely read-only sdcard. The idea of it being completely read-only is to make it completely resistant to uncontrolled powering off. Power this thing by USB from the car radio, so it goes on and boots up with the car's ignition. One CSI camera in the camera port, and one USB camera in the USB port. The hardware on a raspberry pi supports h264 encoding. A stripped kernel and userspace should allow this to boot up and run very quickly. We can connect it with the car radio using wifi and stream h264 frames to ffmpeg running on the car radio.

I'm just hoping that a pi zero w will be strong enough to do this. Since there won't be any encoding done in the software, this may be possible.

[lbdroid]

Hmm, raspivid seems to have some networking capability. I'll have to investigate that. Unfortunately, however, this will only be applicable for the CSI camera, not the USB camera.

Or I think that rpi camera is exposed as v4l2 now, and that ffmpeg has access to OMX video encoder, so that may be another option. Wonder how efficient it would be to feed the raw video through ffmpeg though?

[lbdroid]

There is some talk about the v4l2 driver here; https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=131581

Suggests that there may be a connection to the h264 hardware encoder right in the v4l2 driver. That might be a sufficiently efficient process that I could work with. Feed two h264 streams over two ports to the car radio. Hopefully, the car radio doesn't end up being too much of a bottleneck... but there is a possibility that this could actually work a LOT better than trying to capture from USB cameras directly on the car radio.

[lbdroid]

I'm really feeling like Wile E. Coyote right now with this ACME/Amazon instant delivery thing... Ordered a couple of Pi Zero W's today, and 10 minutes after I get home from work, knock on the door and they were there. Exceptional.

I have a couple of foot-long camera cables on order as well, but they're not in yet (different vendor). Fortunately, the Pi Z-W's came with a short (about 3") camera cable. The pi zero has a more compact camera connector, so you can't use the same camera cable as a regular pi.

I also had already a couple of sainsmart wide angle pi cameras. They cost $31 CDN each. So all in, I'm under $50 for each Pi Z-W + camera.

A quick little rig up with one of them and its working with a v4l2 camera interface, and yes, its exposing H264 as an option. I'll determine later how they perform. I'm hoping that I can do one CSI and one USB camera on a single Pi-Z. Since the UVC camera is sending pre-compressed frames, this hopefully won't overwhelm the single core 1.0 GHz ARMv6. Guess we will find out, though I suppose if it is really necessary, could use a Pi2 or Pi3 for this.

[lbdroid]

Well I've got two cameras plugged into a single pi-z-w, and they're both recognized, as three v4l2 devices. The usb camera is a bit of a wack job, it appears as two devices, one of which provides YUV and MJPEG, the other provides H264. Tomorrow I will do some tests to see how it all performs.

[lbdroid]

Boot time on this is about 15 seconds, which is perfectly reasonable. I might be able to get that shortened down later on, but I don't see any problem with it.

[lbdroid]

This is going to work beautifully. First off, I had no success using the raspbian/debian provided build of avconv. That seems to be built without support for H.264, even as a direct stream copy. I was successful using the "official" static armel build of ffmpeg found here; https://www.johnvansickle.com/ffmpeg/

I have two cameras connected right now, both are capturing H.264 1280x720 at 30 fps onto the rpi's sdcard. I currently have it running as two separate instances of ffmpeg, though in theory, I could arrange a single container having multiple video tracks. If I were to go that route, I'd probably want one of the tracks to be an mjpeg track, since it is unlikely for the I-frames of two H.264 streams to align for clean segmentation.

CPU idle is averaging about 75%, fluctuating between 70 and 80%. Keep in mind that this is IDLE, so total load is averaging about 25%, which includes waiting for I/O.

And this is on a raspberry pi zero w, which is a practically brainless little board with a 1.0 GHz ARMv6. I mean the thing is not even an ARMv7 with hard float. Its an ancient ARMv6.

Now I am observing one bottleneck right now. After the parallel capture, I am now in the process of retrieving the files over sftp/wifi. I'm getting a whopping 600 KB/s. If that's the limit of the wifi on this thing, then that really sucks. I'll have to investigate the reason why this is so slow. Its possible that this is just a bad wifi environment. EDIT: bottleneck appears to be local issue. Nothing changed, but now its 1.4-1.5 MB/s, at that, its using 60% CPU to transfer it over SSH, which is quite high and quite unnecessary (because SSH encrypts the data). I think the other part of the issue is this crap laptop I'm using.

Both videos came through beautifully. No frames dropped, clear, smooth. Everything you want.

[lbdroid]

In fact, these videos are both a lot nicer than the ones captured through the chinese radio. It pretty pathetic how poorly that radio performs. If the RPI Z-W can do this, then the chinese radio should be able to handle a whole WHACK of them without breaking a sweat. After all, that chinese radio has a 4-core x86 with AMD64.

I'm going to change the dashcam program into an interface to the RPI in order to display status and handle video saving and filesystem maintenance.

[HDaddict]

What will be the price of such a setup? If the custom kernel won't work and this setup will be expensive, I think I'll simply buy the joying dashcam... I'm still on November 2016 rom

[lbdroid]

RPI Z-W is a $10 board. If you're going to use a camera you already have, you can just plug it into the RPI's USB port directly and call it good. Well, the rpi zero's have a micro-USB, so you'll need an OTG-to-Asocket wire, which you probably already have, or wire up a custom camera wire, pretty easy since they're only 4 conductor.

Now here is a neat thing about the rpi zero w solution if you happen to have a "lower end" (read as: no h264) camera; you can build ffmpeg to interface with the rpi zero's h264 encoder, which means that you gain the option of storing the video with a much higher compression ratio.

[HDaddict]

Great! Do you have a link to such a board?

[lbdroid]

I wouldn't call it "such a board". Its a specific product. See here; https://www.raspberrypi.org/products/pi-zero-w/

On the right side of that page, is a list of distributors. It is NOT exhaustive, you can check also your local amazon or ebay, though those vendors tend to add in a pretty substantial markup.

Note that the zero W is in a semi-short supply. Its starting to become easier to find, but its such a neat little board that a lot of people are buying them up.

Also note that I haven't written up any software to work with this just yet! So don't go buying one today expecting everything to work exactly the way you want it to immediately ;)

[lbdroid]

So a few ideas; 1) use bluetooth instead of wifi? Control channels? Data channels? 2) could we implement a GPS receiver on the rpi instead of using the inaccurate and unreliable GPS that is built in to china? This actually would be really sweet, especially using something like an adafruit ultimate GPS hat, which has a built in RTC. The adafruit gps also can be set to a faster refresh rate than the china GPS (10 Hz instead of just 1 Hz). 3) we could feed SWI through the rpi for greater flexibility -- I was trying to do something like this using an adafruit feather m0, but found that china choked on Bluetooth LE -- it would pair, but it wouldn't accept any data transfer. The rPi is a full bluetooth device, not just LE, so china shouldn't object to it. 4) Controlled power off? I was figuring on using the time between the ACC off and the actual china radio shutdown to send a halt to the rpi, but there are fancier things we can do here, like feed a constant 5V up to the rpi to power it, and using the USB power just as a signal. The USB power can feed (through a diode) the FET/relay, and one of the GPIO pins on the rPi, with another GPIO pin also feeding the FET/relay. This would allow the rPi to "detect" when the car radio turns off, and keep itself on (a) long enough to execute a controlled shutdown, and (b) indefinitely if you press some SWI input combination to trigger an emergency "don't stop recording" mode. This stuff could be built into the adafruit GPS hat, which has an array of pins for doing cool stuff like this.

[lbdroid]

So I was thinking a bit more about this last night. There are a LOT of really neat features that can be implemented via a connected rPi. The main consideration, however, is that they need to be MODULAR. That means, of course, that anyone should be able to pick and choose the features of it that they are more interested in having.

So for the first feature; 1) dashcam recording to bare rPi board using EITHER a CSI or a UVC camera, or both, or multiple UVC cameras through a HUB (rPi 1+ include built in USB hub and multiple sockets, rPi 0[W] have only single uUSB OTG port, but can have external hub attached such as https://www.amazon.ca/Phoneix-Micro-Host-Adapter-Cable/dp/B011R3H11E , https://www.amazon.ca/ELEGIANT-Multifuntional-Charging-Adapter-Connector/dp/B00T00LP14 , or https://www.amazon.ca/MakerSpot-Stackable-Raspberry-Connector-Bluetooth/dp/B01IT1TLFQ) -- it should be possible to set up dashcam recording to begin automatically, in case where someone selects rPi that lacks wireless networking. -- rPi should be able to display recording status via LED. Probably best to hook an external LED to GPIO pin, since it is unlikely that anyone would want to stick the board somewhere that its built in LED could be seen. This feature can be implemented whether or not the user adds an LED. If they don't, they simply will not see it. -- rPi TIME? If the unit has no valid time source, file names should be sequentially generated. If the unit DOES have a time source, file names should include DATETIME. -- if there is a wireless network connection to the car radio, the car radio can be used as a time source. -- if the rPi has an RTC and/or GPS, the system time can be used as a time source. -- if the rPi has wireless network, there must be an application for car radio or phone to access and configure it. This application would also be able to provide time synchronization to the rPi. -- rPi dashcam recording should work whether or not there is an Android car radio.

2) RTC feature. A better time source for rPi than syncing up with car radio or phone, is to have its own time source. This allows recording to begin sooner, since upon powering on, it would otherwise have to wait for a time synchronization before beginning recording due to ability to generate valid filename timestamps. A bare RTC for rPi costs $6. They are also available as part of certain GPS units.

3) GPS feature. The chinese radio GPS is generally unreliable, and very imprecise. It has "speed" jumps of approximately 4 kph. It loses satellites frequently. It doesn't work correctly upon the unit waking up and often requires connections from GPS applications to be restarted (usually by killing the application). We can use a GPS on the rPi to supply GPS data to the android car radio, and also use that GPS data to perform position logging on the rPi along with the dashcam.

4) Safely powering off. I'm thinking, for simplicity, to do this with a ULN2003 (about $0.75). This is a nice big PDIP chip, easy to work with. What these are, is a relay driver. To oversimplify, you put power in on one side, and more power comes out the other side -- enough to switch on a relay. The chip has 7 drivers built in. We can use either ONE driver and two diodes, or two drivers. There will be two inputs -- the switched 5V from the car radio USB port (5V), and a GPIO pin from the rPi (3.3V). We also need the rPi to know the state of the car power, which means reading that 5V on one of the rPi's GPIO pins ---BUT!!!--- the rPi GPIO pins are NOT 5V tolerant! That means adding a couple of resistors (3K3 and 2K2) to turn the 5V down to 3V. About another $0.02. And, of course, a relay. Well, actually... that isn't necessarily a requirement. The drivers can be activated in parallel to add up their individual currents, which are 500 mA each, so we could potentially just power the pi directly off the ULN2003's outputs. While recording 1080p video, the pi-ZW is said to draw about 230 mA, and this is with a keyboard and HDMI monitor plugged in. The highest draw is the piB, which draws 480 mA in the same conditions. 2B and 3B draw 350. Now of course, the wireless will add to that, and also an additional camera will add even more. If we run it with two drivers, that will provide it with 1A, which is certainly enough power for it. --- note: It is not absolutely essential to do this. We could just run it straight off the radio's USB power, or an ACC controlled 5V adapter. This could corrupt some of the last segment of the final recording, but the filesystem (ext4) is journaled and will be fine upon reboot even if it abruptly powers off. We could also perform a semi-safe power off if using an rPi with wireless, since the 5V line remains on for a period AFTER ACC off. We can use a bluetooth state connection to inform the rPi to stop (and restart, if power resumes before ACC power back on) recording. --- the advantage to a controlled power off is to allow power to remain ON during an emergency. I.e., steering wheel input combination triggers a "don't shut off" mode to keep recording even if the ACC is turned off. --- we also gain the ability to keep recording for a configurable period after the ACC is switched off, even indefinitely. Will have to make power consumption measurements to determine how long it can safely record for after the car is turned off in order to avoid draining the battery too far.

Note: The chip has SEVEN drivers, so we can do other cool things with it as well, like overriding the ACC power input to the car radio, or controlling the ejector seat.

[lbdroid]

Some initial (and not complete) stages to implement; 1) make it record videos, with manual configuration for one or two cameras. This stage will be completely independent of car radio. 2) Interface it with car radio to configure recording parameters via GUI. Car radio will log GPS. 3) Car radio to be able to access and view videos stored on rpi, protect/unprotect, upload to gdrive. 4) Car radio will optionally send start (with current time), and stop instructions to rPi. 5) Add RTC to rPi, car radio will be able to set use of RTC, and send time updates as needed. rPi will then be able to use system time to set video filenames. 6) Add GPS to rPi, rPi will then be able to log GPS (car radio will stop logging GPS if rPi has one). 7) Add power control to rPi. 8) Add SWI to rPi.

[lbdroid]

Regarding safe shutdown process...

Shorting pin5 to GND apparently is able to resume the device from halt -- effectively, a partial reboot.

There is a potential for a dead zone situation to happen. ULN2003 with two control inputs for power control. GPIO OR 5V_USB. Say the USB goes off, then the pi begins a controlled shutdown (sync; halt), what happens if the USB comes back on after the halt, but before the power runs out? The board could be halted with no way to wake up or reset it.

Can it?

Perhaps not, discussed later.... But the point is, adding a FET to the end of the 5V_USB in order to pull pin5 to GND would resume the board from halt when the 5V_USB switches back on.

The way around the halt problem without worrying about resetting it from halt; don't halt it. The reason for halting it would be to protect the filesystem on the sdcard from losing cohesion when the power is removed. That's it. Aside from that specific issue, there is no other reason to perform a controlled shutdown.

So instead of performing a full standard shutdown, stop all processes that write to the filesystem, synchronize all the filesystems, remount them read-only, and then instead of actually halting it, just switch off power to the ULN2003's input.

The key at this point, is to keep watching the GPIO where the 5V_USB input is received. If it comes back on before the power is switched off (which means that the power does not actually get switched off), then we remount everything RW and restart all the processes that write to the filesystem.

There is actually one other element to the halt problem that could cause the power switch to fail if we tried it; does the processor reset GPIO pins to their default state when it halts? If it does not, then the only way we would be able to halt and power off, would be to also add a capacitor to the ULN2003's input that would keep the switch turned on for a moment after the GPIO turns off. A moment long enough to halt the system. This seems to add some complexity that isn't strictly necessary. 1 FET, 1 cap, 1 diode.

However, there is actually an alternative to all the complexity that might make it possible to reduce ALL of the power control parts down to just a single FET.... piZ idles at 0.51 W, which turns into 12.2 Wh per day, or roughly 1% of a car battery. Now (1) that is a piZ, not a piZW, and (2) that is idle, which is obviously going to make it higher than one that has everything turned off and in a halted state.

So maybe what we do is just take that 5V_USB signal and use it to trigger that FET in order to pull pin5 low and resume the thing. After booting up, we can also use the state of pin5 to find out when the 5V_USB goes off, and use that signal to halt.

I'm seeing claims of 15-30 mA for "any raspberry pi after halt" (which I will have to validate, of course). PiZ would be on the lower end of that, so 15 mA. That would turn into 0.075 W, or 1.8 Wh per day, or less than 0.2% of a car battery per day, or 65.7% of a car battery PER YEAR. So assuming no other parasitic power drains on the car, and an imaginary battery that doesn't lose charge all on its own just by sitting there, you could leave a car powering a halted rPiZ for over a year, and still start it. I think the 1-FET solution is viable.

[lbdroid]

Stage 1 status: partial.

How to implement; 1) Obtain a raspberry pi, preferred pi zero w. 2) Obtain one or more cameras, one of which may be a raspberry pi compatible CSI camera, all others must be UVC cameras -- because the pi only has ONE CSI port. 3) Download raspbian jesse lite and dd it to a uSD card; https://www.raspberrypi.org/downloads/raspbian/ 4) mount the ext4 (second) partition on the device and edit the file /etc/wpa_supplicant/wpa_supplicant.conf, add your wifi network to the end of it, like this;

network={
  ssid="YOUR_WIFI_NETWORK_NAME"
  psk="YOUR_WIFI_NETWORK_PASSWORD"
  key_mgmt=WPA-PSK
}

5) put it in the rpi and turn it on. Shortly you should be able to ping the device by its IP address. You can figure out the device's IP address because your router should show a new lease for a device named "raspberrypi". 6) ssh into the device. Username is "pi", password is "raspberry". 7) sudo apt-get update 8) sudo apt-get install screen 9) sudo raspi-config 10) Find INTERFACING OPTIONS --> CAMERA --> YES (enable it). 11) sudo su 12) echo "bcm2835-v4l2" >> /etc/modules 13) vi /etc/rc.local 14) add the following(-ish) line BEFORE "exit 0" screen COMMAND 15) cd /home/pi/ 16) wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-armel-32bit-static.tar.xz 17) tar -xvf ffmpeg.tar.xz 18) mkdir bin 19) cp ffmpeg-static/ffmpeg bin/ 20) chmod +x bin/ffmpeg 21) sync; reboot

The COMMAND I used to test it was this: /home/pi/bin/ffmpeg -f video4linux2 -input_format h264 -video_size 1280x720 -i /dev/video0 -f video4linux2 -input_format h264 -video_size 1280x720 -i /dev/video2 -c:v copy -map 0 -map 1 -f segment -segment_time 60 -reset_timestamps 1 test%03d.mkv

/dev/video0 is the CSI camera, /dev/video2 is the UVC camera's h264 output (/dev/video1 has mjpeg and raw on this particular camera)

Note that each "-f video4linux2 -input_format h264 -video_size 1280x720 -i /dev/video0" is ONE VIDEO INPUT. After all of the video inputs, "-c:v copy" means just record the frames directly without any kind of processing or re-encoding. "-map 0 -map 1" means to map the 0th and 1st of the input tracks to the output -- this can be left out if there is only one input. "-f segment -segment_time 60 -reset_timestamps 1" means to cut the file every 60 seconds (approximately -- next I-frame of the first track) with timestamps running from 0 - 60. "test%03d.mkv" is the file name, where %03d will be replaced by the sequence number.

Two things to do before stage 1 is completed; 1) each time ffmpeg starts, it will overwrite, or try to overwrite, the existing files. Need to come up with a way to generate a unique prefix for the file name. This problem will not be present if device has RTC, since the filenames will be datetime based. What I'll do here is pretty simple; store a number in a file called "FILEPREFIX", read that file into a variable in the rc.local, increment it, write it back to the file, and set the filename using that prefix, say "$PREFIX-dashcam_%03d.mkv". If you start your car up 100 times every day, then in 10 years, it will only grow to 365000, so no worry about growing too big. 2) cleanup script -- that will end up being a cronjob that runs every few minutes to wipe out the oldest files in the event that storage space drops below a threshold. I actually have exactly the right script on my surveillance camera computer, when I have a moment, I'll grab it and fix it up for this application.

[lbdroid]

Changed my mind about what is left to do in previous message.

My initial implementation will involve some rudimentary communication between the car radio and the rpi.

• send time.
• send start/stop signals and set filesystem RW/RO to make power off safe.
• receive recording state. ** this will solve (1) above without requiring RTC, and will allow the recorder to be simply plugged in to the radio's USB port. I.e., easier wiring without compromising filesystem.

I think that the best way to approach this is to go with a combination of maybe libmicrohttpd and sqlite3 to write a daemon in C for the rPi. This will minimize the impact on the rpi in terms of memory, storage, and CPU time, which is especially important for those going with original pi or piZ[W], which are much weaker in memory and CPU than Pi2 or 3.

The daemon will listen on a tcp port for instructions (update system time and start recording, stop recording, save new configuration, show current state, list recordings, retrieve recording, protect recording, unprotect recording). And on a dedicated thread, it will periodically check the storage state and delete old unprotected recordings.

I'm thinking of also doing a bluetooth control channel. The connection and pairing is simpler than trying to figure it out with wifi. The main issue with bluetooth is that it may shut off immediately with ACC power, which would prevent it from sending a STOP. To work around that, since we know that the power stays on the USB for about 5 minutes after ACC power off, we can use a watchdog timer to keep the rpi in a "recording" state. So lets say the rpi should expect to read a bark every 3 minutes in order to continue recording. We can SEND a bark every, lets say 30 seconds.

[lbdroid]

The approach I'm starting with, and I have to emphasise that this is just STARTING with, will be to put the car radio into wifi hotspot mode, and connecting the rpi to the car radio using that.

After the connection has been established, the car radio will send start, heartbeat, etc. signals to the rpi using http.

And maybe just share the video using dlna running something like mediatomb? Might be the simplest solution....

[lbdroid]

I've got a really good start on the rpi side software. Lots of features are in place to manage and control an instance of ffmpeg for dashcamming from one or more cameras. Its at the point now where I need to switch over to the pi and do some actual debugging.

Not doing anything stupid like mediatomb. Pretty much everything is going to be self contained in "picamd".

Description... HTTP requests it responds to. GET /stop -- stop ffmpeg, remount,ro datafs. GET /list -- lists all recordings, xml, including flag indicating if file is "protected". Ordered in reverse chrono. GET /check -- checks the status of ffmpeg, one of "not started", "running", "error", or "terminated". I think I'll update that to xml to be consistent. POST / with "protect=filename" -- protect "filename" from the reaper. POST / with "unprotect=filename" -- allow file to be reaped. POST / with "record=unixtime" -- set the system clock to "unixtime" and start recording. ** all of the post functions involve writes to the fs, so they'll remount,rw as needed, and in the case of protect/unprotect, they will resume ro following the move if the fs was ro before.

Configurations (currently hardcoded, but going to break them out); port -- tcp port. sdev -- the device file containing the data filesystem on which we will record data. path -- the mountpoint of the data filesystem. useWD -- boolean, indicates whether or not we will be enabling the watchdog timer (automatically stop recording and remount,ro after 2 minutes without a bark) hasRTC -- boolean, true to ignore the unixtime value of the "record" parameter. If the pi has an RTC, we should use that as a time source. standalone -- boolean, set to true to begin recording automatically rather than waiting for a POST / with "record=blah".

Notes: when using standalone mode, we will be mounting the data FS as SYNCHRONOUS, which is a little bit slower, but keeps the filesystem more up to date, so if the power is suddenly lost, the amount of data that will be lost will only amount to a few frames at most. I'm thinking about mounting it that way anyway, but will have to see how much impact it has on performance. It may not have any significant impact, since this FS will be mainly linear write.

I've also identified a mechanism through which to improve the robustness of the rpi sdcard filesystems, and the rootfs in particular; which is to mount the rootfs as readonly, with a readwrite overlayfs backed by tmpfs (ramdisk). That, along with synchronous data partition, and fsck'ing the data partition upon startup, should keep it sufficiently intact for long term dashcamming, even if not using the watchdog.

[lbdroid]

Did some early on-device testing, and its working surprisingly well.

Required 3 very minor changes to deal with raspbian shipping a somewhat out of date version of libmicrohttpd and different compiler than I'm used to, but aside from that, it actually responded properly to all http commands I sent it, with one very very minor exception where it spit back an error (note: not a crash, just an error message) instead of the status of the ffmpeg process after it had been terminated.

Now I need to implement the overlayfs as described in previous message, and do the car radio side, which is fairly simple relaying of http commands, and start actually using it.

[lbdroid]

I'm down to 3 2 items on the "TODO" list for the daemon process running on the rpi;

1) File naming when operating STANDALONE and without RTC. In this case, we cannot rely on the rpi's clock, which will always start up at the same incorrect time, leading to filename collisions and incorrect timestamps. My solution will be to store a file at /mnt/data/PREFIX, containing a prefix number for the recording set that is incremented each time that ffmpeg is started. From that, ffmpeg will also perform a second set of incremental filenaming. The file name will look something like this; "cam{prefixnum}{seqnum}.mkv" -- the reaper will also have to be made aware of this so that it knows which files are the oldest by their prefix and sequence numbers -- basically, a reverse alphasort rather than a reverse chronosort.

2) Some mechanism for loading the parameter set for ffmpeg besides the current hardcoded commandline. I might use another file for that, /mnt/data/CMD

3) avahi/zeroconf for advertising the device's presence on the network (head unit's wifi hotspot).

?) perhaps instead of keeping those values (1 and 2) in dumb files, they should be kept in an sqlite database. This would be good for future GPS logging as well.

After that, its going to be this; 1) Headunit side to implement avahi/zeroconf to discover the rpi on the network. 2) Headunit to switch to wifi hotspot mode always/automatically. I'm thinking of doing this in the bluetooth tethering application. 3) Headunit side application to send proper commands to the rpi and interpret responses.

FUTURE;

• I'm going to have to expand on the networking options, since some people use their head unit's wifi to obtain a data connection with their phone. You can't put the head unit into client and hotspot mode simultaneously.
• implement changes for use of RTC.
• implement GPS logging.
• feed GPS data via wifi/http to the head unit to overcome its bullpoop GPS.

[lbdroid]

Hahahaha, turns out that avahi-daemon is already installed and running on raspbian. Nothing to do there.

[lbdroid]

Ok, very very close here. Remaining to do before it is actually daily usable; -- rpi daemon: In standalone mode, switch to reverse alphasort for reaper function.

-- Android side; 1) auto wifi hotspot, 2) send GET /list and turn response into list of files, 3) implement protect/unprotect, 4) implement PLAY (with VLC or other media player by sending URI) 5) implement send to for remote backup.

*** NOTE: Android side is already able to discover rpi via avahi, instruct it to record/stop, and query recording state (every 5 seconds, secondary use is to reset the watchdog timeout that the rpi will use to decide when to shutdown recording).

Further note: rpi boots up into a READ ONLY ROOT mode, and auto-launches the daemon process. A recording data only partition is created on its SDCARD, which is mounted RO/RW+sync as needed when filesystem changes must be made.

[lbdroid]

pi daemon is pretty well in place now, all major functionality is implemented, and it seems to work pretty solidly. I've even adjusted the configuration of raspbian to make it possible to record audio as well. The pi really doesn't have the ability to compress the audio, so it comes in as raw PCM and runs about 11-12 MB/min. The h264 1280x720 stream produced by my UVC camera is running about 30 MB/min, so we are looking at about 1/3 increase in file size for audio. May or may not be valuable to everyone, but the option is now there.

So now I'm onto getting the major functionality for the car radio side in place.

Future for the pi side will be this; 1) Enable GPS logging. 2) Enable GPS serving (to provide better location to the car radio than its own butchered GPS).

[lbdroid]

Actually something that would be really fun... storing videos encrypted using public key type cryptography. I wonder if I could do that on the pi? This would prevent the video from being grabbed by, for instance, law enforcement. Not that you have anything to hide, but as a matter of principle, things you own shouldn't rat you out. I.e., if you run a red light, the cops still should have to prove it the same old way, not steal data from you to back them up.

[lbdroid]

I think that the little CPU on this thing is getting a bit too stressed out when using SYNC and also recording audio. Or it could also be that there is something messing up with the audio stream itself, I'm getting some DTS errors on the console when recording audio. I'm going to try turning off SYNC and seeing how that handles audio. If it clears it up, then that was the issue. If not, then I'll look into applying my frame re-sequencing hacks to ffmpeg again for rpi.

[lbdroid]

The Android application has the following TODO list;

1) If the android service is already started before the rpi connects to wifi, it isn't auto-starting it. Easy fix. Fixed 2) When a file is protected, the checkbox on the UI doesn't reflect that state. Again, easy fix. Fixed 3) Have not yet implemented facility to upload files. DONE -- its actually incredibly cool how I implemented this... ContentProvider implementing a PipeDataWriter that feeds the http stream from the rpi right out to whatever is asking for it. 4) Optional GPS logging. Easy, just need to re-enable code that I already have. PARTIAL 5) Send unix time to rpi in record request. Done

** Note about GPS logging: it is implemented in that the car radio will store gps logs on its own internal storage, it will reap them after 1 million lines (about 24 hours worth of logs), and it will upload them to google drive. HOWEVER, I would also like to set it up to be able to feed the gps logs to the rpi. In addition, I will later on implement the ability for the rpi to record gps logs from its own gps receiver, in which case we will need to inform the car radio to upload gps logs from the rpi, but not record any gps logs at all. *** PARTIALLY IMPLEMENTED: pi side is in place. It creates a gps.db file in the data storage directory, and can be fed logs using "POST /" with "gpslog={something}", also implemented the reaper on the pi.

Other Android TODO: 1) Auto-enable wifi hotspot mode. DONE. Also implemented in Bluetooth Tethering controller, which is also a better place to run it from.

Non-Android TODO: 1) I seem to have the reaper's settings backwards. Its deleting files until the free space exceeds 90%, it should be deleting files until the free space exceeds (100-90)%. Trivial fix. Fixed. 2) A bit more testing and measuring with respect to AUDIO recording, multi-stream video recording, and impact of mounting the FS with SYNC. It can handle a HELL of a lot more when SYNC is turned off. About all I can get with SYNC ON is a single h264 video stream by itself. With SYNC OFF, I'm easily getting multiple video streams, as well as audio. Hmm, seems not entirely. I am again getting some glitches that are tied specifically to the audio. Its probably the audio DTS errors its printing out, screwballing the entire file. I'll have to look into that some more and see what I can do to smooth it out. I really would like to capture audio.

If it turns out that we can do 2 video streams + 1 audio stream with SYNC OFF, then I'll go back to only enabling SYNC for standalone mode, where it really is needed for FS integrity. Sync is now on ONLY for standalone mode.

[lbdroid]

Only one more thing to do before I am ready to share this; Android side must be able to send gps logs to the pi.

[lbdroid]

Ok, that's in place. Only thing is that it seems to be a bit heavy on the CPU. Its at 10% CPU utilization just for recording the logs. Its probably because there are a lot of posts being made... roughly 12 per second. Maybe what I'll do to improve it is to cache several seconds of them and send them in a single request, say every 30 seconds or 1 minute.

[lbdroid]

Got that under control just by turning off sqlite's sync option. Last bug is pretty ugly, seems that when the rpi is not accessible, the gps logs are building up pretty severely -- they can't time out and die fast enough.

[lbdroid]

Its still a little bit heavier on CPU than what I think I can get it to with more work, but I think its ready to start sharing. I'll package it up over the weekend and write up some instructions. Some time next week it should be ready to go.

Out of the box, it should work with either a Raspberry Pi Zero W, or a Raspberry Pi 3B, since these two have built in wifi. Others should possibly work, but they'll need a wifi dongle, and possibly some other configurations that I won't be looking into.

[lbdroid]

I've uploaded the files needed for this. See this project's README file for details.

[lbdroid]

I'm moving ahead with the GPS stuff. I have the adafruit gps hat working on the pi zero w. One minor concern that I had with it is that the pi zero w might share the pi 3B's "uart defect". My concerns were warranted, as it turns out that, in fact, it does share that defect. The good news is that it isn't that big of a deal to fix. The whole internet said that the uart was used by the "wifi bluetooth module", well it turns out that it is only actually used by the bluetooth component of the wifi/bt module, AND, the "pi people" even had a simple way to disable the bt and return the uart to normal mode. They also provide a way to assign the "lesser" uart to the bt module, but I'm not sure how well that would work. Presumably, it would work just fine in bluetooth highspeed scenarios, which actually piggybacks on the wifi hardware (too fast for uart!).

So gps is working, withOUT disabling the wifi.

And now that I've been looking at gpsd, I think that sharing the gps with the crappy car radio will actually be incredibly simple. Basically, gpsd can do it just by applying the correct configurations. And yes, the Android side makes use of gpsd as well, so its just a matter of tying them together.

[lbdroid]

I've got the GPS working nicely from the system's point of view, not yet integrated into picamd. That includes using the GPS time along with the PPS signal for very high precision timekeeping, and configuring it to share the GPS NMEA stream via gpsd.

ntp is reading a jitter on the PPS signal of just 0.006 0.002 (oh my!) -- the longer it runs, the tighter ntp ties in to it, to put that into perspective, the NMEA stream is giving a jitter of 14.558, and the internet NTP servers are all in the range of 5-14. So this is quite some amazing precision here.

Of course, you need to actually have a GPS with PPS in order to get that. Adafruit gps hat does.

Note that no special system configurations are needed to take advantage of this. Any conventional 3.3v GPS that speaks NMEA over UART will do the job, just wire it in. If the GPS and PPS signals are not present, it will fall back to remote NTP servers. If there is no access to remote NTP servers, picamd will accept a system time update in the record parameter. If all else fails, it will use fake-hwclock, which reads the "last" system time from /etc/fake-hwclock.data and resumes from there. Note that the value of this clock is always wrong, but its kind of needed in order to avoid a problem caused by the time moving backwards (i.e., always resetting to January 1st, 1970).

Next up, I'm going to add the ability for picamd to log gps from its own gps. This will be controlled by setting the record key with the value "gps".

After that, I have a few more requests for picamd to be able to respond to, and it will be feature complete.

1) getcams (GET) -- dump the capabilities of all of the cameras (and sound inputs) connected to the pi. DONE 2) setcams (POST setcams=...) -- write the ffmpeg commandline back to the pi. DONE 3) setwifi (POST setwifi=...) -- write the settings for wpa_supplicant.conf to the pi. DONE 4) delete (POST delete=...) -- delete that video. DONE 5) reboot (GET) -- reboot the pi. DONE

[lbdroid]

Last TODO on the rPi side.... --Get it to record its own GPS to a log file when the camera(s) are on by using POST / record=gps DONE

[lbdroid]

Well... I wasn't expecting to have to modify gpsd for this project, but I found a couple of issues with it that needed to be fixed up. And yes, I did submit my patches to upstream gpsd project.

First off, there was a bug in the gpsd client/server mode (where one instance of gpsd feeds off of another). It was a kind of two pronged bug in that the client side was sending requests in a bad manner, and the server side wasn't being smart about receiving them. A fix to either side would have done the job, so I patched both sides. That means that a patched client will work with an unpatched server, and an unpatched client will work with a patched server.

The second issue isn't so much a bug, as it is a missing feature. Or maybe more correctly, a conflicting feature. Basically, gpsd has the ability to update the system time using ntp. The problem is that gpsd is coded not to start updating ntp until it has received a valid gps fix. That sounds good, and it usually is, EXCEPT if your gps happens to have a built in RTC, which means that it knows what time it is, even without a fix. So I added a parameter "-r" to it, that tells it to update the ntp even without a satellite fix.

[lbdroid]

Well, I've been working all day trying to figure out a way to mux a generic text stream into an mkv file using ffmpeg. Subtitles came to mind, so I modified gpspipe to output SubRip formatted subtitles to a fifo, and attempted to read that with ffmpeg. Unfortunately, ffmpeg isn't up to the task, and demands that the entire file be read at once. That obviously isn't going to happen in a live streaming situation.

Which means that I'm back to logging it using sqlite3.

BUT, I have come up with another brilliant idea! And that brilliant idea is to select the appropriate set of subtitles from the database and mux it into the file when it is being requested back for playback or uploading.

[lbdroid]

So I just finished the last of the pi-side TODO's, but now I have one to add, which is selecting from the GPS log and muxing the data into the video file that is requested by the client.

And since I'm now thinking of GPS logs, I should adjust my reaper to work through sqlite and make sure that the logs correspond to the remaining files. I.e., delete gps logs only when the corresponding video has been reaped.

[leaftail]

I've been following your thread for a while now. Sounds like you are getting really close to being able to start testing bitrates from the two cameras. I look forward to seeing this solution come to fruition. Please keep the updates coming.

[lbdroid]

I've actually captured more than 100 hours of video using a dual camera configuration like this already. There are no concerns at all regarding bitrates. The card I'm using seems able to consistently handle a bit more than 4 MB/s (240 MB/min) sequentially written. With a resolution of 1280x720, that is theoretically enough for 8 cameras in h264 (30 MB/min), or 4 cameras in mjpeg (60 MB/min). However, the pi zero doesn't have enough cpu for that, it would put it over 90% average load. If you want to go nuts though, a 3B can certainly handle it.

On June 30, 2017 7:49:36 PM EDT, leaftail notifications@github.com wrote:

I've been following your thread for a while now. Sounds like you are getting really close to being able to start testing bitrates from the two cameras. I look forward to seeing this solution come to fruition. Please keep the updates coming.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/lbdroid/FFMpeg-DashCam/issues/19#issuecomment-312395470

[lbdroid]

I'm all done on the Pi side now. Well, feature complete and "lightly tested". There are undoubtedly some bugs somewhere in there, its a pretty complex piece of software now with multi-processing, threading, sqlite databases, gps logging, http, and a whole slew of other things that have to be coordinated in a sane manner.

I'm going to upload a new Pi side image tomorrow. The current Android side should still work with it for the most part (gps logging using an android source will be broken), but some updates are due, which I will get to after uploading new Pi side image.

There are now two ways to ask the Pi for a video clip... GET /filename.mkv -- to get the ORIGINAL file.... and GET /filename.mkv?gpslog -- which will take a "few seconds" longer (3-5), but will result in a file containing a subtitles track. The subtitles track contains the NMEA sentences that were captured at the same time as that video was taken, and in fact, they will be time-sequenced to within a few hundredths of a second from the actual video.

The gps subtitles track will NOT display by default. In, for example, VLC, you would have to go to Subtitle --> Sub Track --> Track 1 to turn it on. Viewing the log isn't really the purpose anyway, but rather the purpose is to store the gps logs in with the dashcam footage as a concise bundle of "everything applicable". The subtitle track can be easily extracted back to a text file for (i.e. forensic analysis) using the following; ffmpeg -i file.mkv -f srt - 2>/dev/null | grep "\$G"

One thing to note; the subtitles track will be a fixed length of 62 seconds, beginning at the same moment as the video file, unless there are not 62 seconds worth of gps logs at that time. It will always be longer than the actual video file, and will cause a media player to output a file length that is 62 seconds. The first segment video of any capture is going to be a random length from 0 to 61 seconds inclusive (max 60 for mjpeg) -- depends on what portion of that minute has elapsed when the video begins. Subsequent segments will be precisely 60 seconds for an mjpeg capture, or anywhere from 59-61 seconds for an h264 capture, depending on the position of the i-frame. The final segment will be again random, since it depends on the portion of that minute that has elapsed before the recording is stopped.

[lbdroid]

FYI: My gpsd patch has been accepted upstream!

[lbdroid]

Well... here is one unfortunate consequence of... something. I don't yet know precisely WHAT, although I do have a guess.

I'm getting some interference with GPS reception. Got in the car this morning and there was zero GPS signal. No surprise on the china car radio, frustrating on the rpi, and completely baffling on the Nexus 6, which is the world champion of the under 1 second lock.

Hit the RESET button on the car radio, and the other two units instantly grab a lock. After the china boots up, it too grabs a lock, but all three devices now have a WEAK lock (during reboot, the lock was strong, after, weak). I don't think that the pi rebooted when the china did, I think that the USB power remained continuous. That would imply that the pi is not the source of the interference. This is also confusing since the china didn't have too bad of gps problems before the pi was there, and certainly not to the extent of messing with the nexus 6. I would almost be surprised if it wasn't caused by the pi, and the incredibly long (1 yard) CSI ribbon cable. Hmm..... Worth noting is that the interference was independent of whether or not the CSI was actually operational. Experiment 2: unplug the pi from the china. Interference goes away. Yet another Hmmmm...... Ok, so what I'm thinking is that MAYBE, the pi is acting as an antenna for some interference that is being created by the china. Essentially, if the china is sending some kind of interference up the power wire, that could be driven right into the CSI ribbon, which is unshielded and would make a great antenna.

Also worth noting is that I was able to get a GLONASS lock from the china, and eventually also, it seems, from the Nexus 6. GPS status was showing zero satellites locked, yet was reporting a position, and despite being somewhat "off", it was far too dynamic of a position to be cell tower triangulation.

I'm 100% sure that it isn't noisy car power that is responsible. I also tested with the engine shut off, which would result in a perfectly clean power supply straight off the battery. No difference.

I'm going to try with an independent power supply powering the pi.

EDIT: According to other people on the interwebz, it is the pi camera. There are lots of reports of it massacring the GPS signal. I guess it actually did reboot when resetting the china. Undoubtedly, this is far worse on account of the extreme length of the csi cable I'm using (1 yard). But also, there is good indication that shielding the cable will solve the problem, and that is pretty easy and cheap to do. A little bit of aluminum duct tape, and of course, remembering to tie the aluminum duct tape to the car chassis.

[lbdroid]

Ok, well the solution was easy. The implementation was difficult, but only because it meant having to remove and then reinstall the ribbon wire in the car headliner. I have the CSI camera mounted on the top of the back window, and the pi mounted in the overhead console, with the ribbon wire running between them.

Just wrapped the ribbon wire with some aluminum duct tape from home depot, and near one end added in a wire that I hooked up to a point on the car chassis to tie it in to common negative. To the best of my ability to detect, the interference has been completely eliminated, so GPS's are working perfectly.