Camera Video download

[Lotuslisa]

Following is the steps to download videos:

1. src/subdirPathSetup.m, 2. src/extra/videoDownload/downloadSingleCamera.m There are three parameters for downloadSingleCamera - camera number, video path (relative to CITY_DATA_PATH), and number of minutes to download. For video path I suggest to use data/camdata/camXXX/Oct11-XXh.avi, where "XXh" is e.g. "10h" if video was recorded at 10 am EST (24h format).

But when I donwload the video, there is a bug: in src/extra/videoDownload/downloadSingleCamera.m there is one line "frameWriter.writeNextFrame (frame);" but it seems that in FrameWriterVideo.m the function writeNextFrame is deleted

[kukuruza]

Oh shit, will fix it in 15 min

[Lotuslisa]

Prof. Moura suggest to select cameras on the same street. I browse many cameras on the map (http://nyctmc.org/), but some are covered with dust, and others are with too crowded pedestrian and cars. Finally I found these cameras: in front of a bridge, on the bridge, and after a bridge. They have simple background, no cross road:

[kukuruza]

It was fine in the master, you just have to update to the current version.

[Lotuslisa]

Great! Thank you!

[Lotuslisa]

could you check if you have pushed the new codes? the newest commit on git is yesterday

[kukuruza]

Commit a5f758c55c6e38d7e54aff23cf0c7a479668ec68 introduces a new function to download multiple cameras: extra/downloadVideo/downloadMultipleCams.m and the way to set up automatic scheduler to run it.

Besides, the time zone was fixed up. Originally, time on a running computer was used for timestamps and video file name. Now that's fixed and is set by default to the NYC time.

downloadMultipleCams

Usage from Matlab:

cd src/extra/downloadVideo
downloadMultipleCams('camdata/schemes/ShanghangChoice.txt', 2);

Usage from Unix command line (cronTask.sh will open maltab and call the function):

src/extra/downloadVideo/cronTask.sh camdata/schemes/ShanghangChoice.txt 2

Both examples above will download 2 minutes of video from cameras listed in $CITY_DATA_PATH/camdata/schemes/ShanghangChoice.txt (the file is in google drive). They will save the videos in $CITY_DATA_PATH/camdata/camXXX/, where 'XXX' is the number of each camera.

Important issue: By default Matlab will create only as many parallel tasks as the number of cores on you machine. That is if you have 4 cores, but want 8 cameras, then first 4 videos will be downloaded, then another 4. To fix that go to Matlab -> Preferences -> Parallel Computing Toolbox -> click Cluster Profile Manager -> click Edit in menu. Then set 8 or whatever is the number of cameras you plan to work on. Then click Validation Results and validate if it works.

Another issue: A pool of 8 workers failed to initialize once on my machine with 2 cores. I could not reproduce the error though.

Schedule

Cron is a Unix tool which runs different tasks on schedule (e.g. check for system updates). It can be used to download video on schedule on Linux and Mac.

Please look up how to set up the scheduler with crontab -e. You can give it the path to <your_city_project_path>/src/extra/downloadVideo/cronTask.sh with parameters. You can't use environmental variables like $CITY_PATH$ in Cron. Instead please replace <your_city_project_path> with the full path on your machine.

Example on my machine below will download 2 minutes of video every day at 7:00 AM for cameras listed in ShanghangChoice.txt (it's in google drive)

0 7 * * * /Users/evg/projects/City-Project/src/extra/videoDownload/cronTask.sh camdata/schemes/ShanghangChoice.txt 2 > /Users/evg/projects/City-Project/log/downloadVideo.log 2>&1

[Lotuslisa]

Cron time setting:

30 6 * * * /home/lotus/src/City-Project/src/extra/videoDownload/cronTask.sh camdata/schemes/ShanghangChoice.txt 60 > /home/lotus/src/City-Project/log/videoDownload.log 2>&1

30 7 * * * /home/lotus/src/City-Project/src/extra/videoDownload/cronTask.sh camdata/schemes/ShanghangChoice.txt 90 > /home/lotus/src/City-Project/log/videoDownload.log 2>&1

00 12 * * * /home/lotus/src/City-Project/src/extra/videoDownload/cronTask.sh camdata/schemes/ShanghangChoice.txt 60 > /home/lotus/src/City-Project/log/videoDownload.log 2>&1

00 15 * * * /home/lotus/src/City-Project/src/extra/videoDownload/cronTask.sh camdata/schemes/ShanghangChoice.txt 60 > /home/lotus/src/City-Project/log/videoDownload.log 2>&1

30 18 * * * /home/lotus/src/City-Project/src/extra/videoDownload/cronTask.sh camdata/schemes/ShanghangChoice.txt 60 > /home/lotus/src/City-Project/log/videoDownload.log 2>&1