Closed tfabris closed 5 months ago
I've checked in one big fix for this, the fix is currently sitting in the issue69 code branch (not merged to master yet): 01a112e
This improves the speed so that it only takes 3 minutes to run instead of 11 minutes. I think I can get that 3 minutes down even lower if I work on the next section. The next slow part is the part where it checks each of the item's timestamps and sees if the timestamp is more than x days old, and decides whether to remove that video or not. That loop is pretty slow and comprises the majority of the remaining slowness. I am leaving this bug open until I can optimize that.
The slowness is in this SED command:
uploadsOutput=$( sed "s/\"$oneVideoId\"/&,\"actualStartTime\": \"$actualStartTime\", \"actualEndTime\": \"$actualEndTime\"/g" <<< $uploadsOutput )
The string "uploadsOutput" is large, and I'm re-sedding it dozens upon dozens of times. Each one takes about one second to process.
Doing "echo $uploadsOutput | sed" is no faster than doing "sed <<< $uploadsOutput", I tried that.
Investigating other methods.
I am going to close this as "finished". My initital optimization improved the runtime from 11 minutes to 3 minutes, and that's more than enough.
The remaining optimization was being a problem.
Here's what I was planning to do:
The problems:
So I'm halting this work for now and calling it good as-is. I'll save off the temporary work-in-progress stuff because there are some good notes in that code, some good things I could put to use later, but for now, this is dead in the water.
The script CrowCamCleanup.sh runs slower than I would like. With my current video playlist it's taking about 11 minutes to run on my Synology. The biggest inefficiency is the code loop here:
I think it can be made much speedier by re-doing the code more like this (the following code is from something I recently added to TestFile.sh):