Tweak output information in Log File

[IAmVigneswaran]

Could we print and state the current version of MarkersExtractor in the Log file before starting the operation?

[orchetect]

Could do, yes.

You can also do it yourself by piping the output of markers-extractor-cli -v to the log first.

While on this point, it could be a good idea to include the version in the result json file.

[IAmVigneswaran]

This feature could be particularly useful for our Marker Data app for debugging.

If Marker Data have several release down the line, when users encounters a bug or issue, it might be easier to trace?

The user might not be on the latest version of Marker Data.

[orchetect]

That sounds more like you want to log Marker Data version. It's more likely that the same version of MarkersExtractor would be used with more than one release of Marker Data. So logging MarkersExtractor's version alone would only help for debugging extraction. It wouldn't really help for debugging Marker Data.

[IAmVigneswaran]

It's more likely that the same version of MarkersExtractor would be used with more than one release of Marker Data.

True.

So logging MarkersExtractor's version alone would only help for debugging extraction. It wouldn't really help for debugging Marker Data.

Yes. We can have 2 log file. One for extraction and one for Marker Data's program log.

The idea is

/Users/UserID/Library/Application Support/Marker Data/Logs/csv2notion-neo_log.txt
/Users/UserID/Library/Application Support/Marker Data/Logs/airlift_log.txt
/Users/UserID/Library/Application Support/Marker Data/Logs/markerdata_log.txt
/Users/UserID/Library/Application Support/Marker Data/Logs/MarkersExtractor_log.txt

Would it make sense?

[orchetect]

Yeah, whatever works for you.

I can add the version to the CLI log output and a key to the result json file.

[IAmVigneswaran]

Thank you!

[orchetect]

Version output to both log and result file is done. Will be in 0.2.9.

[IAmVigneswaran]

Will close this for now.

[IAmVigneswaran]

When I try and use -v, I get an error.

Error: Unknown option '-v'

[IAmVigneswaran]

The version is shown in the terminal but it is not printed in the log file.

Also are we able to print exactly like in the terminal into the log file?

MarkersExtractor 0.3.0
Using Notion export profile.
Parsing XML from "XYZ"
Note that this may take several seconds for very large projects. Please wait...
Extracting Markers...
5%
10%
15%
Project start timecode: 00:00:00:00 @ 25 fps.
Found project media file: "XYZ".
Generating PNG thumbnail images into "XYZ".
Generating metadata file(s) into "XYZ".
20%
Exporting marker icons...
24%
Generating PNG images for markers...
.
.
.

[orchetect]

When I try and use -v, I get an error.

Sorry, I thought the argument parser had both -v and --version but now that I check, it's just --version.

[IAmVigneswaran]

I have just tested this 0.3.2

But for some reason, the full terminal is not printed into the log file.

I am using --log-level debug --log $ERROR_LOG.

This is what I get in the log file.

2024-01-04 17:56:40 info: 5%
2024-01-04 17:56:40 info: 10%
2024-01-04 17:56:40 info: 15%
2024-01-04 17:56:40 info: 20%
2024-01-04 17:56:40 info: 24%
2024-01-04 17:56:40 info: 27%
2024-01-04 17:56:40 info: 31%
2024-01-04 17:56:40 info: 34%
2024-01-04 17:56:40 info: 37%
2024-01-04 17:56:40 info: 40%
2024-01-04 17:56:40 info: 44%
2024-01-04 17:56:40 info: 47%
2024-01-04 17:56:40 info: 50%
2024-01-04 17:56:40 info: 53%
2024-01-04 17:56:40 info: 57%
2024-01-04 17:56:40 info: 60%
2024-01-04 17:56:40 info: 63%
2024-01-04 17:56:40 info: 67%
2024-01-04 17:56:40 info: 70%
2024-01-04 17:56:40 info: 73%
2024-01-04 17:56:40 info: 76%
2024-01-04 17:56:40 info: 80%
2024-01-04 17:56:40 info: 83%
2024-01-04 17:56:40 info: 86%
2024-01-04 17:56:40 info: 89%
2024-01-04 17:56:40 info: 93%
2024-01-04 17:56:40 info: 96%
2024-01-04 17:56:40 info: 100%
ata Demo_V1 2024-01-04 05-56-40".
2024-01-04 17:56:40 info: Exporting marker icons...
2024-01-04 17:56:40 info: Generating PNG images for markers...
2024-01-04 17:56:40 info: Creating result file "/Users/XX/Desktop/Marker_Data/App/CLI/result.json".
2024-01-04 17:56:40 info: Done

[orchetect]

It works fine when --no-progress is present.

Progress reporting to the log file is being called from a background thread I believe, which may be corrupting the log output.

A simple solution may be to just omit progress percentage output from the log file. It's not really useful and adds extra noise to the log.

[IAmVigneswaran]

It works fine when --no-progress is present.

Progress reporting to the log file is being called from a background thread I believe, which may be corrupting the log output.

Will using --no-progress affect's Marker Data's ability in reading MarkersExtractor progress percentage?

A simple solution may be to just omit progress percentage output from the log file. It's not really useful and adds extra noise to the log.

Let's omit progress reporting in the log file in the next build? We can also remove --no-progress flag.

[orchetect]

Progress in console, progress in log file, and progress reporting in GUI are 3 different unrelated concepts.

Progress will remain in console but be omitted from log file. GUI progress has nothing to do with this.

[orchetect]

I discovered why the log was getting corrupted. Mistakenly, the log file was being opened for writing twice - once by the extractor, and once by the progress reporter. That's now resolved.

• Console will continue to show progress (or hide it if --no-progress is passed) which is the same as before.
• Now, progress will always be omitted from the log file. --no-progress obviously has no effect there.
• GUI progress is unrelated. Nothing was broken with it, and thus nothing has changed.

Will be in 0.3.3.

Console output:

MarkersExtractor 0.3.2
Using CSV export profile.
Parsing XML from "/Users/stef/Dropbox/_coding/MarkersExtractor/GitHub Issues/Issue 37/SAM-BRS-TEST/BRS_LOCKED6_20210521 - 01 Opening Scene.fcpxmld/Info.fcpxml"
Note that this may take several seconds for very large projects. Please wait...
Extracting Markers...
5%
10%
15%
Project start timecode: 00:59:50:00.00 @ 25 fps.
Bypassing media file.
Generating metadata file(s) into "/Users/stef/Desktop/me-cli-output/BRS_LOCKED6_20210521 - 01 Opening Scene 2024-01-06 06-47-44 [CSV]".
20%
Exporting marker icons...
24%
96%
Creating result file "/Users/stef/Desktop/me-cli-output/result.json".
100%
Done

Log file output:

2024-01-06 18:47:42 info: MarkersExtractor 0.3.2
2024-01-06 18:47:42 info: Using CSV export profile.
2024-01-06 18:47:42 info: Parsing XML from "/Users/stef/Dropbox/_coding/MarkersExtractor/GitHub Issues/Issue 37/SAM-BRS-TEST/BRS_LOCKED6_20210521 - 01 Opening Scene.fcpxmld/Info.fcpxml"
2024-01-06 18:47:42 info: Note that this may take several seconds for very large projects. Please wait...
2024-01-06 18:47:43 info: Extracting Markers...
2024-01-06 18:47:44 info: Project start timecode: 00:59:50:00.00 @ 25 fps.
2024-01-06 18:47:44 info: Bypassing media file.
2024-01-06 18:47:44 info: Generating metadata file(s) into "/Users/stef/Desktop/me-cli-output/BRS_LOCKED6_20210521 - 01 Opening Scene 2024-01-06 06-47-44 [CSV]".
2024-01-06 18:47:44 info: Exporting marker icons...
2024-01-06 18:47:44 info: Creating result file "/Users/stef/Desktop/me-cli-output/result.json".
2024-01-06 18:47:44 info: Done

[IAmVigneswaran]

Thank you!