ranveerm commented 2 years ago

"Year in Review" posts contain several "content" sections, with each containing several elements that are arranged in a specified format (see image below).

The trivial mechanism to implement this within a post is to create a HTML block for each one of the elements present. However, this has numerous disadvantages-

HTML component is repeated for every element present, and hence does not scale well with the number of elements
A change in styling in the future requires each of the above block to be accordingly modified, which is not only arduous, but also error prone
Data is manually copied instead of automatically retrieving it

Day One Export

An alternative solution is to use the data existing is Day One (which acts as the source) and create a "view" that dynamically retrieves and presents this data. This addresses all of the issues from the trivial solution from above.

Day One facilitates data to be exported as a JSON file, but its contents are not configurable. The directory structure along with the for this JSON file is specified below.

├── <data>.json
└── photos
    └──<photo ID>.jpeg

{
    "metadata" : {
        "version" : <String>
    },
    "entries" : [
        {
            "creationOSName" : <String>,
            "editingTime" : <Double>,
            "creationOSVersion" : <String>,
            "text" : <Markdown Text>,
            "isPinned" : <Bool>,
            "creationDeviceType" : <String>,
            "uuid" : <String>,
            "creationDevice" : <String>,
            "modifiedDate" : <String>,
            "creationDeviceModel" : <String>,
            "isAllDay" : <Bool>,
            "location" : {
                "region" : {
                    "center" : {
                        "longitude" : <Double>,
                        "latitude" : <Double>
                    },
                    "radius" : <Int>
                },
                "localityName" : <String>,
                "country" : <String>,
                "timeZoneName" : <String>,
                "administrativeArea" : <String>,
                "longitude" : <Double>,
                "placeName" : <String>,
                "latitude" : <Double>
            },
            "photos" : [
                {
                    "fileSize" : <Int>,
                    "orderInEntry" : <Int>,
                    "creationDevice" : <String>,
                    "duration" : <Int>,
                    "favorite" : <Bool>,
                    "type" : <String>,
                    "identifier" : <String>,
                    "date" : <String>,
                    "height" : <Int>,
                    "fnumber" : <String>,
                    "width" : <Int>,
                    "md5" : <String>,
                    "isSketch" : <Bool>,
                    "focalLength" : <String>
                }
            ],
            "weather" : {
                "moonPhaseCode" : <String>,
                "temperatureCelsius" : <Double>,
                "weatherServiceName" : <String>,
                "windBearing" : <Int>,
                "conditionsDescription" : <String>,
                "pressureMB" : <Double>,
                "moonPhase" : <Double>,
                "visibilityKM" : <Double>,
                "relativeHumidity" : <Int>,
                "windSpeedKPH" : <Double>,
                "weatherCode" : <String>
            },
            "creationDate" : <String>,
            "duration" : <Int>,
            "starred" : <Bool>,
            "richText" : <Rich text>,
            "timeZone" : <String>
        }
    ]
}

An example for the markdown text is shown below-

"# Brave New World\n\n![](dayone-moment:\/\/84503A6D603E487884B3F6C10A07912B)\n\nI don’t believe I enjoyed the reading experience, but I was satisfied nonetheless\\."

Scheme Conversion Script

Note that most the the above fields are not used for the visual present at the top (and it is not desirable to even expose a few of the keys present due to their sensitive nature). A script needs to be generated that converts the above scheme to the below-

{
    "entries" : [
        {
            "title" : <String>,
            "subtitle" : <String>,
            "description" : <String>,
            "photo" : <String>,
            "creationDate" : <String>,
        }
    ]
}

Resources

ranveerm commented 2 years ago

The first task undertaken was to perform the json of json file described in https://github.com/ranveerm/ranveerm.com/issues/16#issue-1161061547. Given it is desirable to have this process repeatable and idempotent (to allow integration with a potential CI pipeline in the future), 2 separate data files should be maintain- 1 for the source (which should not be mutated) and one for the data used in the website (re-written each time the script is run).

This procedure also brings to light the need to maintain follow a strict guideline when filling in DayOne entries, which might not always be desirable. In addition, the output of the data from DayOne might not always be in a form that is presentable in the post. For instance, DayOne might light each TV season for a particular program independently, but it might be more appropriate to aggregate them into a single entry for the final post.

As a result of the above, the script must facilitate an additional "modifier" data file that modifies the consumed data (i.e. replaces certain entries with the specified entries). The logic diagram for the described process is depicted below.

ranveerm commented 2 years ago

Following on from the pipeline described in https://github.com/ranveerm/ranveerm.com/issues/16#issuecomment-1083053098, all the contents for a particular year must reside in a directory named after the year. This directory must also reside within _data directory in the root of the project, to ensures it is consumable by Jekyll. The directory structure for this is displayed below. Note that content listed in [] represents multiple instances of this context (eg. [category] could entail multiple categories such as movies and tv-shows), and are followed by ... in the line directly below (in the same level) to emphasise this.

$ tree
.
└── <year>
    ├── DayOne exports
    │   ├── [<DayOne export>.zip]
    │   ├── ...
    │   │
    │   ├── [<category zip unwrapped>]
    │   │   ├── Journal.json
    │   │   └── photos
    │   │       ├── [<photo ID>.png]
    │   │       └── ...
    │   └── ...
    │
    ├── [<category>-modifier.json]
    ├── ...
    │
    ├── [<category>-processed.json]
    ├── ...
    │
    └── Images combined
        ├── [<photo ID>.png]
        └── ...

There are a few critical design decisions that have been made when constructing the above, which are rationales below-

Having the content contained within a specific directory will allow it to me supplied as a single input parameter within an Xcode project running a script
Source files are exported from from DayOne and manually i. copied to Day One Export Raw directory, ii. extracted and iii. renamed appropriately. Restricting all the within the confines of this directory ensures it can easily be prevented from being tracked by source control (which is important as the source data might contain sensitive information).
The <-modifier> and <-processed> files are the root of the current scope and not contained within a directory. While this adds clutter, it brings with it convenience (with respect to how it is referenced from content files), and given the currently limited amount of files and the clear naming scheme, the impact of the clutter is reduced.
All the images from source directories are copied to a central photos-combined directory. This is because Day One Export Raw will ignored by git and GitHub Pages will still need access to these images to render them.

ranveerm commented 2 years ago

17 Introduces a the first iteration of a script that converts DayOne exports to the relevant data formats. Whilst https://github.com/ranveerm/ranveerm.com/issues/16#issuecomment-1083086835 suggests placing the images in the same directory as DayOne exports data (i.e. `_data` being the root directory), this made referencing those images within a Jekyll post untenable as directories with a prefix of `_` have a special meaning. As a result, the images are copied to an appropriate directory within `assets` (in the root directory), which benefits by being in the same location of other images used within posts.

The directory layout (with reference to root) is hence-

├──assets/
│   └── images
│       └── posts
│          ├── [year-in-review]
│          │   └── day-one-exports
│          │       └── [<photo ID>.png]
│          │       └── ...
│          └── ...
└── _data
    ├── [year]
    │   ├── DayOne exports
    │   │   ├── [<DayOne export>.zip]
    │   │   ├── ...
    │   │   │
    │   │   ├── [<category zip unwrapped>]
    │   │   │   ├── Journal.json
    │   │   │   └── photos
    │   │   │       ├── [<photo ID>.png]
    │   │   │       └── ...
    │   │   └── ...
    │   │
    │   ├── [<category>-modifier.json]
    │   ├── ...
    │   │
    │   ├── [<category>-processed.json]
    │   ├── ...
    └── ...

In addition, the following outstanding tasks remain after #17-

[ ] Create facility to process multiple years within a single run of the script (currently the year is manually passed in as an input parameter)
[ ] Processing of "modifier" json file is yet to be implemented due to the complexity involved

ranveerm / ranveerm.com

Create a script to present Day One content #16

Day One Export

Scheme Conversion Script

Resources