arturdryomov commented 7 years ago

The End Game

Input

YAML as episodes declaration and Markdown for episode shownotes.

File Structure

├── episodes
│   ├── 11-migration-to-rxjava-2
│   │   ├── episode.yaml
│   │   └── shownotes.md
│   └── 12-instant-apps
│       ├── episode.yaml
│       └── shownotes.md
└── people.yaml

Subdirectories for episode grant some flexibility. Maybe we’ll decide to use different artwork for each episode or something like that.
Meaningful names instead of just 11 and 12 provide better search and browse experience.

`episode.yaml`

episode:
  number: 10
  title: Kotlin Language Design Nitpicking with Dmitry Jemerov from JetBrains
  hosts:
    - *hannes.dorfmann
    - *artem.zinnatullin
  guests:
    - *dmitry.jemerov
  links:
    - title: Book: Kotlin in Action written by Dmitry Jemerov and Svetlana Isakova
      url: https://www.manning.com/books/kotlin-in-action
    - title: Official Kotlin website with photo of lighthouse on Kotlin island
      url: http://kotlinlang.org/

`people.yaml`

# Not sure about the YAML anchor syntax here.

people:
  - &hannes.dorfmann
    name: Hannes Dorfmann
    twitter: sockeqwe
    blog: http://hannesdorfmann.com
    github: sockeqwe
  - &artem.zinnatullin
    name: Artem Zinnatullin
    twitter: artem_zin
    blog: https://artemzin.com/blog
    github: artem-zinnatullin

Output

RSS

<item>
    <guid>https://artemzin.com/static/thecontext/episodes/The.Context.episode.11.mp3</guid>
    <title>Episode 11: Migration to RxJava 2 with Artur Dryomov from Juno</title>
    <link>https://github.com/artem-zinnatullin/TheContext-Podcast/blob/master/show_notes/Episode_11.md</link>
    <atom:link rel="replies" type="text/html" href="https://github.com/artem-zinnatullin/TheContext-Podcast/issues/71"/>
    <itunes:duration>52:46</itunes:duration>
    <pubDate>Mon, 15 May 2017 09:30:00 EST</pubDate>
    <enclosure url="https://artemzin.com/static/thecontext/episodes/The.Context.episode.11.mp3" length="50658006" type="audio/mpeg"/>

    <atom:contributor>
        <atom:name>Artur Dryomov</atom:name>
        <atom:uri>https://github.com/ming13</atom:uri>
    </atom:contributor>

    <itunes:summary>
        <![CDATA[

        <h1>Episode 11: Migration to RxJava 2 with Artur Dryomov from Juno</h1>

        <h3><a href="https://github.com/artem-zinnatullin/TheContext-Podcast">How to listen &amp; subscribe</a></h3>

        <ul>
            <li><a href="https://github.com/artem-zinnatullin/TheContext-Podcast/issues/71">Discussion after the episode</a></li>
        </ul>

        <p>We've talked to Artur about Juno's way to migrate their Android Riders App from RxJava 1 to RxJava 2. We have covered:</p>

        <ul>
            <li>Migration strategy at Juno</li>
            <li>What's different in RxJava 2.0?</li>
            <li>Null values in RxJava 2</li>
            <li>Flowable or Observable?</li>
            <li>Other Rx Types: Single, Maybe and Completable</li>
            <li>RxJava 2 interop library</li>
            <li>RxJava 2: Default UncaughtExceptionHandler logs exceptions silently in JVM unit tests</li>
            <li>RxJava 3</li>
        </ul>

        <h4>Guest:</h4>

        <ul>
            <li>Artur Dryomov <a href="https://github.com/ming13">GitHub</a></li>
        </ul>

        <h4>Hosts:</h4>

        <ul>
            <li>Hannes Dorfmann <a href="https://twitter.com/sockeqwe">@sockeqwe</a>, <a href="http://hannesdorfmann.com">personal blog</a>, <a href="https://github.com/sockeqwe">GitHub</a></li>
            <li>Artem Zinnatullin <a href="https://twitter.com/artem_zin">@artem_zin</a>, <a href="http://artemzin.com">personal blog</a>, <a href="https://github.com/artem-zinnatullin">GitHub</a></li>
        </ul>

        <h4>Additional links:</h4>

        <ul>
            <li><a href="https://github.com/gojuno/engineering/blob/master/specs/rxjava-2-migration.md">Juno's Android Rider Team RxJava 2 Migration Specification</a></li>
            <li><a href="https://github.com/ReactiveX/RxKotlin">RxKotlin</a></li>
            <li><a href="https://github.com/akarnokd/RxJava2Interop">RxJava 2 interop library</a></li>
            <li><a href="https://github.com/ReactiveX/RxJava/wiki/What%27s-different-in-2.0">What's different in RxJava 2.0</a></li>
            <li><a href="https://github.com/ReactiveX/RxJava/wiki/What%27s-different-in-2.0#which-type-to-use">RxJava 2: Which type to use</a></li>
            <li><a href="https://github.com/ReactiveX/RxJava/issues/5234">RxJava 2: Uncaught errors fail silently in junit tests</a></li>
            <li><a href="https://github.com/ReactiveX/RxJava/issues/4644#issuecomment-256684743">RxJava 2: Discussion about null values</a></li>
            <li><a href="https://github.com/reactive-streams/reactive-streams-jvm/issues/204">Reactive Streams specification: null value</a></li>
            <li><a href="https://github.com/ReactiveX/RxKotlin/issues/103">RxJava 2 Kotlin SAM issue combineLatest()/etc</a></li>
            <li><a href="https://github.com/ReactiveX/RxJava/issues/4564">Why RxJava 3: Split the library into two or adding types for symmetry</a></li>
            <li><a href="https://github.com/akarnokd/RxJava3-preview">RxJava 3 Preview</a></li>
        </ul>

        ]]>
    </itunes:summary>
</item>

Markdown

# Episode 11: Migration to RxJava 2 with Artur Dryomov from Juno

### [How to listen & subscribe](https://github.com/artem-zinnatullin/TheContext-Podcast)

 - [Discussion after the episode](https://github.com/artem-zinnatullin/TheContext-Podcast/issues/71)

We've talked to Artur about Juno's way to migrate their Android Riders App from RxJava 1 to RxJava 2. We have covered:

 - Migration strategy at Juno
 - What's different in RxJava 2.0?
 - Null values in RxJava 2
 - Flowable or Observable?
 - Other Rx Types: Single, Maybe and Completable
 - RxJava 2 interop library
 - RxJava 2: Default UncaughtExceptionHandler logs exceptions silently in jvm unit tests
 - RxJava 3

#### Guest:

 - Artur Dryomov [GitHub](https://github.com/ming13)

#### Hosts:

  - Hannes Dorfmann [@sockeqwe](https://twitter.com/sockeqwe), [personal blog](http://hannesdorfmann.com), [GitHub](https://github.com/sockeqwe)
  - Artem Zinnatullin [@artem_zin](https://twitter.com/artem_zin), [personal blog](http://artemzin.com), [GitHub](https://github.com/artem-zinnatullin)

#### Additional links:

  - [Juno's Android Rider Team RxJava 2 Migration Specification](https://github.com/gojuno/engineering/blob/master/specs/rxjava-2-migration.md)
  - [RxKotlin](https://github.com/ReactiveX/RxKotlin)
  - [RxJava 2 interop library](https://github.com/akarnokd/RxJava2Interop)
  - [What's different in RxJava 2.0](https://github.com/ReactiveX/RxJava/wiki/What%27s-different-in-2.0)
  - [RxJava 2: Which type to use](https://github.com/ReactiveX/RxJava/wiki/What%27s-different-in-2.0#which-type-to-use)
  - [RxJava 2: Uncaught errors fail silently in junit tests](https://github.com/ReactiveX/RxJava/issues/5234)
  - [RxJava 2: Discussion about null values](https://github.com/ReactiveX/RxJava/issues/4644#issuecomment-256684743)
  - [RxJava 2 Kotlin SAM issue combineLatest()/etc](https://github.com/ReactiveX/RxKotlin/issues/103)
  - [Reactive Streams specification: null value](https://github.com/reactive-streams/reactive-streams-jvm/issues/204)
  - [Why RxJava 3: Split the library into two or adding types for symmetry](https://github.com/ReactiveX/RxJava/issues/4564)
  - [RxJava 3 Preview](https://github.com/akarnokd/RxJava3-preview)

The Process

Validation

Before any generation steps it is possible to run some validations.

# podcaster.jar --validate

Check that links actually work.
Check optional and required fields.

Generation

YAML + Markdown → RSS

Required for various podcast apps.

$ podcaster.jar --create-rss

I think it is pretty safe to create Markdown from YAML episode declaration and Markdown shownotes, then render the result as HTML and put to the <itunes:summary> section.
This can be combined with other properties and the resulting RSS can be done.

YAML + Markdown → Markdown

Required for the Jekyll-based website.

$ podcaster.jar --create-markdown

Probably will just repeat the <itunes:summary> step from RSS generation.

Workflow

PR

After opening a PR to the episodes repository the CI is being invoked.
Validation is being invoked and fails the build if something is wrong.
Generation is being invoked and posts resulting Markdown and RSS to the PR as a comment using GitHub API.

Publishing

Probably will be invoked manually just as on CI.
The website repository will store generated Markdown files.
RSS probably should be posted to a hosting.

artem-zinnatullin commented 7 years ago

Can you please give example of episode yml with complex description that uses lists and links?

You can use some real one https://github.com/artem-zinnatullin/TheContext-Podcast/blob/master/show_notes/Episode_11.md

sockeqwe commented 7 years ago

I still like pretty much the idea of having a preview of the (generated) show note (markdown) posted back from CI to github pull request as comment so that reviewer can review rendered markdown.

Therefore podcaster.jar needs to know which files has been changed by the pull request to only post back the preview of the changed files / episode (travis provide the sha of the commits tgatbhasbchabged by this pull request). Hence, I wouod like to suggest to create only 1 yml file which includes meta data and show notes markdown (not a separte markdown file) because then its easier to determine the changes.

Proposal for episode.yml file

episode:
  number: 10
  title: Kotlin Language Design Nitpicking with Dmitry Jemerov from JetBrains
  hosts:
    - *hannes.dorfmann
    - *artem.zinnatullin
  guests:
    - *dmitry.jemerov
  links:
    - title: Book: Kotlin in Action written by Dmitry Jemerov and Svetlana Isakova
      url: https://www.manning.com/books/kotlin-in-action
    - title: Official Kotlin website with photo of lighthouse on Kotlin island
      url: http://kotlinlang.org/

---

Show notes **markdown** section

arturdryomov commented 7 years ago

@artem-zinnatullin,

Can you please give example of episode yml with complex description that uses lists and links?

The idea was to store just links in episode.yml and a regular text with lists and such in a separate Markdown file so the formatting can be whatever you need. @sockeqwe wanted to keep links in episode.yml so they can be validated without parsing Markdown if I remember correctly.

@sockeqwe,

I still like pretty much the idea of having a preview of the (generated) show note (markdown) posted back from CI to github pull request as comment so that reviewer can review rendered markdown.

That’s mentioned in the Workflow#PR section, but yeah, the workflow is not clear enough.

Putting shownotes directly into episode.yaml has some disadvantages.

We are kind of changing YAML format. It does support multiline strings, but syntax is meh.
We are losing an ability to preview Markdown shownotes in editors or just on GitHub.
If we’re removing subdirectories, we cannot scale to another (possible) content.

I guess it is possible to determine which files were changed. We can pick subdirectories from these files and generate only changed ones. podcaster.jar can accept a list of directories to do the job.

I’m a little bit worried about comment spam on multiple changes in an episode though, but it is manageable.

sockeqwe commented 7 years ago

@sockeqwe wanted to keep links in episode.yml so they can be validated without parsing Markdown if I remember correctly.

actually its not because of parsing links (we do should parse the markdown for links too and check if they link properly). Actually i would like to have the layout of the full show notes file consistence across all episodes. So the idea is that the layouting things etc should not be the responsibility of the creator of the show notes md file.

expamlple (I have added a note which is generated from "meta data"):

# Episode 11: Migration to RxJava 2 with Artur Dryomov from Juno (generated from meta data)

### [How to listen & subscribe](https://github.com/artem-zinnatullin/TheContext-Podcast) (generated from meta data)

 - [Discussion after the episode](https://github.com/artem-zinnatullin/TheContext-Podcast/issues/71) (generated from meta data)

--- Begin section of show notes provided by creator of show notes ---
We've talked to Artur about Juno's way to migrate their Android Riders App from RxJava 1 to RxJava 2. We have covered:

 - Migration strategy at Juno
 - What's different in RxJava 2.0?
 - Null values in RxJava 2
 - Flowable or Observable?
 - Other Rx Types: Single, Maybe and Completable
 - RxJava 2 interop library
 - RxJava 2: Default UncaughtExceptionHandler logs exceptions silently in jvm unit tests
 - RxJava 3

--- end section ---

#### Guest: (generated from meta data)

 - Artur Dryomov [GitHub](https://github.com/ming13)

#### Hosts: (generated from meta data)

  - Hannes Dorfmann [@sockeqwe](https://twitter.com/sockeqwe), [personal blog](http://hannesdorfmann.com), [GitHub](https://github.com/sockeqwe)
  - Artem Zinnatullin [@artem_zin](https://twitter.com/artem_zin), [personal blog](http://artemzin.com), [GitHub](https://github.com/artem-zinnatullin)

#### Additional links: (generated from meta data)

  - [Juno's Android Rider Team RxJava 2 Migration Specification](https://github.com/gojuno/engineering/blob/master/specs/rxjava-2-migration.md)
  - [RxKotlin](https://github.com/ReactiveX/RxKotlin)
  - [RxJava 2 interop library](https://github.com/akarnokd/RxJava2Interop)
  - [What's different in RxJava 2.0](https://github.com/ReactiveX/RxJava/wiki/What%27s-different-in-2.0)
  - [RxJava 2: Which type to use](https://github.com/ReactiveX/RxJava/wiki/What%27s-different-in-2.0#which-type-to-use)
  - [RxJava 2: Uncaught errors fail silently in junit tests](https://github.com/ReactiveX/RxJava/issues/5234)
  - [RxJava 2: Discussion about null values](https://github.com/ReactiveX/RxJava/issues/4644#issuecomment-256684743)
  - [RxJava 2 Kotlin SAM issue combineLatest()/etc](https://github.com/ReactiveX/RxKotlin/issues/103)
  - [Reactive Streams specification: null value](https://github.com/reactive-streams/reactive-streams-jvm/issues/204)
  - [Why RxJava 3: Split the library into two or adding types for symmetry](https://github.com/ReactiveX/RxJava/issues/4564)
  - [RxJava 3 Preview](https://github.com/akarnokd/RxJava3-preview)

@ming13 sorry, i have overlooked that in your proposal. 👍

That’s mentioned in the Workflow#PR section, but yeah, the workflow is not clear enough.

We can pick subdirectories from these files and generate only changed ones. podcaster.jar can accept a list of directories to do the job.

I'm ok with that, but then it requires strong conventions. Example: episode.yaml, shownotes.md otherwise we can't map a change in shownotes.md to the corresponding .yml file.

But, I'm fine with that.

Also a thing that is missing in your proposal is that ideally CI creates a "discussion after show notes issue", add a git tag for the episode and uploads the mp3 file into the release section of target repository https://github.com/artem-zinnatullin/TheContext-Podcast

sockeqwe commented 6 years ago

As discussed in #4 it is not fully clear what requirements we have for this podcast-ci tool.

Requirements

[ ] Generate Show Notes files in markdown
[ ] Validate Show Notes and links like url's, twitter handles of persons and so on
[ ] Generate RSS feed
[ ] Validate RSS feed

Nice to have

Not a strong requirement but nice to have:

[ ] Generate website (jakyll) with show notes and optional a web player to play the mp3 file directly from website. Why do we need a website? Mainly to provide a nice link that can be shared / submitted on reddit, android weekly, twitter and so on.
[ ] During the validation phase (see Workflow and usage section) it would be nice to see a full show notes preview (generated from episode.yaml and shownotes.md) to the corresponding pull request. This means, that a reviewer of a pull request for a new episode can simply review the whole generated and formatted preview of the show notes (comfortable to check for spelling, clicking on links and so on), rather than "raw" episode.yaml and shownotes.md files.
[ ] Use github api to create an issue in artem-zinnatullin/TheContext-Podcast to create an issue for "discussion after the show", add git tag and upload mp3 file into release section.
[ ] Spell checker for show notes.

Workflow and usage:

We have a dedicated repository TheContext/episodes where we put people.yml and episode.yml files as described by @ming13 in #1

├── episodes
│   ├── 11-migration-to-rxjava-2
│   │   ├── episode.yaml
│   │   └── shownotes.md
│   └── 12-instant-apps
│       ├── episode.yaml
│       └── shownotes.md
└── people.yaml

To add (or change) an episode a host must create a new pull request in TheContext/episodes with episode.yaml and shownotes.md file as described in above. On every pull request we run our ci tool (called podcaster.jar) in "validation mode" like $ podcaster.jar --validate. As the name already suggests "validation mode" just validates but doesn't generate anything.
On every git push event on master branch (i.e. once a pull request to add a new episode is merged into master) we run our ci tool in "generation mode": $ podcaster.jar --create-rss --create-markdown --create-website. This will actually generate rss feed (xml) and a single markdown show note file for each episode and website jakyll stuff.
The files (rss.xml and single markdown show notes file) generated in step number 2 (see above) are "deployed" (by using git) to artem-zinnatullin/TheContext-Podcast. Website (generated jakyll stuff) is "depolyed" to TheContext/TheContext.github.io (by using git). Optionally (see nice to have section) we use github api to create an issue for "discussion after the show", add git tag and upload mp3 file into release section in artem-zinnatullin/TheContext-Podcast.

arturdryomov commented 5 years ago

More or less implemented, at least from the development perspective.

TheContext / podcaster

Design #1