Improve TWiN consistency

[miversen33]

There seems to be a highly inconsistent schedule on when those are posted (which I thought was automated?) and the last one was 3ish weeks ago.

[codicocodes]

I have been on vacation for the past couple of weeks, and expect to release one next week.

The schedule has definitely been inconsistent. They are not automatically generated and take several hours to curate each one at the moment.

I have been aiming for releasing one every 2 weeks, but have not really hit that target consistently.

I could move to more auto generation and would improve consistency. But I am worried the quality would suffer.

[codicocodes]

I am curious to understand if consistency and how often it is released is valued higher than spending time into improving the quality by manually sourcing images, texts, plugins and core news. If it's not valued I could auto generate a version which in my opinion would be much lower quality.

[miversen33]

My personal opinion is that I use TWIN to see what new interesting plugins have come out. The rest of the fluff is nice but IMO I like to see what new things people are doing. "This Week in Neovim" isn't really "This Week" if its "whenever" ;)

But I also am not the one maintaining it and I don't know just how hard it is to create each entry. I would definitely be open to helping setup the automation behind it if that doesn't already exist though. Its been something I have been thinking about for a while anyway.

Please note, I am certainly not asking you to exert extra work just for me. I am more asking about the reliability of TWIN as I was under the impression that dotfyle was taking it over to ensure it stayed alive.

I do not know if my opinion is one that others share or not, please don't make a decision based only on my input.

[codicocodes]

My personal opinion is that I use TWIN to see what new interesting plugins have come out. The rest of the fluff is nice but IMO I like to see what new things people are doing. "This Week in Neovim" isn't really "This Week" if its "whenever" ;)

I hear you, and I definitely feel bad that I haven't been able to keep consistency to the level I want and the community expects - it's been a challenge! I definitely hope and intend to keep TWiN alive, and if the community values higher consistency I will try to improve it.

But I also am not the one maintaining it and I don't know just how hard it is to create each entry. I would definitely be open to helping setup the automation behind it if that doesn't already exist though. Its been something I have been thinking about for a while anyway.

There is some automation on the roadmap I haven't had the time to implement, and certainly help to implement these would be very appreciated.

• Automatically parsing https://github.com/neovim/neovim/commits/master/runtime/doc/news.txt.atomfor Neovim core updates
• Improve image parsing to reduce time spent manually sourcing images
• Detect reddit posts about plugins by scanning reddit (while staying under the new rate limit)
• Add form for users to share news about a specific plugin with TWiN
• Enable users to add new plugins with form (form exists but not available for all users)

Moving to more automatically generated issues will change the quality of the issues, but I think with these improvements the quality can be high enough to meet expectations, and could radically improve consistency by reducing manual work required.

Would you be interested in helping with any of the above points?

[miversen33]

If you or someone is willing to help get me started, absolutely. The first point seems to be the most tangible for me as someone with no real experience with dotfyle's codebase.

I don't have any idea where I would go once I have created ingestion code for said atom file though and that is kind of where this will end up getting caught up. I was on the discord a while ago but am no longer there.

Basically, yes provided I get some training wheels to get me up and going with the codebase, dataset, etc :)

Edit: I will get started (at some point this week?) consuming the CONTRIBUTING.md file and see how far I can get on my own :)

[codicocodes]

I am happy to help you get started! Feel free to ping me on discord if you're having any issues, I'll try to layout a plan in the ticket and the other of the tickets as well.

[alxndr]

How about a "best of both worlds" approach — e.g. an RSS feed for "realtime" updates (automatic, low-touch), and then a monthly newsletter of images and videos (hand-built, high-touch)?

[bassamsdata]

What if we create a Markdown template that can be manually updated for new plugins or updates, and then automate parsing only this markdown file? Here are the benefits:

1. More contributors without the burden of understanding or setting up the dotfyle core.
2. Inclusion of non-JavaScript/TypeScript programmers.
3. Maintaining quality with a human touch, yet simplifying parsing by focusing on one file.

I believe automating Neovim core news is essential unless there's significant news.

From my end, I can contribute updates on new plugins or Reddit-related content, as I'm daily on Reddit. I can allocate specific times on two different days for updating the content.

[miversen33]

If you or someone is willing to help get me started, absolutely. The first point seems to be the most tangible for me as someone with no real experience with dotfyle's codebase.

I don't have any idea where I would go once I have created ingestion code for said atom file though and that is kind of where this will end up getting caught up. I was on the discord a while ago but am no longer there.

Basically, yes provided I get some training wheels to get me up and going with the codebase, dataset, etc :)

Edit: I will get started (at some point this week?) consuming the CONTRIBUTING.md file and see how far I can get on my own :)

I have very basic processing of the atom file worked out (its just xml so really any xml parser will work) but I haven't had any time to work through figuring out how to feed the underlying dotfyle core API data (or really related to dotfyle itself) :(

[codicocodes]

If you or someone is willing to help get me started, absolutely. The first point seems to be the most tangible for me as someone with no real experience with dotfyle's codebase. I don't have any idea where I would go once I have created ingestion code for said atom file though and that is kind of where this will end up getting caught up. I was on the discord a while ago but am no longer there. Basically, yes provided I get some training wheels to get me up and going with the codebase, dataset, etc :) Edit: I will get started (at some point this week?) consuming the CONTRIBUTING.md file and see how far I can get on my own :)

I have very basic processing of the atom file worked out (its just xml so really any xml parser will work) but I haven't had any time to work through figuring out how to feed the underlying dotfyle core API data (or really related to dotfyle itself) :(

This file would be the relevant file to use the parser in: https://github.com/codicocodes/dotfyle/blob/main/src/lib/server/twin/builder.ts

We'd have to put a template into https://github.com/codicocodes/dotfyle/blob/main/twin/template.md as well.

Feel free to open a PR and i can add some pointers or some commits for it.

[codicocodes]

Automatically parsing of Neovim core updates has been merged. I think this will decrease the amount of time it takes to write TWiN by a lot :)

The next step here is probably to add a form for users to submit new plugins.

[codicocodes]

We now have a form where you can add plugins directly to Dotfyle:

https://dotfyle.com/neovim/plugins/add please take a look and let me know if you have any feedback!

[codicocodes]

I additionally did some image parsing improvements, I am hopeful maintaining TWiN will be a lot easier now :) So let's see if I can keep it up!

• [x] Automatically parsing https://github.com/neovim/neovim/commits/master/runtime/doc/news.txt.atomfor Neovim core updates
• [x] Improve image parsing to reduce time spent manually sourcing images
• [ ] Detect reddit posts about plugins by scanning reddit (while staying under the new rate limit)
• [ ] Add form for users to share news about a specific plugin with TWiN
• [x] Enable users to add new plugins with form (form exists but not available for all users)

@bassamsdata If you want to help contributing you are very welcome to help add plugins from Reddit via the form on https://dotfyle.com/neovim/plugins/add

I'll be adding a form for sharing news about a specific plugin later as well

[codicocodes]

How about a "best of both worlds" approach — e.g. an RSS feed for "realtime" updates (automatic, low-touch), and then a monthly newsletter of images and videos (hand-built, high-touch)?

@alxndr This is already available kind of available at: https://dotfyle.com/neovim/plugins/new

I'd consider an rss feed for new plugins too though

[codicocodes]

FYI there is now an RSS feed for new plugins at https://dotfyle.com/neovim/plugins/rss.xml

[alxndr]

@codicocodes

How about a "best of both worlds" approach — e.g. an RSS feed for "realtime" updates (automatic, low-touch), and then a monthly newsletter of images and videos (hand-built, high-touch)?

@alxndr This is already available kind of available at: https://dotfyle.com/neovim/plugins/new

I'd consider an rss feed for new plugins too though

FWIW I'm more interested in the curated approach than a raw feed of everything new.