Improve TWiN consistency

miversen33 commented 8 months ago

There seems to be a highly inconsistent schedule on when those are posted (which I thought was automated?) and the last one was 3ish weeks ago.

codicocodes commented 7 months ago

I have been on vacation for the past couple of weeks, and expect to release one next week.

The schedule has definitely been inconsistent. They are not automatically generated and take several hours to curate each one at the moment.

I have been aiming for releasing one every 2 weeks, but have not really hit that target consistently.

I could move to more auto generation and would improve consistency. But I am worried the quality would suffer.

codicocodes commented 7 months ago

I am curious to understand if consistency and how often it is released is valued higher than spending time into improving the quality by manually sourcing images, texts, plugins and core news. If it's not valued I could auto generate a version which in my opinion would be much lower quality.

miversen33 commented 7 months ago

My personal opinion is that I use TWIN to see what new interesting plugins have come out. The rest of the fluff is nice but IMO I like to see what new things people are doing. "This Week in Neovim" isn't really "This Week" if its "whenever" ;)

But I also am not the one maintaining it and I don't know just how hard it is to create each entry. I would definitely be open to helping setup the automation behind it if that doesn't already exist though. Its been something I have been thinking about for a while anyway.

Please note, I am certainly not asking you to exert extra work just for me. I am more asking about the reliability of TWIN as I was under the impression that dotfyle was taking it over to ensure it stayed alive.

I do not know if my opinion is one that others share or not, please don't make a decision based only on my input.

codicocodes commented 7 months ago

My personal opinion is that I use TWIN to see what new interesting plugins have come out. The rest of the fluff is nice but IMO I like to see what new things people are doing. "This Week in Neovim" isn't really "This Week" if its "whenever" ;)

I hear you, and I definitely feel bad that I haven't been able to keep consistency to the level I want and the community expects - it's been a challenge! I definitely hope and intend to keep TWiN alive, and if the community values higher consistency I will try to improve it.

But I also am not the one maintaining it and I don't know just how hard it is to create each entry. I would definitely be open to helping setup the automation behind it if that doesn't already exist though. Its been something I have been thinking about for a while anyway.

There is some automation on the roadmap I haven't had the time to implement, and certainly help to implement these would be very appreciated.

Automatically parsing https://github.com/neovim/neovim/commits/master/runtime/doc/news.txt.atomfor Neovim core updates
Improve image parsing to reduce time spent manually sourcing images
Detect reddit posts about plugins by scanning reddit (while staying under the new rate limit)
Add form for users to share news about a specific plugin with TWiN
Enable users to add new plugins with form (form exists but not available for all users)

Moving to more automatically generated issues will change the quality of the issues, but I think with these improvements the quality can be high enough to meet expectations, and could radically improve consistency by reducing manual work required.

Would you be interested in helping with any of the above points?

miversen33 commented 7 months ago

If you or someone is willing to help get me started, absolutely. The first point seems to be the most tangible for me as someone with no real experience with dotfyle's codebase.

I don't have any idea where I would go once I have created ingestion code for said atom file though and that is kind of where this will end up getting caught up. I was on the discord a while ago but am no longer there.

Basically, yes provided I get some training wheels to get me up and going with the codebase, dataset, etc :)

Edit: I will get started (at some point this week?) consuming the CONTRIBUTING.md file and see how far I can get on my own :)

codicocodes commented 7 months ago

I am happy to help you get started! Feel free to ping me on discord if you're having any issues, I'll try to layout a plan in the ticket and the other of the tickets as well.

alxndr commented 7 months ago

How about a "best of both worlds" approach — e.g. an RSS feed for "realtime" updates (automatic, low-touch), and then a monthly newsletter of images and videos (hand-built, high-touch)?

bassamsdata commented 7 months ago

What if we create a Markdown template that can be manually updated for new plugins or updates, and then automate parsing only this markdown file? Here are the benefits:

More contributors without the burden of understanding or setting up the dotfyle core.
Inclusion of non-JavaScript/TypeScript programmers.
Maintaining quality with a human touch, yet simplifying parsing by focusing on one file.

I believe automating Neovim core news is essential unless there's significant news.

From my end, I can contribute updates on new plugins or Reddit-related content, as I'm daily on Reddit. I can allocate specific times on two different days for updating the content.

miversen33 commented 7 months ago

If you or someone is willing to help get me started, absolutely. The first point seems to be the most tangible for me as someone with no real experience with dotfyle's codebase.

I don't have any idea where I would go once I have created ingestion code for said atom file though and that is kind of where this will end up getting caught up. I was on the discord a while ago but am no longer there.

Basically, yes provided I get some training wheels to get me up and going with the codebase, dataset, etc :)

Edit: I will get started (at some point this week?) consuming the CONTRIBUTING.md file and see how far I can get on my own :)

I have very basic processing of the atom file worked out (its just xml so really any xml parser will work) but I haven't had any time to work through figuring out how to feed the underlying dotfyle core API data (or really related to dotfyle itself) :(

codicocodes commented 7 months ago

If you or someone is willing to help get me started, absolutely. The first point seems to be the most tangible for me as someone with no real experience with dotfyle's codebase. I don't have any idea where I would go once I have created ingestion code for said atom file though and that is kind of where this will end up getting caught up. I was on the discord a while ago but am no longer there. Basically, yes provided I get some training wheels to get me up and going with the codebase, dataset, etc :) Edit: I will get started (at some point this week?) consuming the CONTRIBUTING.md file and see how far I can get on my own :)

I have very basic processing of the atom file worked out (its just xml so really any xml parser will work) but I haven't had any time to work through figuring out how to feed the underlying dotfyle core API data (or really related to dotfyle itself) :(

This file would be the relevant file to use the parser in: https://github.com/codicocodes/dotfyle/blob/main/src/lib/server/twin/builder.ts

We'd have to put a template into https://github.com/codicocodes/dotfyle/blob/main/twin/template.md as well.

Feel free to open a PR and i can add some pointers or some commits for it.