Removal of feed endpoints

[ghost]

I have now fixed this, check out https://github.com/jonjomckay/fritter/pull/698

Github Releases Source Code

Comment on feeds

https://github.com/jonjomckay/fritter/issues/668#issuecomment-1528784190

Describe the bug I get the error type 'Null' is not a subtype of type 'String' in type cast when refreshing my feed

To Reproduce Steps to reproduce the behavior:

1. Go to 'Feed'
2. Click on 'Refresh'
3. See error

Screenshots https://user-images.githubusercontent.com/123662124/233856792-46a33b7b-c09d-47f1-9c06-25a1d773e3ed.png

Device Model: Moto G62 5G OS: Android 12 (API 31) Locale: en Version: 3.0.0-beta7+300000957 Flavor: Github

[moocow1452]

Same, it was working earlier for me after a clean install, now I have the same error.

Edit: Did some troubleshooting. Clearing Cache and Storage got it into a workable state again, but once the app was closed and I restarted, the error returned after a pull refresh.

[shayne3717]

I was able to get it working again by unchecking the "Show Replies" option in the overflow menu; it seems like unchecking either that or "Show Retweets" fixes the issue. Checking both causes the issue to return.

[moocow1452]

That's not a consistent fix on my side, error returns after a refresh, and no amount of toggling will bring the feed back.

[Undisputed00x]

Same thing here. Again app became unusable because of this bug.

[android-smiley]

Toggling "Show Retweets/Replies" does fix for a moment, but stops working if both are re-enabled at the same time. After some more toggling of both options it fails completely to load. The error message states type 'Null' is not a subtype of type 'String' in type cast (same as above).

The API access seems to be working now, I assume this is now client-side.

[JVT038]

I have the same issue.

[anton-isidore]

This happens practically immediately after closing the app.

I assume something bad gets saved into local storage that keeps crashing the app.

Sadly this bug renders the app practically unusable.

[hounsell]

Having a quick look into this issue, it looks like the issue is that the top cursor is missing in some stored chunks, which trips this up: https://github.com/jonjomckay/fritter/blob/69bdee9d35dbf332afaad455e4ace724e39f95c7/lib/group/_feed.dart#L148-L150

I've worked around this in my local copy just by adding in an additional null check for cursor_top before trying to cast it, like so:

if (latestChunk != null && latestChunk['cursor_top'] != null) {
    searchCursor = latestChunk['cursor_top'] as String;
} else {

And this does seem to get the app functioning again, but now some tweets in my feed get duplicated. I guess this is because the chunks are getting re-fetched, or at least another chunk is requested that contains some of the same tweets because the cursor wasn't in place to delineate where the existing chunk finished off. The proper fix is probably then to look at why some stored chunks are missing their top cursor.

[ghost]

Having a quick look into this issue, it looks like the issue is that the top cursor is missing in some stored chunks, which trips this up:

https://github.com/jonjomckay/fritter/blob/69bdee9d35dbf332afaad455e4ace724e39f95c7/lib/group/_feed.dart#L148-L150

I've worked around this in my local copy just by adding in an additional null check for cursor_top before trying to cast it, like so:

if (latestChunk != null && latestChunk['cursor_top'] != null) {
    searchCursor = latestChunk['cursor_top'] as String;
} else {

And this does seem to get the app functioning again, but now some tweets in my feed get duplicated. I guess this is because the chunks are getting re-fetched, or at least another chunk is requested that contains some of the same tweets because the cursor wasn't in place to delineate where the existing chunk finished off. The proper fix is probably then to look at why some stored chunks are missing their top cursor.

Thanks for this, I am using this while a fix can be integrated into the app

[j-fbriere]

This #679 PR should fix the problem.

[moocow1452]

Not sure if that also fixes this new issue. Screenshot_20230427-003347.png

[j-fbriere]

It doesn't fix the new issue that appeared today unfortunately. It seems that the graphql endpoint has been shut down. At least for the moment. Since a few hours ago, it returns a 404 error.

[jonjomckay]

I'm sorry to say it, but I think this is probably going to be the end of feed support in Fritter. It was always a fragile hack on top of the search API, and now it looks like Muskrat has finally removed support for searching publicly.

I don't see any way around this, besides logging in with an account, which is something Fritter won't support by design, due to the privacy concerns, and the point of Fritter is to browse Twitter as invisibly as possible.

Before people jump in and say "why can't you combine multiple profile tweet feeds into one, and use that?" - it's not feasible, unfortunately. Only 20 tweets can be fetched at a time, from a single profile, and there are now heavy rate limits in place for those endpoints. Even having a feed of 10 users who tweet semi-regularly would encounter constant rate limit errors, and be incredibly slow to load.

I'll shortly be removing the code that handles and displays feeds, and will be modifying groups to provide a list of accounts, rather than a feed view. I know that's going to be bad news for some of you, but I can't leave the functionality in and broken.

On the plus side, supporting the feed was always where the majority of my time was spent while maintaining Fritter, and it was always the most frustrating part to work on, due to its fragility. Not having to maintain it any more will allow me to better utilise the time I spend on Fritter to work on the other parts of the app 🙂

Thanks for understanding, and I hope you all continue to use and enjoy the app, until Elon decides to remove even more public endpoints 🙈

[Mrnofish]

@jonjomckay before you get rid of the feed altogether, have you taken a look at nitter ?

There is some talk about using unofficial tokens, which might be worth looking into.

Another alternative is operating a bunch of accounts to circumvent the search ban.

This latter approach, as I understand it, would work more or less like Aurora Store in anonymous mode. Not perfect, I know.

Since the nitter folks seem intent on not getting shut out, that project might be also an avenue to keep in mind.

It should already be possible to recreate the feed by fetching RSS feeds for individual accounts and then aggregating them. Actually, nitter already had a multi-user feed feature that could make it even easier, as long as they find a workaround.

I understand of course that even looking into any of this is a big ask, with Mr "Free" Speech at the helm it was always going to be an uphill fight, but I cannot think of a better way of flipping the bird, other than finding ways to keep using the service on our terms rather than head honcho's.

[jonjomckay]

Yep, I've been following the threads on the Nitter project, but the solutions won't fit for Fritter.

• Fritter was already using these "unofficial tokens" previously, and they no longer work for the search API, nor for the GraphQL endpoints, which both Nitter and Fritter moved to
• I simply don't have the time or financial resources to operate a service that rotates a bunch of (continously-banned or rate limited) accounts, and this would go against the privacy-first stance that Fritter stands for, by having to proxy all requests through a third-party service (even if it is operated by the me, the maintainer)
• Nitter's RSS feeds don't contain nearly enough information to populate Fritter's UI

I'm open to solutions, but I'm only able to commit a small amount of time to Fritter, so any solutions will need to be pretty much maintenance-free, which I don't hold much hope for.

[Mrnofish]

Perhaps I could've been clearer - what I meant to say is that a client app could also plug into a Nitter instance, leaving the task of managing the sacrificial accounts to the instances' admins.

Such app could rotate through several instances and ensure no single operator enjoys an exclusive gatekeeper position on the user's data, or allow the user to pick one or more instance that they trust.

There are several apps that have done something along the lines for other privacy-oriented frontends, covering e.g. Reddit, YouTube and other major services.

Again, not asking that you personally do something, I can certainly appreciate how much thankless work goes into even the simplest of projects.

You've done an excellent job with Fritter, and I've enjoyed using it - thank you so much for everything that you've done so far.

[geominorai]

Thanks for keeping up with this issue @jonjomckay despite its unfortunate conclusions.

If the restrictive rate limits make building a real-time feed impractical, perhaps a workaround and compromise would be to build a feed by caching tweets from individual profiles queried over time (with some configurable storage limit, e.g. 3 months) as if intending for offline viewing, and have the cache update when manually visiting a profile or during a global update. This global update of all subscribed profiles on the feed could be distributed over the course of the day as a background task for the app.

This is similar in spirit to some manga reader apps that scrape its content from third-party websites. Hammering a site all at once for updates and downloads on-demand would basically be DoS-ing it and risk getting throttled or blocked. Caching and spreading out the queries is much more friendly.

The trade-off, of course, would be the feed having profiles updated only at certain times a day, but for those of us not glued to Twitter and only check occasionally for a summary of updates during work breaks for example, this would suffice.

In this case, a feed would have to let new updates float to the top of the feed, not necessarily in chronological order with the "read" Tweets. Perhaps it can take some inspiration from how unread mail are grouped in an inbox for an email app.

[geominorai]

Adding to the above, caching Tweets also allows for estimating the profile's posting frequency. This allows for the global updates to be adaptive, such as checking once every few days for very infrequent (monthly/bi-monthly) tweeters while those more frequently posting could be checked once every few hours.

This leaves room for a possible priority updates category users may be able to configure for very important profiles like storm and earthquake alerts (if they have not already left Twitter).

[geominorai]

The advantage for this approach is that it is just piggybacking on the individual profile feeds being able to load. There is no extra endpoint to manage.

In the event there is downtime that requires an app update, the feed can try to slowly repopulate missed Tweets from the profile's last checked timestamp once the user installs the updated apk. If there are too many posts to load at a time, it may take a few global updates to finish fetching everything, but users should be understanding if informed about throttling due to rate limits.

[summersab]

@jonjomckay - since the feed doesn't (and likely won't) work, would it be possible to trigger a web search intent for the trending topic/hashtag in the default browser when an item is clicked? That way, users can at least figure out why something is trending - that's good enough for me.

It would be great if you could provide the option to open the search in the default browser or in a WebView tab within Fritter, but that functionality isn't as important - it would be nice to have to avoid having to switch apps.

Thank you for your hard work!

[eye-writ]

Don't know why, but the feed function is working for me this morning.

Version v3.0.0-beta7+300000957

[ghost]

My feed is also working now on beta7

On Thu, 4 May 2023, 07:24 eye-writ, @.***> wrote:

Don't know why, but the feed function is working for me this morning.

Version v3.0.0-beta7+300000957

— Reply to this email directly, view it on GitHub https://github.com/jonjomckay/fritter/issues/668#issuecomment-1534151834, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5PO6LGZX3UOQK5XGNBUV4LXENDRZANCNFSM6AAAAAAXIVXB3Y . You are receiving this because you authored the thread.Message ID: @.***>

[Undisputed00x]

@TheHCJ @eye-writ yeah until error shows up , thats the case on my part

[eye-writ]

@TheHCJ @eye-writ yeah until error shows up , thats the case on my part

After a while I get the error too, but the old workaround of clearing the app's storage data still works.

[ghost]

I am working on getting the feed and tweet search working again, feel free to check out the repo below Repo

[Undisputed00x]

So far so good in terms of functionality but I'm experiencing some really annoying freezes and lag while scrolling the feed

[ghost]

Going to fix that later today 😃

[eye-writ]

I am working on getting the feed and tweet search working again, feel free to check out the repo below Repo

Sorry to be a divvy, but how do I install your fix? 😕

[ha-gorem]

I am working on getting the feed and tweet search working again, feel free to check out the repo below Repo

Sorry to be a divvy, but how do I install your fix? confused

from here https://github.com/TheHCJ/fritter/releases

[ghost]

Download here: https://github.com/TheHCJ/fritter/releases

[eye-writ]

Thanks. Ha-gorem kindly led me by the hand and it's currently working well. 👍🏻

[Undisputed00x]

@TheHCJ Found a bug , if default tab is set from blank to Feed , tweets get duplicated

[ghost]

I am aware and working on a fix! 😃

On Fri, 5 May 2023, 16:07 AndiDink00x, @.***> wrote:

@TheHCJ https://github.com/TheHCJ Found a bug , if default tab is set from blank to Feed , tweets get duplicated

— Reply to this email directly, view it on GitHub https://github.com/jonjomckay/fritter/issues/668#issuecomment-1536400126, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5PO6LH7ZFYZ36UXYBG4KFLXEUJSVANCNFSM6AAAAAAXIVXB3Y . You are receiving this because you were mentioned.Message ID: @.***>

[moocow1452]

No releases on your repo and the APK link you gave 404d on me. Thanks for your contributions.

[ghost]

I have found a fix on this repo here APK

[Undisputed00x]

@TheHCJ thats a new pipe apk

[ghost]

@TheHCJ thats a new pipe apk

Shoot, wrong apk :0

[Undisputed00x]

@TheHCJ Damn dude just installed and the app is flawless. Saved the whole app and probably made thousands of its users happy and able to use Fritter again.

[eye-writ]

It's a lifesaver of an update from HCJ, but continued love for Jonjo too. Not easy when the Tesla nonce throws a spanner in the works!

🔧

[ghost]

I will try to get a pull request in tomorrow to fix this in the main repo

On Fri, 5 May 2023, 20:16 eye-writ, @.***> wrote:

It's a lifesaver of an update from HCJ, but continued love for Jonjo too. Not easy when the Tesla nonce throws a spanner in the works! 🔧

— Reply to this email directly, view it on GitHub https://github.com/jonjomckay/fritter/issues/668#issuecomment-1536670559, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5PO6LCOI75JZ2SVMBLKUXLXEVGZLANCNFSM6AAAAAAXIVXB3Y . You are receiving this because you were mentioned.Message ID: @.***>

[Igaru22]

Crashes on open

[ghost]

Crashes on open

Seems the api has broke again, I feel like feeds cannot be fixed until Elon opens up the api or Fritter starts using it's own way of getting data off twitter.

[eye-writ]

I'm probably speaking too soon, but it's still working fine for me.

[Undisputed00x]

@TheHCJ still works pretty good

[Igaru22]

Yesterday it worked on my Android 11 device but crashed on open on my Android 5 tablet. Today neither works

[Undisputed00x]

@Igaru22 did you download the correct apk from @TheHCJ ?

[moocow1452]

Beta 7 has been working fine for me with regular storage clears. I haven't been able to access HCJ's build yet.

On Sat, May 6, 2023, 3:22 PM AndiDink00x @.***> wrote:

@Igaru22 https://github.com/Igaru22 did you download the correct apk from @TheHCJ https://github.com/TheHCJ ?

— Reply to this email directly, view it on GitHub https://github.com/jonjomckay/fritter/issues/668#issuecomment-1537206880, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKRYGAS6TXUS2VOCQLL5XLXE2QILANCNFSM6AAAAAAXIVXB3Y . You are receiving this because you commented.Message ID: @.***>

[ghost]

Just made a pull request to fix this, I will attach an apk soon! https://github.com/jonjomckay/fritter/pull/698

EDIT: Github Release

[Igaru22]

@Igaru22 did you download the correct apk from @TheHCJ ?

Yes I tried them all. The Android 5 does not seem to work on any 3.x version. The Android 11 worked with Beta 7 for a while. I'm about to check out the new fix

[ghost]

Just added an updated apk in the description of this issue! 😀