stuck in "Loading.." when opening rss feed

kibadi commented 4 years ago

Hi,

congratulations to your app, it`s great, especially to read full texts which was not possible with other apps. I found an issue with one feed, the app remains in "Loading.." state for minutes (probably forever) when trying to open it. Could it be that the feed url breaks the parser due to the %5B and %5D chars ("[" and "]") in the url? Or could it be related to the response? the response is the menu plan of a college dining hall possibly containing umlauts like ä,ü,ö,ß.

https://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum/?mid=37&tx_akafoespeiseplan_mensadetails%5Baction%5D=feed&tx_akafoespeiseplan_mensadetails%5Bcontroller%5D=RSSFeed&cHash=00668abfcb211d080183ffcf28623873

Thank you and best regards, Stefan

Wowfunhappy commented 4 years ago

☹️

I'd really like to fix this, but it's not my code that's causing the problem. The app currently uses the rss2json api to retrieve data from a feed in a Javascript-friendly format. Go to rss2json.com, paste your feed into the big text box, and watch what happens. It's not good.

The only way to resolve this is to stop using rss2json altogether and process the xml ourselves. Which is in fact something I would like to do (see #6), but my previous attempts have all hit a brick wall. If you feel so inclined, you could try using rss2json's contact link to let them know the feed doesn't work on their site (no need to mention my app), but since we're not paying customers I'm not sure they'll care.

As an entirely unhelpful aside, my app has error handling, but for obvious reasons it can't do anything unless rss2json actually tells us there's an error. Why the heck is rss2json not returning http status 4xx like it's supposed to?

Wowfunhappy commented 4 years ago

Hang on a moment... when I go to that link in my browser I get a 404 error! 😄 Are you sure the url is right?

kibadi commented 4 years ago

Hang on a moment... when I go to that link in my browser I get a 404 error! 😄 Are you sure the url is right?

Thank you for responding immediately :)

sorry, I didn't check the rss url again in the browser, it was probably outdated due to the last hash part. I tried this url

https://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37&tx_akafoespeiseplan_mensadetails[action]=feed&tx_akafoespeiseplan_mensadetails[controller]=RSSFeed

in rss2json.com and it converts to the correct json. If you paste the latter url in the browser you can download the application+xml file for the feed. Unfortunately the latter url still doesn`t work in the app.

I will try to raise this issue with rss2json.com as you suggested. Otherwise I keep an eye on the open issue #6 that you're on.

kibadi commented 4 years ago

{ "status": "ok", "feed": { "url": "https://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37&tx_akafoespeiseplan_mensadetails[action]=feed&tx_akafoespeiseplan_mensadetails[controller]=RSSFeed", "title": "Speiseplan Ruhr Universität Bochum - Bistro", "link": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum", "author": "", "description": "Speiseplan der Ruhr Universität Bochum - Bistro im RSS-Format.", "image": "" }, "items": [ { "title": "Speiseplan für Montag, den 21.10., an der Ausgabestelle 'Ruhr Universität Bochum - Bistro'", "pubDate": "2019-10-24 15:55:40", "link": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-21", "guid": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-21", "author": "", "thumbnail": "", "description": "\nEssen 1\n <ul>\n<li>Käsespätzle mit Kochschinken (S) (2,3,8) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Gemüsefrikadelle mit Klarem Kartoffelsalat (V) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Lammgulasch Bohnengemüse Salzkartoffeln (L) (1) 3,80 EUR - 5,00 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Bohnengemüse (V) (1) 0,80 EUR - 0,90 EUR</li>\n <li>Salzkartoffeln (VG) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "content": "\nEssen 1\n <ul>\n<li>Käsespätzle mit Kochschinken (S) (2,3,8) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Gemüsefrikadelle mit Klarem Kartoffelsalat (V) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Lammgulasch Bohnengemüse Salzkartoffeln (L) (1) 3,80 EUR - 5,00 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Bohnengemüse (V) (1) 0,80 EUR - 0,90 EUR</li>\n <li>Salzkartoffeln (VG) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "enclosure": {}, "categories": [] }, { "title": "Speiseplan für Dienstag, den 22.10., an der Ausgabestelle 'Ruhr Universität Bochum - Bistro'", "pubDate": "2019-10-24 15:55:39", "link": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-22", "guid": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-22", "author": "", "thumbnail": "", "description": "\nEssen 1\n <ul>\n<li>Kasslerbraten (S) (2,3) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Vegetarische Pizza (VG) (2,3) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Hähnchenbrust natur Paprika- Ananas Sauce Kokosreis und einem gemischten Blattsalat (G) 3,80 EUR - 5,00 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Sauerkraut (VG) (1) 0,80 EUR - 0,90 EUR</li>\n <li>Kartoffelpüree (V) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "content": "\nEssen 1\n <ul>\n<li>Kasslerbraten (S) (2,3) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Vegetarische Pizza (VG) (2,3) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Hähnchenbrust natur Paprika- Ananas Sauce Kokosreis und einem gemischten Blattsalat (G) 3,80 EUR - 5,00 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Sauerkraut (VG) (1) 0,80 EUR - 0,90 EUR</li>\n <li>Kartoffelpüree (V) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "enclosure": {}, "categories": [] }, { "title": "Speiseplan für Mittwoch, den 23.10., an der Ausgabestelle 'Ruhr Universität Bochum - Bistro'", "pubDate": "2019-10-24 15:55:39", "link": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-23", "guid": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-23", "author": "", "thumbnail": "", "description": "\nEssen 1\n <ul>\n<li>Kassslerbraten (S) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Mediterrane Pennepfanne (VG) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Kichererbsenbällchen mit Kräuterquark, Couscous Salat, Klarer Chinakohlsalat (V) 3,80 EUR - 5,00 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Rotkohl (VG) (3,9) 0,80 EUR - 0,90 EUR</li>\n <li>Semmelknödel (V) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "content": "\nEssen 1\n <ul>\n<li>Kassslerbraten (S) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Mediterrane Pennepfanne (VG) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Kichererbsenbällchen mit Kräuterquark, Couscous Salat, Klarer Chinakohlsalat (V) 3,80 EUR - 5,00 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Rotkohl (VG) (3,9) 0,80 EUR - 0,90 EUR</li>\n <li>Semmelknödel (V) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "enclosure": {}, "categories": [] }, { "title": "Speiseplan für Donnerstag, den 24.10., an der Ausgabestelle 'Ruhr Universität Bochum - Bistro'", "pubDate": "2019-10-24 15:55:40", "link": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-24", "guid": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-24", "author": "", "thumbnail": "", "description": "\nEssen 1\n <ul>\n<li>Schweinenackensteak Schmorzwiebeln (S) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Gegrillte Aubergine mit Gurken-Minz-Joghurt und Fladenbrot (A,V) (2,9) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Vollkornnudeln-Zucchini-Paprika-Auflauf, Paprikasalat (V) 3,00 EUR - 4,10 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Kaisergemüse (VG) 0,80 EUR - 0,90 EUR</li>\n <li>Kartoffelecken (V) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "content": "\nEssen 1\n <ul>\n<li>Schweinenackensteak Schmorzwiebeln (S) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Gegrillte Aubergine mit Gurken-Minz-Joghurt und Fladenbrot (A,V) (2,9) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Vollkornnudeln-Zucchini-Paprika-Auflauf, Paprikasalat (V) 3,00 EUR - 4,10 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Kaisergemüse (VG) 0,80 EUR - 0,90 EUR</li>\n <li>Kartoffelecken (V) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "enclosure": {}, "categories": [] }, { "title": "Speiseplan für Freitag, den 25.10., an der Ausgabestelle 'Ruhr Universität Bochum - Bistro'", "pubDate": "2019-10-24 15:55:39", "link": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-25", "guid": "http://www.akafoe.de/gastronomie/speiseplaene-der-mensen/bistro-der-ruhr-universitaet-bochum?mid=37/19-10-25", "author": "", "thumbnail": "", "description": "\nEssen 1\n <ul>\n<li>\"Chili Con Carne\" mit Baguette (R) (2,3) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Germknödel mit Pflaumenmusfüllung Vanillesauce (V) (1) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Seelachs im Backteig, Fingerkarotten, Salzkartoffeln (F) 3,80 EUR - 5,00 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Fingerkarotten (VG) 0,80 EUR - 0,90 EUR</li>\n <li>Salzkartoffeln (VG) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "content": "\nEssen 1\n <ul>\n<li>\"Chili Con Carne\" mit Baguette (R) (2,3) 2,20 EUR - 3,20 EUR</li>\n </ul>\nAktion\n <ul></ul>\nEssen 2\n <ul>\n<li>Germknödel mit Pflaumenmusfüllung Vanillesauce (V) (1) 2,20 EUR - 3,20 EUR</li>\n </ul>\nSTUDY&FIT\n <ul>\n<li>Seelachs im Backteig, Fingerkarotten, Salzkartoffeln (F) 3,80 EUR - 5,00 EUR</li>\n </ul>\nBeilagen\n <ul>\n<li>Fingerkarotten (VG) 0,80 EUR - 0,90 EUR</li>\n <li>Salzkartoffeln (VG) 0,80 EUR - 0,90 EUR</li>\n </ul>\n", "enclosure": {}, "categories": [] } ] }

this is the json that rss2json produces for the feed; could it be that loading the feed takes minutes? i should maybe give it more time to load?

thanks, Stefan

Wowfunhappy commented 4 years ago

Don’t bring it up with rss2json if it works on their end!

Okay, this is good, it means I can fix it. I don’t know when I’ll next have a chance, but it’s on my list!

It would help a great deal if there’s a stable url.

kibadi commented 4 years ago

Thank you for looking into it. The url should be stable (delivers an application+xml document for the current week's menu).

kibadi commented 4 years ago

Hi, I looked into your app.js and it seems that the feed.url is passed "as is" to api.rss2json.com, without html encoding. I re-tested the rss url directly in rss2json.com and there the feed url gets html-encoded and delivers correct json.

This is the original rss feed for example:

https://www.akafoe.de/gastronomie/speiseplaene-der-mensen/ruhr-universitaet-bochum?mid=1&tx_akafoespeiseplan_mensadetails%5Baction%5D=feed&tx_akafoespeiseplan_mensadetails%5Bcontroller%5D=RSSFeed&cHash=a185a06e54f18d5d021625e0f336c7bf

If I test this rss feed on the website rss2json.com the API call is displayed as:

https://api.rss2json.com/v1/api.json?rss_url=https%3A%2F%2Fwww.akafoe.de%2Fgastronomie%2Fspeiseplaene-der-mensen%2Fruhr-universitaet-bochum%3Fmid%3D1%26tx_akafoespeiseplan_mensadetails%255Baction%255D%3Dfeed%26tx_akafoespeiseplan_mensadetails%255Bcontroller%255D%3DRSSFeed%26cHash%3Da185a06e54f18d5d021625e0f336c7bf

In the app.js the url seems to not get encoded (at least I don't see that it gets encoded). Could that be a reason that this rss feed shows no results on the Pebble?

Thank you for looking into it. Stefan

Wowfunhappy commented 4 years ago

Thank you for the research. Yes, that sounds likely.

I unfortunately still don't know when I'll next have a chance to look at this. :( It will happen eventually.

If you wanted to submit a pull request, that would be great. You're probably on the right track, I don't think I'm doing any encoding/decoding, except for when I load the data from Settings.

kibadi commented 4 years ago

Thank you for your instant response and your confirmation that the missing url encoding might be the missing point here. As I can modify and replace the app.js in the pbw file as I learned some time ago, I can try to fix it myself and add the missing encoding and test it by sideloading the modified pbw to my pebble 2. If it works for me, I would like to make a small contribution and submit it as a pull request,

Wowfunhappy commented 4 years ago

You're very welcome, I'm sorry I haven't gotten around to fixing this myself after three months!

Just a heads up, while you can absolutely edit the code that way, the copy of app.js in the compiled pbw has a lot of extra PebbleJS boilerplate that isn't present in the original source code. I do recommend trying to get the Pebble SDK set up if you can. Testing changes becomes much quicker that way, and you can log statements to the console, which for me was invaluable.

Regardless, the function you want to look at is getArticles(feed).

If you can't get it, I do intend to look at this at some point, but it could be in another six months.

kibadi commented 4 years ago

Unfortunately I haven't set up the Pebble SDK so far although I have been planning to do it and still am.

Nevertheless I managed to fix it yesterday in the pbw downloaded from the rebble store with only a small modification in the pebble-app.js in line 93:

jsonUrl = "https://api.rss2json.com/v1/api.json?rss_url=" + feed.url;

I changed it to:

jsonUrl = "https://api.rss2json.com/v1/api.json?rss_url=" + encodeURIComponent(feed.url);

since encodeURIComponent( ) was used in other locations in that file, too, and it worked for my RSS feed which originally is (before encoding):

https://www.akafoe.de/gastronomie/speiseplaene-der-mensen/ruhr-universitaet-bochum?mid=1&tx_akafoespeiseplan_mensadetails[action]=feed&tx_akafoespeiseplan_mensadetails[controller]=RSSFeed&cHash=a185a06e54f18d5d021625e0f336c7bf

If you find the time maybe you could add this encodeURIComponent( ) call on the master branch (if you agree that it's the correct fix for this issue), given that it is such a small modification? Would be great, thanks in advance!

Wowfunhappy / Pebble-RSS-Reader

stuck in "Loading.." when opening rss feed #8