search

fossar / selfoss

multipurpose rss reader, live stream, mashup, aggregation web application
https://selfoss.aditu.de
GNU General Public License v3.0
2.36k stars 343 forks source link

Question: Reddit content only in markdown? #1208

Closed seyo-IV closed 4 years ago

seyo-IV commented 4 years ago

Is it possible to show reddit content not in markdown. Its kind of odd, is it a missconfiguration is that how it is?

jtojnar commented 4 years ago

If I recall correctly, we should show the HTML content. But there were some Reddit fixes since 2.18 was released. Which version of selfoss do you use?

Could you share an example of such broken reddit URL? Or maybe try the latest build?

seyo-IV commented 4 years ago

I'm using 2.18 currently. Will try upgrading later.

seyo-IV commented 4 years ago

i failed updating and broke my setup :C. I was using alpine and compose came up with missing dependencies which i did not had after that i upgraded my container os and and then my php config was broken. So il close the issue since my problem would have been fixed if the provided container would use the newer version.

seyo-IV commented 4 years ago

I have update to de lates build as u liked and baked up my data folder and default conf but now im getting this when I tryt o curl:

<!doctype html><html><head><meta charset="utf-8"><title>selfoss</title><meta name="referrer" content="no-referrer"><meta name="viewport" content="width=device-width, user-scalable=yes, initial-scale=1"><meta name="viewport" content="initial-scale=1, user-scalable=yes" media="(device-height: 568px)"><meta name="application-name" content="selfoss"><meta name="apple-mobile-web-app-capable" content="yes"><meta name="mobile-web-app-capable" content="yes"><link rel="apple-touch-icon-precomposed" href="icon-128x128.7be57b65.png"><link rel="icon" type="image/png" href="icon-32x32.15a3e8ca.png"><link rel="shortcut icon" type="image/x-icon" href="favicon.97cdf3fd.ico"><meta name="theme-color" content="#191718"><link rel="stylesheet" href="style.95d11a6e.css"><link rel="manifest" href="selfoss.webmanifest"></head><body> <div id="message"></div> <p id="js-loading-message">selfoss requires JavaScript to run, please enable it on this page.</p> <script>document.getElementById("js-loading-message").textContent="selfoss is still loading, please wait.";</script> <script src="js.1abd9e08.js" onerror="document.getElementById('js-loading-message').textContent = 'Error loading selfoss JavaScript file. Please check the browser console or your web server logs.'"></script>

jtojnar commented 4 years ago

That looks correct – selfoss is a browser app so the web page does not contain much but a code that tries to load the app and print error when it fails. What do you see if you try to open it in the web browser? Do you see any failures in the browser console (F12 key).

seyo-IV commented 4 years ago

im getting bad gateway 502 and when i open console i see an error. Im using nginx as reverse proxy and hes doing basically just that and no more, the conf is pretty simple.

GET https://HOST:PORT/ 502 (Bad Gateway)

GET https://HOST:PORT/favicon.ico 502 (Bad Gateway)

The example above is when i use curl on my host. PS im using a docker container for selfoss.

You know when the new version will be on github?

jtojnar commented 4 years ago

Hmm, 502 would be likely caused by nginx not being able to communicate with the proxied web server. Are you able to access the js.1abd9e08.js file using curl on the host, e.g. using curl --head http://127.0.0.1:port/js.1abd9e08.js? What docker expression do you use?

The archive gets uploaded to GitHub when new release is tagged. The archives I linked are generated and uploaded to bintray for every commit and uploaded to bintray for tagged commits using the same script: https://github.com/SSilence/selfoss/blob/664481dba5387f783627faccda1998d8e2085f3e/.travis.yml#L51-L65. Personally, I use curl https://api.bintray.com/packages/fossar/selfoss/selfoss-git | jq .latest_version --raw-output to get the latest development build.

seyo-IV commented 4 years ago

getting 404 with the js file. Tried the same inside docker, and its the same mybe i have a nginx missconfiguration? Could you share an example of yours? I have build a docker container myself and tried installing. (without compose or a dockerfile)

That what i get when i curl on my host:

<?php

require __DIR__ . '/src/common.php';

// Load custom language
$lang = $f3->get('language');
if ($lang != '0' && $lang != '') {
    $f3->set('LANGUAGE', $lang);
}

// define routes

// all users
$f3->route('GET /', controllers\Index::class . '->home'); // html
$f3->route('GET /api/about', controllers\About::class . '->about'); // json
$f3->route('GET /password', controllers\Authentication::class . '->password'); // html
$f3->route('GET /login', controllers\Authentication::class . '->login'); // json
$f3->route('POST /login', controllers\Authentication::class . '->login'); // json
$f3->route('GET /logout', controllers\Authentication::class . '->logout'); // json
$f3->route('GET /update', controllers\Sources\Update::class . '->updateAll'); // text

// only for loggedin users or on public mode
$f3->route('GET /rss', controllers\Rss::class . '->rss'); // rss
$f3->route('GET /feed', controllers\Rss::class . '->rss'); // rss
$f3->route('GET /items', controllers\Items::class . '->listItems'); // json
$f3->route('GET /tags', controllers\Tags::class . '->listTags'); // json
$f3->route('GET /stats', controllers\Items\Stats::class . '->stats'); // json
$f3->route('GET /items/sync', controllers\Items\Sync::class . '->sync'); // json
$f3->route('GET /sources/stats', controllers\Sources::class . '->stats'); // json

// only loggedin users
$f3->route('POST /mark/@item', controllers\Items::class . '->mark'); // json
$f3->route('POST /mark', controllers\Items::class . '->mark'); // json
$f3->route('POST /unmark/@item', controllers\Items::class . '->unmark'); // json
$f3->route('POST /starr/@item', controllers\Items::class . '->starr'); // json
$f3->route('POST /unstarr/@item', controllers\Items::class . '->unstarr'); // json
$f3->route('POST /items/sync', controllers\Items\Sync::class . '->updateStatuses'); // json

$f3->route('GET /source/params', controllers\Sources::class . '->params'); // html
$f3->route('GET /sources', controllers\Sources::class . '->show'); // html
$f3->route('GET /source', controllers\Sources::class . '->add'); // html
$f3->route('GET /sources/list', controllers\Sources::class . '->listSources'); // json
$f3->route('POST /source/@id', controllers\Sources\Write::class . '->write'); // json
$f3->route('POST /source', controllers\Sources\Write::class . '->write'); // json
$f3->route('DELETE /source/@id', controllers\Sources::class . '->remove'); // json
$f3->route('POST /source/delete/@id', controllers\Sources::class . '->remove'); // json
$f3->route('POST /source/@id/update', controllers\Sources\Update::class . '->update'); // json
$f3->route('GET /sources/spouts', controllers\Sources::class . '->spouts'); // json

$f3->route('POST /tags/color', controllers\Tags::class . '->color'); // json

$f3->route('GET /opml', controllers\Opml\ImportPage::class . '->show'); // html
$f3->route('POST /opml', controllers\Opml\Import::class . '->add'); // json
$f3->route('GET /opmlexport', controllers\Opml\Export::class . '->export'); // xml

// dispatch
$f3->run();

Update: just added a php section in my nginx server block and now i get proxied to my docker container but still cant open selfoss, when i remove the php section it trys to downlaod the php index file :S .

seyo-IV commented 4 years ago

Getting closer to the solution, here is anoter error from the default.log:

[2020-08-03 19:12:14] selfoss.ERROR: could not find driver {"exception":"[object] (PDOException(code: 0): could not find driver at /var/www/html/vendor/bcosca/fatfree-core/db/sql.php:519)
[stacktrace]
#0 /var/www/html/vendor/bcosca/fatfree-core/db/sql.php(519): PDO->__construct('sqlite:/var/www...', NULL, NULL, Array)
#1 [internal function]: DB\\SQL->__construct('sqlite:/var/www...', NULL, NULL, Array)
#2 /var/www/html/vendor/level-2/dice/Dice.php(95): ReflectionMethod->invokeArgs(Object(DB\\SQL), Array)
#3 /var/www/html/vendor/level-2/dice/Dice.php(69): Dice\\Dice->Dice\\{closure}(Array, Array)
#4 /var/www/html/vendor/level-2/dice/Dice.php(182): Dice\\Dice->create('DB\\\\SQL', Array, Array)
#5 /var/www/html/vendor/level-2/dice/Dice.php(95): Dice\\Dice->Dice\\{closure}(Array, Array)
#6 /var/www/html/vendor/level-2/dice/Dice.php(69): Dice\\Dice->Dice\\{closure}(Array, Array)
#7 /var/www/html/vendor/level-2/dice/Dice.php(140): Dice\\Dice->create('daos\\\\sqlite\\\\Dat...', Array)
#8 /var/www/html/vendor/level-2/dice/Dice.php(182): Dice\\Dice->expand(Array, Array, true)
#9 /var/www/html/vendor/level-2/dice/Dice.php(95): Dice\\Dice->Dice\\{closure}(Array, Array)
#10 /var/www/html/vendor/level-2/dice/Dice.php(69): Dice\\Dice->Dice\\{closure}(Array, Array)
#11 /var/www/html/vendor/level-2/dice/Dice.php(182): Dice\\Dice->create('daos\\\\Database', Array, Array)
#12 /var/www/html/vendor/level-2/dice/Dice.php(95): Dice\\Dice->Dice\\{closure}(Array, Array)
#13 /var/www/html/vendor/level-2/dice/Dice.php(69): Dice\\Dice->Dice\\{closure}(Array, Array)
#14 /var/www/html/vendor/level-2/dice/Dice.php(140): Dice\\Dice->create('daos\\\\sqlite\\\\Ite...', Array)
#15 /var/www/html/vendor/level-2/dice/Dice.php(182): Dice\\Dice->expand(Array, Array, true)
#16 /var/www/html/vendor/level-2/dice/Dice.php(95): Dice\\Dice->Dice\\{closure}(Array, Array)
#17 /var/www/html/vendor/level-2/dice/Dice.php(69): Dice\\Dice->Dice\\{closure}(Array, Array)
#18 /var/www/html/vendor/level-2/dice/Dice.php(182): Dice\\Dice->create('daos\\\\Items', Array, Array)
#19 /var/www/html/vendor/level-2/dice/Dice.php(100): Dice\\Dice->Dice\\{closure}(Array, Array)
#20 /var/www/html/vendor/level-2/dice/Dice.php(69): Dice\\Dice->Dice\\{closure}(Array, Array)
#21 /var/www/html/src/common.php(164): Dice\\Dice->create('controllers\\\\Ind...')
#22 /var/www/html/vendor/bcosca/fatfree-core/base.php(1864): {closure}('controllers\\\\Ind...', Array)
#23 /var/www/html/vendor/bcosca/fatfree-core/base.php(1898): Base->grab('controllers\\\\Ind...', Array)
#24 /var/www/html/vendor/bcosca/fatfree-core/base.php(1727): Base->call('controllers\\\\Ind...', Array, 'beforeroute,aft...')
#25 /var/www/html/index.php(57): Base->run()
#26 {main}
"}
jtojnar commented 4 years ago

Do you mean you create a raw Alpine container and then install selfoss inside it manually? What web server do you use inside and how do you run PHP there?

As I understand it, with Docker, one would set it up just like normal web server (though using multiple docker containers for proxied nginx, php-fpm… and managing them using docker-compose might be easier), and then bind the ports from the host to containers (docker-compose should handle this part).

I am not particularly familiar with docker, which is the main reason we still do not have an official docker image. I just use a single nginx instance for all of my web services and php-fpm worker for selfoss:

http {
        ...
        server {
                listen 0.0.0.0:443 ssl http2 ;
                listen [::]:443 ssl http2 ;
                server_name reader.my-host.cz ;
                location /.well-known/acme-challenge {
                        root /var/lib/acme/acme-challenge;
                        auth_basic off;
                }
                root /nix/store/vra92xijaz62f8zjhlln1na0p7cym6ij-selfoss-2.19-664481d;
                ssl_certificate /var/lib/acme/my-host.cz/fullchain.pem;
                ssl_certificate_key /var/lib/acme/my-host.cz/key.pem;
                ssl_trusted_certificate /var/lib/acme/my-host.cz/fullchain.pem;
                location ~* \ (gif|jpg|png) {
                        expires 30d;
                }
                location ~ ^/(favicons|thumbnails)/.*$ {
                        root /var/www/my-host.cz/reader/data;
                }
                location ~* ^/(data\/logs|data\/sqlite|config\.ini|\.ht) {
                        deny all;
                }
                location / {
                        index index.php;
                        try_files $uri /public/$uri /index.php$is_args$args;
                }
                location ~ \.php$ {
                        fastcgi_pass unix:/run/phpfpm/reader.sock;
                        include /nix/store/qsq6ni4lxd8i4g9g4dvh3y7v1f43fqsp-nginx-1.18.0/conf/fastcgi.conf;
                        fastcgi_param PATH_INFO $fastcgi_path_info;
                        fastcgi_param PATH_TRANSLATED $document_root$fastcgi_path_info;
                }
        }
        ...
}

There is a Dockerfile example using Apache in https://github.com/SSilence/selfoss/pull/1170.

Do you see any errors in the docker logs? Adding the following to config.ini in the selfoss repository might help:

logger_destination=file:php://stderr
logger_level=DEBUG
jtojnar commented 4 years ago

Sorry, my network is flaky today, replied too late.

The latest error looks like missing php pdo-sqlite module.

seyo-IV commented 4 years ago

ok now getting this after installing the package in my console:

style.95d11a6e.css:1 Failed to load resource: the server responded with a status of 404 (Not Found)
js.1abd9e08.js:1 Failed to load resource: the server responded with a status of 404 (Not Found)
:7171/favicon.97cdf3fd.ico:1 Failed to load resource: the server responded with a status of 404 (Not Found)
:7171/icon-32x32.15a3e8ca.png:1 Failed to load resource: the server responded with a status of 404 (Not Found)
:7171/selfoss.webmanifest:1 Failed to load resource: the server responded with a status of 404 (Not Found)
:7171/selfoss.webmanifest:1 Manifest: Line: 1, column: 1, Syntax error.
style.95d11a6e.css:1 Failed to load resource: the server responded with a status of 404 (Not Found)

Now after reloading the page i get less 404 but still some errors(i just jecked, files are present in the public directory)

(index):1 GET https://HOST:7171/style.95d11a6e.css net::ERR_ABORTED 404 (Not Found)
selfoss.webmanifest:1 GET https://HOST:7171/selfoss.webmanifest 404 (Not Found)
selfoss.webmanifest:1 Manifest: Line: 1, column: 1, Syntax error.
jtojnar commented 4 years ago

Did the js file load then? It would be weird if it did and CSS did not.

seyo-IV commented 4 years ago

Did the js file load then? It would be weird if it did and CSS did not.

selfoss says Error loading selfoss JavaScript file. Please check the browser console or your web server logs.

Webserver log says:

2020/08/03 19:40:22 [error] 1118#1118: *3 open() "/var/www/html/style.95d11a6e.css" failed (2: No such file or directory), client: 172.17.0.1, server: _, request: "GET /style.95d11a6e.css HTTP/1.0", host: "127.0.0.1:7070"
2020/08/03 19:40:22 [error] 1118#1118: *4 open() "/var/www/html/js.1abd9e08.js" failed (2: No such file or directory), client: 172.17.0.1, server: _, request: "GET /js.1abd9e08.js HTTP/1.0", host: "127.0.0.1:7070"
2020/08/03 19:40:22 [error] 1118#1118: *5 open() "/var/www/html/js.1abd9e08.js" failed (2: No such file or directory), client: 172.17.0.1, server: _, request: "GET /js.1abd9e08.js HTTP/1.0", host: "127.0.0.1:7070"
2020/08/03 19:40:22 [error] 1118#1118: *6 open() "/var/www/html/selfoss.webmanifest" failed (2: No such file or directory), client: 172.17.0.1, server: _, request: "GET /selfoss.webmanifest HTTP/1.0", host: "127.0.0.1:7070"

It looks like its looking at the wrong directory, since the files cant be at the root.

jtojnar commented 4 years ago

Do you use Apache or nginx in the docker container? How did you set it up? With Apache you need to enable mod_rewrite support in .htaccess, with nginx, you need to add the redirect rules to your config as above.

seyo-IV commented 4 years ago

I was missing this part:

                location / {
                        index index.php;
                        try_files $uri /public/$uri /index.php$is_args$args;
                }

Thx for your help and time! 👍

jtojnar commented 4 years ago

Nice. Does it fix the reddit problem?

seyo-IV commented 4 years ago

Nice. Does it fix the reddit problem?

wait il test it right now

seyo-IV commented 4 years ago

Odd its kicking me out after login, then add source :C Top of my screen i see forbidden and on the login screen i see session timedout.

jtojnar commented 4 years ago

Anything in log or browser console?

seyo-IV commented 4 years ago 
:7171/sources:1 Failed to load resource: the server responded with a status of 403 (Forbidden)
jquery.js:9175 GET https://HOST:7171/sources 403 (Forbidden)

nginx log:

2020/08/03 20:06:20 [error] 49#49: *34 FastCGI sent in stderr: "PHP message: HTTP 403 (GET /sources)
PHP message: [vendor/bcosca/fatfree-core/f3.php:39] Base->error()
PHP message: [src/controllers/Sources.php:48] helpers\Authentication->needsLoggedIn()
PHP message: [index.php:57] Base->run()" while reading response header from upstream, client: 172.17.0.1, server: _, request: "GET /sources HTTP/1.0", upstream: "fastcgi://unix:/var/run/php/php7.2-fpm.sock:", host: "127.0.0.1:7070"
jtojnar commented 4 years ago

Hmm, that is weird if you logged in.

Could you try logging out and logging in again?

Another thing to check would be the session cookie in the developer tools, here is how it looks in Firefox:

image

The domain and path should match the URL you used to access selfoss.

seyo-IV commented 4 years ago

thats rly wierd because when i relaod the page im logedin again!?

seyo-IV commented 4 years ago

Another thing to check would be the session cookie in the developer tools, here is how it looks in Firefox:

Session cookie is there the domain is correct.

The domain and path should match the URL you used to access selfoss.

Path is root, domain is also ok, cookie is there... its really odd. Maybe something im missing in my php.ini file some sort of setting?

Since its publically accessable i would not like to remove the password. But it would be a workarround. Remove the password, set sources, set password again.

jtojnar commented 4 years ago

With

logger_level=DEBUG

in config.ini, you might get some extra details in selfoss logs.

Should print something like https://github.com/SSilence/selfoss/blob/664481dba5387f783627faccda1998d8e2085f3e/src/helpers/Authentication.php#L43

seyo-IV commented 4 years ago

hmm ok.. now it works. It seems like my config.ini was corrupt.. i accidentaly broke it by editing(im not filly retarded im just using a ssh gateway and it has some problems with copy papaste) and made a new one, now it works, will try reddit content now.

seyo-IV commented 4 years ago

i do get error when i try to trigger the update:

[2020-08-03 20:58:04] selfoss.DEBUG: Established database connection  
[2020-08-03 20:58:04] selfoss.DEBUG: ---  
[2020-08-03 20:58:04] selfoss.DEBUG: start fetching source "r/Linux (id: 1)   
[2020-08-03 20:58:04] selfoss.DEBUG: spout successfully loaded: spouts\reddit\reddit2  
[2020-08-03 20:58:04] selfoss.DEBUG: fetch content  
[2020-08-03 20:58:05] selfoss.DEBUG: [2020-08-03T18:58:05+00:00] "GET /r/linux.json HTTP/1.1" 200  
[2020-08-03 20:58:05] selfoss.DEBUG: minimum date: 2020-07-04 20:58:05  
[2020-08-03 20:58:05] selfoss.DEBUG: start item fetching  
[2020-08-03 20:58:05] selfoss.DEBUG: start insertion of new item "I'm Jason A. Donenfeld, security researcher, kernel developer, and creator of WireGuard, `pass(1)`, and other various FOSS projects. AMA!"  
[2020-08-03 20:58:05] selfoss.DEBUG: item content sanitized  
[2020-08-03 20:58:08] selfoss.DEBUG: [2020-08-03T18:58:08+00:00] "GET /r/linux/comments/hzyu8j/im_jason_a_donenfeld_security_researcher_kernel/ HTTP/1.1" 200  
[2020-08-03 20:58:11] selfoss.DEBUG: [2020-08-03T18:58:11+00:00] "GET /r/linux/comments/hzyu8j/im_jason_a_donenfeld_security_researcher_kernel/ HTTP/1.1" 200  
[2020-08-03 20:58:11] selfoss.DEBUG: [2020-08-03T18:58:11+00:00] "GET /desktop2x/img/favicon/android-icon-192x192.png HTTP/1.1" 200  
[2020-08-03 20:58:11] selfoss.DEBUG: [2020-08-03T18:58:11+00:00] "GET /desktop2x/img/favicon/apple-icon-180x180.png HTTP/1.1" 200  
[2020-08-03 20:58:11] selfoss.ERROR: Call to undefined function WideImage\imagecreatefromstring() {"exception":"[object] (Error(code: 0): Call to undefined function WideImage\\imagecreatefromstring() at /var/www/html/vendor/smottt/wideimage/lib/WideImage/WideImage.php:227)
[stacktrace]
\0 [internal function]: WideImage\\WideImage::loadFromString('\\x89PNG\
\\x1A\
IHD...')00\\x00\
\1 /var/www/html/vendor/smottt/wideimage/lib/WideImage/WideImage.php(167): call_user_func(Array, '\\x89PNG\
\\x1A\
IHD...')00\\x00\
\2 /var/www/html/src/helpers/Image.php(226): WideImage\\WideImage::load('\\x89PNG\
\\x1A\
IHD...')00\\x00\
#3 /var/www/html/src/helpers/Image.php(102): helpers\\Image->loadImage('https://www.red...', NULL, NULL)
#4 /var/www/html/src/spouts/reddit/reddit2.php(161): helpers\\Image->fetchFavicon('https://www.red...')
eddit2->getIcon()src/helpers/ContentLoader.php(196): spouts\
#6 /var/www/html/src/helpers/ContentLoader.php(52): helpers\\ContentLoader->fetch(Array)
#7 /var/www/html/src/controllers/Sources/Update.php(37): helpers\\ContentLoader->update()
#8 /var/www/html/vendor/bcosca/fatfree-core/base.php(1927): controllers\\Sources\\Update->updateAll(Object(Base), Array, 'controllers\\\\Sou...')
#9 /var/www/html/vendor/bcosca/fatfree-core/base.php(1727): Base->call(Array, Array, Array)
#10 /var/www/html/index.php(57): Base->run()
#11 {main}
"}
seyo-IV commented 4 years ago

im a retard sorry, after making a new conf i forget to set public to 1 and set username/paswwod... so im at the same point again :S

jtojnar commented 4 years ago

imagecreatefromstring is provided by php gd module, we should definitely handle it more gracefully.

seyo-IV commented 4 years ago

ok stuff gets to complicated for me, i think il switch to the docker container i used initially and wait for 2.19 release. Sry for stealing your time. This login issue is really odd and im to tired to investigate further and steal even more of your time.