Closed umputun closed 3 years ago
that sucks. So, there is no way to have any form of persistent storage for iframes?
@Reeywhaar To answer your questions:
Most of the time, I am not getting a window that opens, I have to click on an auth method's link two or more times to have the window show up. When it does, it closes pretty quickly, but I think there might be a token error, at least I was able to catch it once.
Caddy does not display xsrf errors in the logs.
I embedded Remark as per the instructions in the Readme The host property is identical to REMARK_URL.
I am using Safari version 13.0.3
I unfortunately had to switch to a different comment server for production, but I 'm going to spin up a test version on a different domain and see if the environment variables you mentioned make a difference.
By the way, could we mention this in the build section of the frontend in the readme? It would be very useful if someone wishes to build without the use of Docker, which I think can be a bit of an overkill on low-end hardware, like on a Raspberry Pi.
Also, thank you very much for investigating the possibilities regarding third-party cookies.
Edit: With the risk of sounding ignorant, as I haven't extensively looked into how JWTs are stored, would it be feasible to use sessions?
would it be feasible to use sessions?
I don't see how sessions will change/improve anything here. The problem seems to be not in the auth mechanism but in transport/storage. Currently, JWT transferred (and stored) the same way as a typical session - via a cookie. From browser POV the content of that cookie doesn't really matter.
@Reeywhaar I can't reproduce the issue for different subdomains. I.e. remark42 runs on remark42.example.com and the site on something.example.com. To me, auth works even on paranoid setting fo the FF. Is it different because of the subdomains of the same domain considered trusted?
@umputun, As far as I know, as long as Remark runs on a different subdomain, the embedded iframe is trusted.
This, however, would be different if Remark was running on remark.example1.com http://remark.example1.com/, while it is accessed from example2.com http://example2.com/.
On 2019. Nov 24., at 21:00, Umputun notifications@github.com wrote:
@Reeywhaar https://github.com/Reeywhaar I can't reproduce the issue for different subdomains. I.e. remark42 runs on remark42.example.com and the site on something.example.com. To me, auth works even on paranoid setting fo the FF. Is it different because of the subdomains of the same domain considered trusted?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/umputun/remark/issues/26?email_source=notifications&email_token=AAHLD4B2SPNVSIP2LRZOSDLQVLMPXA5CNFSM4E7Y3HR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFATRDY#issuecomment-557922447, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHLD4ELBDLEMSI43UBGT2LQVLMPXANCNFSM4E7Y3HRQ.
Hi, I'm having trouble importing comments from Wordpress. The command returns the error: 413 Request Entity Too Large.
The file has 6Mb.
do you run it behind of nginx? if so, the default limit for nginx is 1M. You can safely change it to 10M with client_max_body_size 10M;
How fast! :) Yes, that was the problem. Thank you!
I install via docker and caddy, and installed in the sub-path
My docker-compose.yaml file is as follows
version: '3'
services:
remark42:
image: umputun/remark42:latest
container_name: remark42
restart: always
environment:
- REMARK_URL=https://example.com/remark42/
- SITE=my site id
- SECRET=my secret
- AUTH_GITHUB_CID=my github cid
- AUTH_GITHUB_CSEC=my github csec
- ADMIN_SHARED_ID=my share id
volumes:
- ./data:/srv/var
logging:
options:
max-size: "10m"
max-file: "1"
My Caddyfile file is as follows
example.com {
gzip
tls mail@example.com
root /srv/www
log /logs/access.log
# remark42
proxy /remark42/ http://remark42:8080/ {
without /remark42
transparent
}
}
Hope to help you guys!
Hi, I am trying to integrate 'remark42' with a ghost blog.
I'm successfully able to access https://remark.domainname.com/web/
I've integrated the oAuth application (Github and Google yet), whenever I try to login from the oAuth application, after some time, it shows this: {"error":"exchange failed"}
I'm unable to figure out the issue, can you help me in this?
docker-compose.yml environment:
version: '2'
services:
remark:
build: .
image: umputun/remark42:latest
container_name: "remark42"
hostname: "remark42"
restart: always
logging:
driver: json-file
options:
max-size: "10m"
max-file: "5"
# uncomment to expose directly (no proxy)
#ports:
# - "8080:8080"
environment:
- REMARK_URL=https://remark.domainname.com
- SECRET=secret@value
- SITE=remark.domainname.com
- STORE_BOLT_PATH=/srv/var/db
- BACKUP_PATH=/srv/var/backup
- DEBUG=true
- AUTH_GOOGLE_CID=key_here
- AUTH_GOOGLE_CSEC=key_here
- AUTH_GITHUB_CID=key_here
- AUTH_GITHUB_CSEC=key_here
- AUTH_FACEBOOK_CID
- AUTH_FACEBOOK_CSEC
- AUTH_DISQUS_CID
- AUTH_DISQUS_CSEC
- AUTH_ANON=true
# - ADMIN_PASSWD=password
volumes:
- ./var:/srv/var
Nginx conf for remark42 -
server {
server_name remark.domainname.com;
location / {
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass http://172.18.0.2:8080;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/remark.domainname.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/remark.domainname.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = remark.domainname.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
server_name remark.domainname.com;
listen 80;
return 404; # managed by Certbot
}
When exchange failed
issue occurs, this is the log in docker-logs
-
2020/05/25 12:37:56.931 [INFO] {rest/httperrors.go:40 rest.SendErrorJSON} exchange failed - Post "https://oauth2.googleapis.com/token": dial tcp 172.217.194.95:443: i/o timeout - 500 - 49.36.155.53 -
Can you help me out in resolving this issue? It'd be really helpful! Let me know if you need any more information. Once, I get it working, I will make the documentation on how to integrate remark42 with a ghost blog.
Thanks.
based on the last line it looks like your browser can't reach https://oauth2.googleapis.com/token
Maybe you have some kind of local proxy behaving oddly?
This is the URL to be precise which gets the above issue - https://remark.domainname.com/auth/google/callback?state=some_state&code=code_here&scope=scope_here
The flow goes like this:
{"error":"exchange failed"}
This occurs.can you show the exact log message for github as well?
This is the log for GitHub :
2020/05/25 12:40:47.944 [WARN] {lgr/adaptor.go:16 lgr.(*Writer).Write} http: superfluous response.WriteHeader call from github.com/go-chi/chi/middleware.Timeout.func1.1.1 (timeout.go:40)
also, what happens as you do curl -X POST https://oauth2.googleapis.com/token
from the same computer you are trying to login from?
If I do only curl -X POST https://oauth2.googleapis.com/token
w/o any token, it gives -
{
"error": "unsupported_grant_type",
"error_description": "Invalid grant_type: "
}
Hit
https://remark.domainname.com/auth/google/callback?state=some_state&code=code_here&scope=scope_here
with curl still gives exchange failed
yeah, ignore the error in curl, expected. Did you do it from your local machine? If so, could you do the same from the server hosting remark42?
When I hit from the server where remark42 is hosted, I get HTML returned rather than the JSON which I getting from the local machine.
Local machine :
{
"error": "unsupported_grant_type",
"error_description": "Invalid grant_type: "
}
remark42 hosted machine : minified HTML - which just tells
POST requests require a Content-length header. That’s all we know.`
<!DOCTYPE html><html lang=en> <meta charset=utf-8> <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width"> <title>Error 411 (Length Required)!!1</title> <style>*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}</style> <a href=//www.google.com/><span id=logo aria-label=Google></span></a> <p><b>411.</b> <ins>That’s an error.</ins> <p>POST requests require a <code>Content-length</code> header. <ins>That’s all we know.</ins>
well, this shouldn't happen. As I said - looks like some kind of proxy on your side
If I tried Kubernetes, it always not there for me, whether AWS or GCP.
However, I succeed using VM (DigitalOcean Droplet). HTTPS included. Not sure if it is powered by Nginx-le?
Qovery do succeed, but I am not sure if 1 GB hard-disk is overkill.
The settings is -- https://gist.github.com/patarapolw/4c72fa4612c6e1c45474b82bac157f4a
Also, I had problem with social logins. Facebook and Twitter is too strict, and I cannot make it.
I am trying to install remark42 on my Debian server. I have good experience with linux and web technologies, but none with Docker.
I must say the Readme had me totally confused... For instance:
copy provided docker-compose.yml and customize for your needs
OK
make sure you don't keep ADMIN_PASSWD=something... for any non-development deployments
What do you mean? Should I change the default password, remove the line, keep it commented?
pull prepared images from the DockerHub and start - docker-compose pull && docker-compose up -d
What is DockerHub? which images? how to pull? I have read the Docker tutorial but I have still no idea what you mean.
etc... At the 3rd line I am totally lost.
Writing a documentation for a unknown public is hard, my advice is just to provide (lots of) examples. This way I can easily check external documentation / pages and go back to your doc and see if what I think I understood matches your examples.
For instance you can add to the 3rd line docker pull umputun/remark42
. It may seem obvious, but it helps reassuring the novice I am that they he is the proper track. Just like you do not just write just "install foobar" but also "apt install foobar" on debian, even if it is obvious.
Also, it is better to start with the "Happy Path". E.g, state first that there are only 3 variables worth setting for a beginner, and then list all the variable guides, rather than the other way around.
In a nutshell: give lots of examples :-)
Edit: I managed to have remark42 running, my comment is to help enhance the Readme for other newcomers.
@ColasNahaboo
In short, install docker-compose
on your remote machine (not just docker-ce).
What is DockerHub? which images? how to pull? I have read the Docker tutorial but I have still no idea what you mean.
Don't worry about that, just some necessary editing, and then docker-compose up -d
, and you are done. Much easier than PM2, for instance.
HTTPS is a little harder to get running though. It required a lot of guesswork, but it's
ports:
- "80:8080"
- "443:8443"
environment:
- SSL_TYPE=auto
- SSL_ACME_EMAIL=<PUT YOUR EMAIL HERE>
- IMAGE_PROXY_HTTP2HTTPS=true
Well... what is PM2?
To continue on what was obscure:
The comment above the ports sections in docker-compose.yml: "uncomment to expose directly (no proxy)" was also puzzling. I think it actually should be not "no proxy", but "no proxy implemented into the docker swarm itself", am I correct?
And this leads to the question: what happens if I use a non-docker proxy (e.g. a virtual host on my existing apache server, which is already using ports 80 & 443 anyways). Do I also need to protect the external port(s) of remark42 (80 & 443 in your example) against direct external use for security reasons? I see that I can go on web sites using remark42 and directly connect to their external port (often 8080). is this a security issue? Should I add iptables/firewall rules to block it, or is there a way in docker or remark42 to limit access to specific (the one of the proxy) IPs?
So, finding what ports: syntax is is doable with some guesswork into the docker docs. But knowing the security implications of opening these ports is what should be covered in Remark42 Readme, as it is quite hard to guess, with no clear way to be sure I guessed right.
The comment above the ports sections in docker-compose.yml: "uncomment to expose directly (no proxy)" was also puzzling. I think it actually should be not "no proxy", but "no proxy implemented into the docker swarm itself", am I correct?
This comment literally meant - no proxy. It has nothing to do with swarm, k8s, or any other orchestration system. Nginx (or Træfik, or caddy or any other web server supporting reverse proxy) may run in front of remark42 and acts as a proxy. You may see examples for nginx and for caddy
what happens if I use a non-docker proxy (e.g. a virtual host on my existing apache server, which is already using ports 80 & 443 anyways).
It doesn't really matter what form/distribution of proxy you running. It also doesn't matter if remark42 runs inside of the container or directly. In both cases, you may need to proxy requests from your edge server (apache in your example) to remark42 server.
Do I also need to protect the external port(s) of remark42 (80 & 443 in your example) against direct external use for security reasons?
No, and the provided proxy has no protection, they just proxying requests.
Generally, I agree - the documentation should be improved. We are working on this and will use your feedback to make it more understandable. In the current form documentation mostly help someone with sufficient technical background and expects some (basic) understanding of docker, proxies, networking, and so on. I don't think we will ever able to describe all steps in small details (i.e. answer question "what is docker hub") but we could make the documentation much better.
this suggestion got me confused:
For instance you can add to the 3rd line docker pull umputun/remark42
I don't really understand why you even need it? If you follow the documentation and doing docker-compose pull && docker-compose up -d
it will pull the image for you, so no need to do it manually.
This comment literally meant - no proxy
But how? If I leave it commented, I cannot reach the Remark42 from outside docker... no ports seems reachable I have used apache2 as a proxy already for a lots of other systems (Solr, Confluence, Tomcat, ...) but on which port will apache reach Remark42?
No, and the provided proxy has no protection, they just proxying requests.
This is where the doc should be more explicit: I cannot currently envision the big picture of what is going on, so I am a bit nervous in leaving ports open to the world. In all my proxy settings, apache is configured for security, especially for weeding out requests from rogue Chinese search engines and other pests (via fail2ban and honeypots, etc...) and the systems behind the proxy are protected from the outside.
So for instance my questions are: can I add an iptable to block remark42 port from the outside or will it break its normal operation?
I don't really understand why you even need it?
It is because I was missing the implicit fact that I should have run these commands in the same directory as the docker-compose.yml file. So this means that the doc should explicit it :-)
I don't think we will ever able to describe all steps in small details (i.e. answer question "what is docker hub")
You don't need to, it is an impossible task. Just give examples, and people will be able to figure it out. The lack of examples was really what I was pointing out. And I missed more the "big picture" overviews than the details, that I could find in the Docker docs (once I knew what to search). For instance, just make the expression "Docker Hub" in the doc a link to the DockerHub site, no need to try to describe it, but pointing to it will save time and remove guesswork.
Ah, FYI I also looked at https://www.vidal-rosset.net/installing_comment_engine_remark42.html for some complementary info... some examples :-)
But how? If I leave it commented, I cannot reach the Remark42 from outside docker... no ports seems reachable I have used apache2 as a proxy already for a lots of other systems (Solr, Confluence, Tomcat, ...) but on which port will apache reach Remark42?
This part has very little to do with remark42. First of all, you don't have to run remark42 inside of docker container and if you run it directly, port exposed directly (--port or env: REMARK_PORT). In case if remark42 runs inside of the container it should be configured on docker level exactly the same way as any other application running inside of docker and exposing ports. The user should map the internal remark42 port (default 8080) to external (exposed by docker) port. The provided compose illustrates mapping to port 80, i.e. - "80:8080"
OK, so I must uncomment the port line to be able to use a native apache server as a proxy to a Remark42 service running in a docker (and I want to run it in Docker as it is your recommended way). This is what I understood, and I think the "(no proxy)" is thus misleading and inducing confusion, because on the opposite you must uncomment it to use it with a proxy in my case.
I guess my use of the term "swarm" has induced you in error, I must admit I just copied it from the Docker networking doc I found.
yeah, this is confusing indeed. I have not expected users to run such a mixed configuration (proxy outside of the docker, remark42 inside). All examples we have here expect both proxy and remark42 to run inside of the docker. In this case, there is no need to expose ports as containers defined in the same compose file see each other "directly"
Well, I think a lot of people wanting to use remark42 already have a running web site, which is probably not running in docker, that they want to add comments to, or replace the comment system they already use (JS-Kit then Discus for me). So I think my configuration is not so uncommon.
My apache runs 19 web applications for instance, each in its virtual host. I wont dockerize them :-)
right, it is not that exotic. Probably you already figured what to do, but just in case - you need to add this section to your docker-compose.yml (replace 12345 with whatever port you like). On apache side proxy remark's traffic to this port. If your apache and remark42 run on the same host there is no need to open any new ports on your firewall as you should be able to use local ip for the proxy, i.e. 127.0.0.1:12345
ports:
- "12345:8080"
In some rare cases (i have seen it with ubuntu UFW) your firewall may confuse docker's iptable rules and remark42 won't be accessible on 127.0.0.1:12345. Btw, you can check it by doing curl http://127.0.0.1:12345/ping
. So, if you don't get pong
you can try docker's bridge ip (ip a | grep docker0
). In my case, ip will be 172.17.42.1 (you may see a different ip):
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
inet 172.17.42.1/24 brd 172.17.42.255 scope global docker0
hope it helps
Thanks, yes i had figured it out and it works. It was not too hard to find by searching the Docker docs except for he "(no proxy)" confusion.
But my question was more on the security implications wrt remark42. And for this I cannot look elsewhere than what is available on remark42 itself.
Just for info, Things went well, I was able to make Google, GitHub, Twitter... OAuths work. Now I am trying to make email auth work, but since I use my own email server, without a login/password, I think have to tweak things a bit.
The only unclear point was the "SITE" env variable, I had to look in the sources to find that you could use multiple values, and that they were comma-separated.
AH, and I still have a question: I would like to get an email as soon as somebody comments anywhere on the site (it is a personal site with not a lot of traffic), is this possible?
I would like to get an email as soon as somebody comments anywhere on the site (it is a personal site with not a lot of traffic), is this possible?
yes, it is possible. see https://github.com/umputun/remark42/blob/master/docs/email.md
The part about site notifications covered in this section
Thanks! I should have read it throuroughly.
Next roadblock is trying to use any email function results in a "Something went wrong. Please try again a bit later." (frontend) because of a [HTTP/1.1 500 Internal Server Error] (backend).
Is there a way to debug / trace the server to know what is crashing? Or better, is there a way for the server to just call /usr/lib/sendmail
or an external program to send email? In my experience, using libs to send email (be it perl, php, java, ...) has always been problematic, whereas just forking sendmail always works reliably.
I see a backend/app/notify/email_test.go
, can it be used for debugging this?
Have you checked logs? You should see them in your docker container if run attached or by doing docker-compose logs -f
. It is also possible to turn on an extra (debug) logging by adding DEBUG=true
to your compose's enviroment section.
The log may help to figure out what's wrong.
Or better, is there a way for the server to just call /usr/lib/sendmail or an external program to send email?
It uses stdlib SMTP and works just fine. I suspect some configuration issue on your side, but without logs, it is hard to imagine what the issue.
Ok, with the logs I was able to see what is happening: remark42 emails are rejected because they do not use a fully qualified host name: "
I relaxed (temporarily) the rules of my mail server to allow non-fqdn names, but then the stdlib SMTP doesn't seems to implement https://tools.ietf.org/html/rfc2821 (but I may be wrong) and fails with greylisting by not re-sending the email later.
This is an example on why it is better to just fork sendmail in my opinion. email is nowadays extremely complex, and most standard email libraries of programming languages cannot keep up.
I will continue to see how I can tweak my mail server rules to make exceptions for remark42.
I'm not sure what the deal with the fully qualified host name, @paskal can you pls take a look.
Ok, it worked. What I did: For my postfix email server:
reject_non_fqdn_helo_hostname
rule in /etc/postfix/main.cfMy SMTP server saw remark42 identify just by its (variable) IP address: connect from unknown[192.168.0.2]
For my postgrey system:
reject_non_fqdn_hostname
rule in /etc/postfix/main.cf/etc/postgrey/whitelist_clients.local
by adding the rule: 192.168.1.1/16Not defining SMTP_USERNAME and SMTP_PASSWORD
isn't this DNS reverse lookup issue?
I guess it is more a docker issue, as the IP I see for the remark42 server from the outside of docker changes every time it is restarted. So I would have to look in the docker docs on how to allocate a fixed IP, to be able to reverse-DNS on it, but it seems an overkill solution. I like the concept of remark42 being in its self-sufficient "black box", I would not want to have to tweak it needlessly.
I do not think it is the proper issue to fix it. Having the mail server relax its rules for hosts in internal IP ranges seems a more general solution that will also work with other docker apps.
I guess you did not encounter the issue because you seem to have used only 3rd party email services, which requires an account and thus can relax the rules once you are authenticated by SMTP_USERNAME & SMTP_PASSWORD. Basically, in my case my mail server have to do the same rules relaxing, but for internal IPs ranges, which seems a more general solution.
In a nutshell, I do not think remark42 should be changed. I just had difficulties in understanding what was going on in my case.
Cannot get email notification to admin to work, whether Zoho mail or SendGrid... Cannot signup to mailgun for some reasons.
So, how do I debug?
I tried subscribing directly, it also wants token.
Same advice from the above apply, please check docker logs at the time of the problem: docker-compose logs -f
Sendgrid seems to "got blocked", even though I already put sendgrid.net
in spf1
TXT record
If I put Zoho SMTP directly, there is no error, yet no mail received.
NVM, totally fixed with Sendgrid and proper domain setup (at NameCheap).
What worked for me was to look at both:
docker-compose logs -f
to be run in the remark42 install dir.There is a potential issue however: the SMTP lib used by remark42 does not seem to retry if there is a temporary error (I may be wrong, but I have seen the same problem with java libs for instance). So they will fail if they try to send email to destinations using greylisting.
Greylisting has pro and cons, but for a personal email server, with an email address that has been in use for more than 20 years, I find it invaluable as it weeds out 90% of spam. But It also rejects legitimate emails from ill-configured systems, which may be unacceptable for business use.
This means people will not be able tu use email authentication to comment with remark42 if they use Greylisting, and remark42 tries to send email directly, not via a well-behaved smarthost that retries sends on temporary errors after a delay. And this seems the case for Zoho: Zoho admins did not want to use greylisting 5 years ago ("I hate greylisting"), but they apparently use it for at leat 2 years now.
So, try to see if you can whitelist your remark42 server in Zoho.
Whether subscription, or no subscription, but always send email to admin is always broken for me.
I still can receive authorization token, though.
Without more info and logs I can just guess - are you subscribed to replies, like you showed above? If this is the case it is for notifications of other people's replies to your comment and not admin (site) notification.
To activate the admin notification set - AUTH_EMAIL_ENABLE=true
and - AUTH_EMAIL_FROM=notify@example.com
As we installed remark42 on https://radio-t.com and https://remark42.com/demo/ we had no significant issues, and I even tried to use our readme instruction for this procedure. However, my experience is not really typical and some steps may not be as clear to others as it obvious to authors.
If you installed remark42 or at least tried to install - pls share what was not clear and how we can improve the procedure and documentation.