patschi / parsedmarc-dockerized

Dockerized self-initializing parsedmarc docker stack for lazy people
GNU General Public License v3.0
69 stars 19 forks source link

Not connect to Gmail Api #10

Closed rabihaggle closed 4 months ago

rabihaggle commented 1 year ago

Hello ! I am testing this utility to process the dmarc. but i have this problem.

parsedmarc -c /etc/parsedmarc/config.ini 0it [00:00, ?it/s] ERROR:cli.py:947:Gmail API Error Traceback (most recent call last): File "/opt/pypy/lib/pypy3.8/site-packages/parsedmarc/cli.py", line 943, in _main oauth2_port=opts.gmail_api_oauth2_port File "/opt/pypy/lib/pypy3.8/site-packages/parsedmarc/mail/gmail.py", line 46, in init creds = _get_creds(token_file, credentials_file, scopes, oauth2_port) File "/opt/pypy/lib/pypy3.8/site-packages/parsedmarc/mail/gmail.py", line 31, in _get_creds oauth2_port=oauth2_port) File "/opt/pypy/lib/pypy3.8/site-packages/google_auth_oauthlib/flow.py", line 497, in run_local_server bind_addr or host, port, wsgi_app, handler_class=_WSGIRequestHandler File "/opt/pypy/lib/pypy3.8/wsgiref/simple_server.py", line 154, in make_server server = server_class((host, port), handler_class) File "/opt/pypy/lib/pypy3.8/socketserver.py", line 452, in init self.server_bind() File "/opt/pypy/lib/pypy3.8/wsgiref/simple_server.py", line 50, in server_bind HTTPServer.server_bind(self) File "/opt/pypy/lib/pypy3.8/http/server.py", line 138, in server_bind socketserver.TCPServer.server_bind(self) File "/opt/pypy/lib/pypy3.8/socketserver.py", line 466, in server_bind self.socket.bind(self.server_address) OSError: [Errno 98] Address already in use

make this change in : docker compose expose port 8080 in parsedmarc

Have you tried using gmail_api?

Thanks

rabihaggle commented 1 year ago

docker compose parsedmarc: image: patschi/parsedmarc:latest volumes:

rabihaggle commented 1 year ago

config.ini

[general] save_aggregate = True save_forensic = True

[gmail_api] credentials_file = /etc/parsedmarc/secret.json

NO EDIT REQURIED DOWN BELOW

[elasticsearch] hosts = elasticsearch:9200 ssl = False

qwell commented 8 months ago

This was the only way I managed to get things to work:

1) Add a new OAuth 2.0 Client ID at https://console.cloud.google.com/apis/credentials. Once created, click Download JSON on the Client ID page and move it to data/conf/parsedmarc/credentials.json

2) In data/conf/parsedmarc/config.ini, add

[gmail_api]
credentials_file = /etc/parsedmarc/credentials.json
oauth2_port = 8080
bind_addr = 0.0.0.0

3) Run docker compose up --no-start to build the containers.

4) Apply my patch for /opt/pypy/lib/pypy3/site-packages/parsedmarc/cli.py and /opt/pypy/lib/pypy3/site-packages/parsedmarc/mail/gmail.py so that it actually understands the bind_addr option.

5) Run docker compose up and stare at the logs intently for a few minutes, to hopefully catch the one line out of the thousands that says to click it to authorize parsedmarc to access Gmail.

I'm sure there was quite a bit more to it than that, and it's almost certainly not going to work, but best of luck to any future readers.

I use this script to handle the patching and start everything.

patschi commented 4 months ago

Appreciate the details from qwell here, but it looks like more a parsedmarc-sided limitation/issue and not related to this packaging project. Hence, closing this one. Hopefully the workaround does the trick.

Feel free to let me know if someone disagrees.

qwell commented 4 months ago

My request here would be to improve documentation for what the config is actually supposed to look like and what needs to be done on the Google side to get the client ID and the like. I've ultimately stopped using parsedmarc, so I'm not really ready to put in that work. You're more than welcome to use info from my comment if you do decide to though.

I would otherwise agree. Changes from my patch really should be committed upstream (and so I'll just say that my diff and script are public domain -- anybody can use it for any reason whatsoever).

patschi commented 4 months ago

Unfortunately neither do I know the parsedmarc code nor am I using Google Mail, so it's hard for me to judge on any code-relevant changes to the upstream project. I'd ask you to create a issue at https://github.com/domainaware/parsedmarc/issues maybe, and contribute it upstream.

So you'd also not need to run any "hacky" workarounds during patching.