charles-001 / dolphie

Your single pane of glass for real-time analytics into MySQL/MariaDB & ProxySQL
https://pypi.org/project/dolphie/
GNU General Public License v3.0
654 stars 40 forks source link

Provide docker image #1

Closed ToshY closed 2 years ago

ToshY commented 2 years ago

Hey @charles-001 πŸ‘‹

This looks like a (visually) cool tool, and I was wondering maybe providing a docker image would simplify setup/usage for those who do not want to install python (and corresponding packages) on their host system.

Let me know what you think.

charles-001 commented 2 years ago

Hi @ToshY! Thanks for your interest! I can certainly create a docker image for Dolphie if that makes it more accessible to users and simplifies things.

charles-001 commented 2 years ago

@ToshY I just uploaded a docker container package. Please give that a try and let me know how it works for you! See README for further information.

ToshY commented 2 years ago

Hey @charles-001

Did you make the package public? When I try to pull I get unauthorized.

Also, I don't see the Dockerfile in the repository yet, is that by choice? And a follow-up question: do you intend to create a workflow to update the docker image on new releases? Or are you going to use another way to do this? Just curious πŸ™‚

charles-001 commented 2 years ago

Oh, oops, I did not make it public. Just did now! I haven't uploaded Dockerfile or did workflow yet, just seeing how well it works atm. Will do that later!

ToshY commented 2 years ago

Hmm I got denied when trying to pull, but I had to create a PAT token and use docker login for ghcr.io (which I normally don't have to) in order to pull it.

After trying to spin it up with the command from the README (docker run -dit --name dolphie ghcr.io/charles-001/dolphie:latest), I get the following response:

WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested

Container died and show logging:

exec /usr/local/bin/python3: exec format error

My VM runs on Ubuntu 20.04.4 LTS

$ uname -a
Linux 5.18.2-051802-generic #202206060740 SMP PREEMPT_DYNAMIC Mon Jun 6 07:46:45 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
$ dpkg --print-architecture 
amd64
charles-001 commented 2 years ago

Hey @ToshY - the reason that is is because it was built on my M1 Mac that uses arm64. I've just pushed a new build that is for amd64 architecture. Please give that a try. As for getting access denied when trying to pull... I'm not quite sure why you have to specify a PAT token. I'm new to all of this, so if you have any idea as to why, let me know :)

edit: Have you been pulling from docker instead of ghcr.io? Maybe that's why.

ToshY commented 2 years ago

edit: Have you been pulling from docker instead of ghcr.io? Maybe that's why.

I'm not sure, seems after pulling it on one of my servers it didn't have any issues. Removed it on VM and pulled again, pulls fine now. Don't ask me why it pulls normally now, it just does πŸ€·β€β™‚οΈ

Hey @ToshY - the reason that is is because it was built on my M1 Mac that uses arm64. I've just pushed a new build that is for amd64 architecture. Please give that a try. As for getting access denied when trying to pull... I'm not quite sure why you have to specify a PAT token. I'm new to all of this, so if you have any idea as to why, let me know :)

latest also seems to work now. Instead of what is denoted in the README, for my use cases I wanted to use Dolphie for existing MySQL instances from other applications, also running in docker. In simple terms, I added the --network option.

# Create
docker run -dit --name dolphie --network=app-network ghcr.io/charles-001/dolphie:latest
# Execute
docker exec -it dolphie dolphie --use-processlist --additional-columns -h mysql -P 3306 -u root --ask-pass

After entering the password, I'm now greeted with the Dolphie dashboard πŸ™‚

image

Now when I access my application (Symfony 6.x + Doctrine ORM) and bunch of queries run in the background, I see in the statistics the QUERIES and SELECT spike, but I don't see anything in the table below.

image

So it does pick up on the queries, but not showing it in the table. I'm not sure if this is either related to my custom MySQL config (not passed to Dolphie, is this required?), or either settings like the refresh interval?

Maybe a better question would be, how are entries/queries displayed in the table (where does it get the information from)?

Additional information

charles-001 commented 2 years ago

Thanks for all of that information @ToshY! I'm glad you got it running :) As for the queries not showing, that is strange. I've tested that thoroughly just now and am not encountering that issue. Your server config should have no effect on Dolphie except for it not being able to use Performance Schema if it's turned off which in this case it is. That wouldn't prevent it from showing queries though since it will use information_schema.PROCESSLIST table. Can you press "I" to see if it shows sleeping threads?

ToshY commented 2 years ago

If I press the uppercase I the following shows in the table image

When running application queries, it briefly shows a thread with sleep (redacted username and database)

image

p.s. Noticed if press ? for help, it doesn't show me the entire help (top is cut-off for keys 1 to P). Using Windows Terminal. image

charles-001 commented 2 years ago

Thanks for trying that. It's possible the queries are so fast that they're not registering in PROCESSLIST table. Can you try it on a busier server? As for the help screen getting cut off, that's due to terminal height size.

ToshY commented 2 years ago

Ah yes that's quite possible, upon further inspection most queries I just tested are processed within 15-200 milliseconds. Also tried some batch operations including INSERTS/UPDATES, but seems that only the statistics panels shows a brief glimpse of these operations. I assume they are too fast to be able to display in the table below.

Can the refresh interval be set to milliseconds? Or would doing this hurt performance?

p.s.

My follow-up questions are getting a bit out of scope regarding the docker addition, should I create separate issues for this?

p.p.s.

Regarding help screen, my screen has a resolution of 1920x1080, and with default settings on Windows Terminal it's just not completely visible. I can zoom out for the current session, which makes it completely visible, so that's fine for now. Maybe for the future, if more settings/panels get added, maybe (arrow-key) pagination would be handy?

charles-001 commented 2 years ago

Refresh interval can't be less than 1 second atm. That would not be good to do really. I can look into arrowing keys for text outside the terminal, but the scroll bar should work in that case.

Feel free to submit other requests that do not relate to the docker container though. Happy to look into them!

On Sun, Sep 11, 2022, 6:32 AM ToshY @.***> wrote:

Ah yes that's quite possible, upon further inspection most queries I just tested are processed within 15-200 milliseconds. Also tried some batch operations including INSERTS/UPDATES, but seems that only the statistics panels shows a brief glimpse of these operations. I assume they are too fast to be able to display in the table below.

Can the refresh interval be set to milliseconds? Or would doing this hurt performance?

p.s.

My follow-up questions are getting a bit out of scope regarding the docker addition, should I create separate issues for this?

p.p.s.

Regarding help screen, my screen has a resolution of 1920x1080, and with default settings on Windows Terminal it's just not completely visible. I can zoom out for the current session, which makes it completely visible, so that's fine for now. Maybe for the future, if more settings/panels get added, maybe (arrow-key) pagination would be handy?

β€” Reply to this email directly, view it on GitHub https://github.com/charles-001/dolphie/issues/1#issuecomment-1242935109, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFBRUKP6KGWGCOFMAG522TV5WYNVANCNFSM6AAAAAAQIUQTBE . You are receiving this because you were mentioned.Message ID: @.***>

ToshY commented 2 years ago

Feel free to submit other requests that do not relate to the docker container though. Happy to look into them!

Okay :+1:


Regarding the docker addition, if you want to automatically update the image, I think creating a workflow to do this for you (preferably with semver tags on images) would be a good start.

charles-001 commented 2 years ago

Sounds good - I will implement the workflow πŸ‘Let me know if queries still aren't showing up under a server with a heavier workload.

On Sun, Sep 11, 2022, 2:45 PM ToshY @.***> wrote:

Feel free to submit other requests that do not relate to the docker container though. Happy to look into them!

Okay πŸ‘

Regarding the docker addition, if you want to automatically update the image, I think creating a workflow https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages to do this for you (preferably with semver tags https://github.com/docker/metadata-action#semver on images) would be a good start.

β€” Reply to this email directly, view it on GitHub https://github.com/charles-001/dolphie/issues/1#issuecomment-1243020432, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFBRUJ4N6ZBESCUPM56XHDV5YSC7ANCNFSM6AAAAAAQIUQTBE . You are receiving this because you were mentioned.Message ID: @.***>

charles-001 commented 1 year ago

Hey @ToshY - just wanted to let you know I've fixed the cutoff screen issue and such with v3.0+. Not sure if you're still using Dolphie or not, but thought I'd reach out anyway :)