search

gravitl / netmaker

Netmaker makes networks with WireGuard. Netmaker automates fast, secure, and distributed virtual networks.
https://netmaker.io
Other
9.42k stars 547 forks source link

[Bug]: mosquitto -> netmaker.. Working! - somewhat #2019

Open bcookatpcsd opened 1 year ago

bcookatpcsd commented 1 year ago

Contact Details

No response

What happened?

Alpine 3.17.2 as of this AM..

following the no docker documentation.. all seems straight forward.. mosquitto is different on alpine and confusing by nature it looks like..

mosquitto plugin is in a different location..

Alpine is musl and not glibc..

So I rebuilt netmaker binary.. ldd (which netmaker) /lib/ld-musl-x86_64.so.1 (0x7f3a5f4c4000) libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f3a5f4c4000)

using this mosquitto config 


per_listener_settings false

listener 8883 127.0.0.1
allow_anonymous false

listener 1883 127.0.0.1
allow_anonymous false

plugin /usr/lib/mosquitto_dynamic_security.so
# plugin_opt_config_file /etc/mosquitto/data/dynamic-security.json
plugin_opt_config_file /etc/netmaker/dynamic-security.json

tied it directly to loopback.. 

 mosquitto -c /etc/mosquitto/mosquitto.conf
1676128127: mosquitto version 2.0.15 starting
1676128127: Config loaded from /etc/mosquitto/mosquitto.conf.
1676128127: Loading plugin: /usr/lib/mosquitto_dynamic_security.so
1676128127: Opening ipv4 listen socket on port 8883.
1676128127: Opening ipv4 listen socket on port 1883.
1676128127: mosquitto version 2.0.15 running
netmaker -c /etc/netmaker/netmaker.yml

 __   __     ______     ______   __    __     ______     __  __     ______     ______
/\ "-.\ \   /\  ___\   /\__  _\ /\ "-./  \   /\  __ \   /\ \/ /    /\  ___\   /\  == \
\ \ \-.  \  \ \  __\   \/_/\ \/ \ \ \-./\ \  \ \  __ \  \ \  _"-.  \ \  __\   \ \  __<
 \ \_\\"\_\  \ \_____\    \ \_\  \ \_\ \ \_\  \ \_\ \_\  \ \_\ \_\  \ \_____\  \ \_\ \_\
  \/_/ \/_/   \/_____/     \/_/   \/_/  \/_/   \/_/\/_/   \/_/\/_/   \/_____/   \/_/ /_/

[netmaker] 2023-02-11 10:09:20 connecting to sqlite
[netmaker] 2023-02-11 10:09:20 database successfully connected
[netmaker] 2023-02-11 10:09:20 no OAuth provider found or not configured, continuing without OAuth
[netmaker] 2023-02-11 10:09:20 Configuring MQ...
[netmaker] 2023-02-11 10:09:20 MQ config exists already, So Updating Existing Config...
[netmaker] 2023-02-11 10:09:20 REST Server successfully started on port  8081  (REST)
[netmaker] 2023-02-11 10:09:20 connecting to mq broker at ws://127.0.0.1:1883 with TLS? false

no idea where to go from here.. 

1676128160: New connection from 127.0.0.1:57860 on port 1883.
1676128160: Client <unknown> disconnected due to malformed packet.
1676128164: New connection from 127.0.0.1:57874 on port 1883.
1676128164: Client <unknown> disconnected due to malformed packet.
1676128168: New connection from 127.0.0.1:57876 on port 1883.
1676128168: Client <unknown> disconnected due to malformed packet.
1676128172: New connection from 127.0.0.1:34296 on port 1883.
1676128172: Client <unknown> disconnected due to malformed packet.
1676128176: New connection from 127.0.0.1:34300 on port 1883.
1676128176: Client <unknown> disconnected due to malformed packet.
1676128180: New connection from 127.0.0.1:46850 on port 1883.
1676128180: Client <unknown> disconnected due to malformed packet.
1676128184: New connection from 127.0.0.1:46854 on port 1883.
1676128184: Client <unknown> disconnected due to malformed packet.
1676128188: New connection from 127.0.0.1:46858 on port 1883.
1676128188: Client <unknown> disconnected due to malformed packet.

I started netmaker before mosquitto to get the json file updated

more googling and reading..

this seems to work:


per_listener_settings false

listener 8883 127.0.0.1
allow_anonymous false

listener 1883
protocol websockets
allow_anonymous false

plugin /usr/lib/mosquitto_dynamic_security.so
# plugin_opt_config_file /etc/mosquitto/data/dynamic-security.json
plugin_opt_config_file /etc/netmaker/dynamic-security.json

note the specific entry for protocol websocks on the 1883 listener..

and now.. we have this:

# mosquitto -c /etc/mosquitto/mosquitto.conf
1676128820: mosquitto version 2.0.15 starting
1676128820: Config loaded from /etc/mosquitto/mosquitto.conf.
1676128820: Loading plugin: /usr/lib/mosquitto_dynamic_security.so
1676128820: Opening ipv4 listen socket on port 8883.
1676128820: Opening websockets listen socket on port 1883.
1676128820: mosquitto version 2.0.15 running
1676128824: New client connected from 127.0.0.1:37478 as LrgsFB7CaZrTFNVxJxwY1pz (p2, c1, k60, u'Netmaker-Admin').
1676128825: New client connected from 127.0.0.1:37484 as DrcowGcaFh8ceqo3b2nNlU0 (p2, c1, k60, u'Netmaker-Server').
1676128922: Error saving Dynamic security plugin config: File is not writable - check permissions.

1676128922: dynsec: LrgsFB7CaZrTFNVxJxwY1pz/Netmaker-Admin | createRole | rolename=alpine

changed the path back to the local dir for the json file..

started netmaker and cp'd the json file to mosquitto and restarted mosquitto..

still seems to work.. 


per_listener_settings false

listener 8883 127.0.0.1
allow_anonymous false

listener 1883
protocol websockets
allow_anonymous false

plugin /usr/lib/mosquitto_dynamic_security.so
plugin_opt_config_file /etc/mosquitto/data/dynamic-security.json
# plugin_opt_config_file /etc/netmaker/dynamic-security.json

Cunningham's Law applies here.. (please correct me if I'm wrong)

Will continue to plod on..

Version

v0.17.1

What OS are you using?

No response

Relevant log output

No response

Contributing guidelines

bcookatpcsd commented 1 year ago

I put this all into runit.. then it stopped working.. 

USER=root
SHLVL=2
HOME=/root
OLDPWD=/etc/sv/mosquitto
RC_SVCNAME=runitd
TERM=linux
RC_SERVICE=/etc/init.d/runitd
SVCNAME=runitd
PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
PWD=/etc/mosquitto
TZ=America/New_York
1676137654: mosquitto version 2.0.15 starting
1676137654: Config loaded from /etc/mosquitto/mosquitto.conf.
1676137654: Loading plugin: /usr/lib/mosquitto_dynamic_security.so
1676137654: Error loading Dynamic security plugin config: File is not readable - check permissions.

1676137654: Opening ipv4 listen socket on port 8883.
1676137654: Opening websockets listen socket on port 1883.
1676137654: mosquitto version 2.0.15 running
1676137658: Client hBtkA7j8guMwZ0Zkivjjil8 closed its connection.
1676137659: Client hBtkA7j8guMwZ0Zkivjjil8 closed its connection.
1676137663: Client hBtkA7j8guMwZ0Zkivjjil8 closed its connection.
1676137663: Client hBtkA7j8guMwZ0Zkivjjil8 closed its connection.
1676137667: Client hBtkA7j8guMwZ0Zkivjjil8 closed its connection.
1676137667: Client hBtkA7j8guMwZ0Zkivjjil8 closed its connection.
1676137672: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137672: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137676: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137676: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137680: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137680: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137685: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137685: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137689: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137689: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137693: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137693: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137698: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137698: Client Z0mksJHrQ9tqfdJzHyac4IQ closed its connection.
1676137702: Client jb46p0G5hjmlKB4BE8ez5rj closed its connection.
1676137702: Client jb46p0G5hjmlKB4BE8ez5rj closed its connection.
1676137703: mosquitto version 2.0.15 terminating

USER=root
SHLVL=2
HOME=/root
OLDPWD=/etc/sv/netmaker
RC_SVCNAME=runitd
TERM=linux
RC_SERVICE=/etc/init.d/runitd
SVCNAME=runitd
PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
PWD=/etc/netmaker
TZ=America/New_York

 __   __     ______     ______   __    __     ______     __  __     ______     ______
/\ "-.\ \   /\  ___\   /\__  _\ /\ "-./  \   /\  __ \   /\ \/ /    /\  ___\   /\  == \
\ \ \-.  \  \ \  __\   \/_/\ \/ \ \ \-./\ \  \ \  __ \  \ \  _"-.  \ \  __\   \ \  __<
 \ \_\\"\_\  \ \_____\    \ \_\  \ \_\ \ \_\  \ \_\ \_\  \ \_\ \_\  \ \_____\  \ \_\ \_\
  \/_/ \/_/   \/_____/     \/_/   \/_/  \/_/   \/_/\/_/   \/_/\/_/   \/_____/   \/_/ /_/

[netmaker] 2023-02-11 12:48:22 connecting to sqlite
[netmaker] 2023-02-11 12:48:22 database successfully connected
[netmaker] 2023-02-11 12:48:22 no OAuth provider found or not configured, continuing without OAuth
[netmaker] 2023-02-11 12:48:22 could not update user myusernameatnm
[netmaker] 2023-02-11 12:48:22 Configuring MQ...
[netmaker] 2023-02-11 12:48:22 MQ config exists already, So Updating Existing Config...
[netmaker] 2023-02-11 12:48:22 REST Server successfully started on port  8081  (REST)
[netmaker] 2023-02-11 12:48:22 connecting to mq broker at ws://127.0.0.1:1883 with TLS? false
[netmaker] 2023-02-11 12:48:29 Stopping the REST server...
[netmaker] 2023-02-11 12:48:29 REST Server closed.
[netmaker] 2023-02-11 12:48:29 http: Server closed

🤷