search

mlan / docker-kopano

Docker image providing Kopano webmail, ActiveSync, ICAL, IMAP and POP3 services
https://hub.docker.com/r/mlan/kopano
MIT License
13 stars 5 forks source link

Moving mail to public folders #17

Closed osnet closed 3 years ago

osnet commented 3 years ago

Hi @mlan, i got a question ...

i am trying to implement this : https://github.com/initdotlove/zarafa-deliver-mail-to-public-folder

the basics are ok but i am facing a problem ... 

mta_1     | Dec 30 16:32:40 postfix/qmgr[355]: warning: transport kpublic failure -- see a previous warning/fatal/panic logfile record for the problem description
mta_1     | Dec 30 16:32:40 postfix/master[353]: warning: process /usr/libexec/postfix/pipe pid 358 exit status 1
mta_1     | Dec 30 16:32:40 postfix/qmgr[355]: warning: private/kpublic socket: malformed response
mta_1     | Dec 30 16:32:40 postfix/qmgr[355]: warning: transport kpublic failure -- see a previous warning/fatal/panic logfile record for the problem description
mta_1     | Dec 30 16:32:40 postfix/master[353]: warning: process /usr/libexec/postfix/pipe pid 359 exit status 1
mta_1     | Dec 30 16:32:40 postfix/qmgr[355]: warning: private/kpublic socket: malformed response
mta_1     | Dec 30 16:32:40 postfix/qmgr[355]: warning: transport kpublic failure -- see a previous warning/fatal/panic logfile record for the problem description
mta_1     | Dec 30 16:32:40 postfix/master[353]: warning: process /usr/libexec/postfix/pipe pid 360 exit status 1
mta_1     | Dec 30 16:32:40 postfix/master[353]: warning: process /usr/libexec/postfix/pipe pid 361 exit status 1
mta_1     | Dec 30 16:32:40 postfix/qmgr[355]: warning: private/kpublic socket: malformed response
mta_1     | Dec 30 16:32:40 postfix/qmgr[355]: warning: transport kpublic failure -- see a previous warning/fatal/panic logfile record for the problem description

bottom master.cf of mta

# Kopano an public Ordner
kpublic     unix -       n       n       -       -      pipe -v
    flags=DORX argv=/usr/sbin/kopano-dagent --public ${nexthop} -C kopadmin

i guess this will not work as far as kopano dagent is not part of mta

any thoughts how to make this working ? :)

thanks in advance

mlan commented 3 years ago

Hi @osnet

I agree with you that since postfix and kopano-dagent does not run in the same container/host the approach above will not work.

What you can do is to send mail from postfix to kopano-dagent using the LMTP protocol, but that does not immediately solve your problem.

I think it is best to start investigating what methods are available to receive mail in public folders, and then choose the one that fits the container paradigm best. Unfortunately, I have not studied this problem in more detail.

osnet commented 3 years ago

aa you have more docker thingy stuff in ur brain than me : )) i guess i have to work with an dedicated user, otherwise i will not reach the point of transfer from postfix to dagent mmh we'll see ^^

osnet commented 3 years ago

I got an idea :

as stated in Kopano Docs

6.7.4. Kopano-DAgent plugins 6.7.4.1. Move to public The move to public plugin moves incoming messages to a folder in the public store.

Enable the move to public plugin, run the following command:

ln -s /usr/share/kopano-dagent/python/plugins/movetopublic.py \
      /var/lib/kopano/dagent/plugins/movetopublic.py

For this plugin is a config file required. Make a copy of the configuration file with the following command:

cp /usr/share/kopano-dagent/python/plugins/movetopublic.cfg /etc/kopano/movetopublic.cfg

there would be a chance to achieve ldap maintenance if the movetopublic.cfg ... containing stuff like :

#
# Move mail to public configuration
#
# This configuration file is used by the dagent plugin 'movetopublic.py'
# The default location is '/etc/kopano/movetopublic.cfg'

# The user name for whom the rule applies
#  rule#_recipient =
#
# The folder name in the public where the message moved to
#  rule#_destination_folder = 
#
#
# Example:
#   rule1_recipient = username1
#   rule1_destination_folder = folderinpublic1
#
#   rule2_recipient = username2
#   rule2_destination_folder = folderinpublic2
#

rule1_recipient = testuser 
rule1_destination_folder = testfolder

would be somehow filled automatically by script

i.e. searching the ldap OU with the shared folder - contact names, counting them, foreach entry in OU=SharedFoldes,OU=Mail,dc=mail,dc=net echo rule$i_recipient = %cn \n echo rule$i_destination_folder = %description \n\n

BLA this configfile would reside on app - container and maybe working xD

i will have a try and let you know

osnet commented 3 years ago

: )) i guess i made the first step

dependencies: apt update && apt install python-ldap -y

test.py 

import ldap
l = ldap.initialize('ldap://ldap.mail.net')
binddn = "cn=Ldap Bind,cn=Users,dc=mail,dc=net"
pw = "password"
basedn = "ou=SharedFolders,ou=Mail,dc=mail,dc=net"
searchFilter = "(&(kopanoAccount=1))"
searchAttribute = ["mail","description"]
searchScope = ldap.SCOPE_SUBTREE
try:
    l.protocol_version = ldap.VERSION3
    l.simple_bind_s(binddn, pw)
except ldap.INVALID_CREDENTIALS:
  sys.exit(0)
except ldap.LDAPError, e:
  if type(e.message) == dict and e.message.has_key('desc'):
      print e.message['desc']
  else: 
      print e
  sys.exit(0)
try:    
    ldap_result_id = l.search(basedn, searchScope, searchFilter, searchAttribute)
    result_set = []
    while 1:
        result_type, result_data = l.result(ldap_result_id, 0)
        if (result_data == []):
            break
        else:
            ## if you are expecting multiple results you can append them
            ## otherwise you can just wait until the initial result and break out
            if result_type == ldap.RES_SEARCH_ENTRY:
                result_set.append(result_data)
    print result_set
except ldap.LDAPError, e:
    print e
l.unbind_s()

output firing python2 test.py

root@25d1d00d5ed4:/# python2 test.py 
[[('CN=rk@mail.net,OU=SharedFolders,OU=Mail,DC=mail,DC=net', {'mail': ['rk@mail.net'], 'description': ['kpublic:Administration\\FolderRK']})]]

now trim the output conform to the config thingy

mlan commented 3 years ago

Hi @osnet

Looks interesting. Please, keep posting progress here. When you find a solution I will definitely consider integrating it.

Many thanks for your efforts.

osnet commented 3 years ago

@mlan my code is ugly but this shit works : P

additional depencency : apt install python-regex -y

my working Code:

import re
import ldap
l = ldap.initialize('ldap://ldap.mail.net')
binddn = "cn=Ldap Bind,cn=Users,dc=mail,dc=net"
pw = "password"
basedn = "ou=SharedFolders,ou=Mail,dc=mail,dc=net"
searchFilter = "(&(kopanoAccount=1))"
searchAttribute = ["mail","description"]
searchScope = ldap.SCOPE_SUBTREE
try:
    l.protocol_version = ldap.VERSION3
    l.simple_bind_s(binddn, pw)
except ldap.INVALID_CREDENTIALS:
  sys.exit(0)
except ldap.LDAPError, e:
  if type(e.message) == dict and e.message.has_key('desc'):
      print e.message['desc']
  else: 
      print e
  sys.exit(0)
try:    
    ldap_result_id = l.search(basedn, searchScope, searchFilter, searchAttribute)
    result_set = []
    while 1:
        result_type, result_data = l.result(ldap_result_id, 0)
        if (result_data == []):
            break
        else:
            ## if you are expecting multiple results you can append them
            ## otherwise you can just wait until the initial result and break out
            if result_type == ldap.RES_SEARCH_ENTRY:
                result_set.append(result_data)
#    print result_set
    i = 1
    for x in result_set:
#        print x
        start1 = "{'mail': ['"
        start2 = "['kpublic:"
        end1 = "'], 'description':"
        end2 = "']})]"
        s = str(x)
        j = str(i)
        print ("rule" + j + "_recipient = " + s[s.find(start1)+len(start1):s.rfind(end1)])
        string = s[s.find(start2)+len(start2):s.rfind(end2)]
        print ("rule" + j + "_destination_folder = " + string.replace("\\\\", "\\") )
        i = i+1
except ldap.LDAPError, e:
    print e
l.unbind_s()

This gives me : 

root@25d1d00d5ed4:/# python2 test.py 
rule1_recipient = rk@mail.net
rule1_destination_folder = Administration\FolderRK
rule2_recipient = fusel@mail.net
rule2_destination_folder = Administration\Folderfusel

as every body knows : python2 test.py > /etc/kopano/movetopublic.cfg prints this stuff to its destination

#############

Conclusion: I am now able to administer the msil to publicfolders thingy as stated in the above mentioned link within my AD. i keep the kpublic: stuff in case there one time will be a baremetal kopano. now the script should be triggered per crontab as one wishes to.

now its your turn : )

osnet commented 3 years ago

some settings have to be adjusted like plugins and stuff ...

keep u posted

osnet commented 3 years ago

Hi @mlan Shit just got real : D

here is the complete HowTo:

app:
install dependencies as mentioned: python-regex python-ldap

app - dagent: enable plugins, maybe triggered with .env ? /etc/kopano/dagent.cfg

lmtp_listen = *:2003
plugin_enabled = yes
plugin_path = /var/lib/kopano/dagent/plugins/

dependency dagent: ln -s /usr/share/kopano-dagent/python/plugins/movetopublic.py /var/lib/kopano/dagent/plugins/movetopublic.py

##################

How you will implement my python thingy is out of my focus, but be aware that the python call just gives the kopano user back, not the whole emailaddress. But i guess one has to adapt this on its own, some are using the mailaddress as user, i am using sAMAccountName without @domain,net

The main thing is:

rule#_recipient = has to be the kopano user

################

Settings in LDAP / ADS: Where ever one places the stuff for his PublicFolder thingy, here are some things to be mentioned:

after that every thing is running fine

mlan commented 3 years ago

Hi @osnet

Wow, that was an impressive study! Many thanks for sharing it here.

I highly value you outlining the implementation steps, which greatly helps efforts reproducing your solution. One thing that could provide additional help is if you also can share the details of the SharedFolders AD/LDAP entry and, as a reference, the AD/LDAP entry of a normal user.

Additionally I want to mention the kopanoSharedStoreOnly AD/LDAP attribute. Using it appears to provide functionality very close to what you describe above. If its usage can fulfill your needs it might be the preferred method since it is already fully supported and does not require any additional software or configuration that needs to be maintained.

I love to hear what you think about using the kopanoSharedStoreOnly attribute and if it can provide the sought functionality.

I used the following AD/LDAP entries to test the kopanoSharedStoreOnly attribute and was able to receive and read shared mails. That is, I was able to send mails to share@example.com and then, when logged into WebApp, as the demo user, view the inbox of share by using Open Shared Mails. 

dn: dc=example,dc=com
dc: example
objectClass: dcObject
objectClass: organization
o: example.com

dn: ou=users,dc=example,dc=com
ou: users
objectClass: top
objectClass: organizationalUnit

dn: uid=demo,ou=users,dc=example,dc=com
cn: demo
objectClass: top
objectClass: inetOrgPerson
objectClass: kopano-user
sn: demo
uid: demo
mail: demo@example.com
telephoneNumber: 0123 123456789
title: MCP
kopanoAccount: 1
kopanoAdmin: 1
kopanoEnabledFeatures: imap
kopanoEnabledFeatures: pop3

dn: uid=share,ou=users,dc=example,dc=com
cn: share
objectClass: top
objectClass: inetOrgPerson
objectClass: kopano-user
sn: share
uid: share
mail: share@example.com
kopanoAccount: 1
kopanoSharedStoreOnly: 1
osnet commented 3 years ago

ee ya i got your point. Many so not fully understand the diference between shared_folders and public_folders : ) The final behavior is the same for the enduser. But to administer there is a slightly difference.

Shared Folders: Folders mostly within a single user, so stored in user mailbox, user deleted shared stuff gone. (worst case ...) The admin has to manually manage via this special user: rights management, filter and policy management. every normal user has to map the shared folder by himself.

Public Folders: Setup by System storage within mailserver himself or storage server. not user related. Filter/rule management centralized every user has these folders mapped autimatically admin just has to manage some rights directly in the folders also mapped to him. EXTRA FACT: public folders are able to be delivered by EAS to mobiledevices (z-push) , shared folders are not

osnet commented 3 years ago

one small thing is bothering me. this function is only working with users, not with emails, if someone would like have splitted in network-admin, network-issues, network-backup or something l ike that. only network* would be able to handle mmh i can live with that. not nice but ok : P

mlan commented 3 years ago

Hi @osnet

Many thanks for explaining the difference between shared and pubic folders. It was very enlightening.

I noticed that the impersonation mechanism allow shared folders to be synced over EAS too.

Admittedly, I can see that both shared and public folders have their uses.

It would be very useful if you are able to share the details of a sample SharedFolders AD/LDAP entry, that you mention above.

osnet commented 3 years ago

could you specify what details you'd like to see?

mlan commented 3 years ago

I was thinking about something similar to what I posted here https://github.com/mlan/docker-kopano/issues/17#issuecomment-752986153.

osnet commented 3 years ago

ah cn ou dc stuff i'll dump it

osnet commented 3 years ago

This i have in my ADS :

AD: mail.net -> OU: Mail 
 -> OU: SharedFolders

i'll keeping this setup for eventually later purposes

dn: OU=Mail,DC=mail,DC=net
changetype: add
objectClass: top
objectClass: organizationalUnit
ou: Mail
distinguishedName: OU=Mail,DC=mail,DC=net
instanceType: 4
whenCreated: 20200626122703.0Z
whenChanged: 20201231065450.0Z
uSNCreated: 7818113
uSNChanged: 10096344
name: Mail
objectGUID:: redacted
objectCategory: 
 CN=Organizational-Unit,CN=Schema,CN=Configuration,DC=mail,DC=net
dSCorePropagationData: 20201230102648.0Z
dSCorePropagationData: 20200626122703.0Z
dSCorePropagationData: 16010101000000.0Z
kopanoAdminPrivilege: CN=Kopano Admin,OU=Users,DC=mail,DC=net
kopanoSystemAdmin: CN=Kopano Admin,OU=Users,DC=mail,DC=net
kopanoCompanyServer: kopano
kopanoViewPrivilege: OU=Users,DC=mail,DC=net
kopanoAccount: 0

dn: OU=SharedFolders,OU=Mail,DC=mail,DC=net
changetype: add
objectClass: top
objectClass: organizationalUnit
ou: SharedFolders
distinguishedName: OU=SharedFolders,OU=Mail,DC=mail,DC=net
instanceType: 4
whenCreated: 20201230102633.0Z
whenChanged: 20201231065450.0Z
uSNCreated: 10085729
uSNChanged: 10096343
name: SharedFolders
objectGUID:: redacted
objectCategory: 
 CN=Organizational-Unit,CN=Schema,CN=Configuration,DC=mail,DC=net
dSCorePropagationData: 20201230102648.0Z
dSCorePropagationData: 16010101000000.0Z
kopanoAdminPrivilege: CN=Kopano Admin,OU=Users,DC=mail,DC=net
kopanoSystemAdmin: CN=Kopano Admin,OU=Users,DC=mail,DC=net
kopanoCompanyServer: kopano
kopanoViewPrivilege: OU=Users,DC=mail,DC=net
kopanoAccount: 0

<snip>

dn: CN=Einkauf,OU=SharedFolders,OU=Mail,DC=mail,DC=net
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: Einkauf
sn: PublicFolder
description: kpublic:EINKAUF - Postfach
givenName: Einkauf
distinguishedName: CN=Einkauf,OU=SharedFolders,OU=Mail,DC=mail,DC=net
instanceType: 4
whenCreated: 20201231074341.0Z
whenChanged: 20201231101306.0Z
displayName: Einkauf PublicFolder
uSNCreated: 10096823
uSNChanged: 10098380
name: Einkauf
objectGUID:: redacted
userAccountControl: 66048
codePage: 0
countryCode: 0
pwdLastSet: 
primaryGroupID: 513
objectSid:: redacted
accountExpires: 
sAMAccountName: einkauf
sAMAccountType: 805306368
otherMailbox: belege@mail.net
userPrincipalName: einkauf@mail.net
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=mail,DC=net
dSCorePropagationData: 16010101000000.0Z
lastLogonTimestamp: 132538743366125747
mail: einkauf@mail.net
kopanoEnabledFeatures: webapp
kopanoEnabledFeatures: imap
kopanoAccount: 1
kopanoHidden: 1
kopanoUserServer: kopano

</snip>
mlan commented 3 years ago

Hi @osnet

Many thanks for sharing this. I will take a closer look at the method for delivering emails to public folders.

mlan commented 3 years ago

Hi @osnet

In 6e3210163432dd50d752ef8d7530829dd481c699 I have added support for LDAP/AD lookup in the move-to-public dagent plugin. See section Move to public with LDAP lookup for details. I love to learn if you find this update useful.