gromox

Name

gromox — Overview of the Gromox groupware server

Description

Gromox is a groupware server capable of serving as a replacement for Microsoft Exchange. Connectivity options include RPC/HTTP (Outlook Anywhere), IMAP, POP3, an SMTP-speaking LDA, and a PHP module with a Z-MAPI function subset.

Gromox relies on other components to provide a sensibly complete mail system, such as Postfix as a mail transfer agent, and grommunio-admin for user management. A web interface is available with grommunio-web. The grommunio distribution ships these essentials and has a ready-to-run installation of Gromox. system.

Manual page listing

Gromox documentation consists of at least a dozen manual pages ("manpages") on its individual components. We have grouped these according to their principal function.

Overview and definitions

• gromox(7) — This page, an overview of the Gromox groupware server.

• mapi(7gx) — Definition for "Messaging Application Programming Interface"

• gromox-selinux(5) — SELinux policy for Gromox

Exchange subsystem and its components

• autodiscover(4gx) — Autodiscover HTTP Service Protocol handler (AutoDiscover responder).

• autodiscover(7) — AutoDiscover protocols responder).

• exchange_emsmdb(4gx) — http(8gx) processing plugin for the Wire Format Protocol (Outlook/Exchange RPCs).

• exchange_nsp(4gx) — http(8gx) processing plugin for the Exchange Server Name Service Provider Interface Protocol.

• exchange_rfr(4gx) — http(8gx) processing plugin for the Address Book Name Service Provider Interface Referral Protocol.

• exmdb_provider(4gx) — Gromox Information Store

• ews(4gx) — http(8gx) processing plugin for EWS requests

• http(8gx)

• mod_cache(4gx) — http(8gx) processing plugin for serving objects from a local filesystem

• mod_fastcgi(4gx) — http(8gx) processing plugin for proxying requests to FastCGI servers

• mod_rewrite(4gx) — http(8gx) processing plugin for altering HTTP request URIs before processing

• mh_emsmdb(4gx) — http(8gx) processing plugin for MAPIHTTP-wrapped EMSMDB requests

• mh_nsp(4gx) — http(8gx) processing plugin for MAPIHTTP-wrapped NSPI requests

• timer(8gx) — deferred command executor

PHP-MAPI subsystem

• zcore(8gx) — Bridge for PHP-MAPI requests

Mail retrieval agent subsystem (MRA)

• imap(8gx) — IMAP server

• event_proxy(4gx) — Event sender service plugin

• event_stub(4gx) — Event receiver service plugin

• midb_agent(4gx) — Service plugin for conversing with midb(8gx)

• pop3(8gx) — POP3 server

Local delivery agent (LDA)

• alias_resolve(4gx) — Alias resolution for delivery(8gx) using MySQL

• delivery(8gx) — Backend for local delivery

• delivery-queue(8gx) — LMTP/SMTP frontend for local delivery

• user_filter(4gx) — Service plugin for application of user login limits

Auxiliary services

• pam_gromox(4gx) — a PAM plugin to authenticate with Gromox

• event(8gx) — Folder change notification daemon

• midb(8gx) — Message Index database daemon

System administration

• gromox-abktconv(8) — Utility for converting between ABKT and JSON

• gromox-abktpull(8) — Utility to extract ABKT templates from LDIF

• gromox-compress(8) — Utility to recompress Gromox content files

• gromox-dbop(8) — User database maintenance utility

• gromox-dscli(8) — Autodiscover command line utility

• gromox-mailq(8) — SMTP queue lister

• gromox-mbck(8) — Mailbox check and repair utility

• gromox-mbop(8) — Mailbox operations utility

• gromox-mbsize(8) — Mailbox size analysis

• gromox-mkmidb(8) — Tool for creating a blank message index database

• gromox-mkprivate(8) — Tool for creating a blank private store

• gromox-mkpublic(8) — Tool for creating a blank public store

Mail import, export and conversion

• gromox-eml2mbox(8) — Utility for converting RFC5322 Internet Mail messages into a RFC4155 mbox-format mailbox

• gromox-eml2mt(8) — Utility for analysis of/importing RFC5322 Internet Mail messages

• gromox-exm2eml(8) — Utility for exporting messages as RFC5322 Internet Mail

• gromox-exm2ical(8) — Utility for exporting messages as RFC5545 iCalendar objects

• gromox-exm2mt(8) — Utility for exporting messages as a Gromox Mailbox Transfer stream

• gromox-exm2tnef(8) — Utility for exporting messages as TNEF objects

• gromox-exm2vcf(8) — Utility for exporting messages as RFC6540 vCard objects

• gromox-ical2mt(8) — Utility for analysis of/importing RFC5545 iCalendar objects

• gromox-kdb2mt(8) — Utility for analysis of/importing Zarafa/Kopano SQL-stored mailboxes

• gromox-mt2exm(8) — Utility for bulk-importing mail items into a Gromox store

• gromox-oxm2mt(8) — Utility for analysis and import of Outlook .msg files

• gromox-pff2mt(8) — Utility for analysis/import of PFF/PST/OST files

• gromox-snapshot(8) — Helper to create btrfs snapshots of mailboxes

• gromox-tnef2mt(8) — Utility for analysis/import of MS-OXTNEF objects

• gromox-vcf2mt(8) — Utility for analysis/import of vCard objects

• kdb-uidextract(8) — Helper for creating a gromox-kdb2mt ACL map

• kdb-uidextract-limited(8) — Helper for creating a gromox-kdb2mt ACL map

Service plugins

Service plugins can be used by various processes (i.e. from different subsystems), as such, we have put them in this general category.

• authmgr(4gx) — Demultiplexer for authentication requests

• dnsbl_filter(4gx) — DNS Blacklist filtering

• ldap_adaptor(4gx) — LDAP connector for authentication

• mysql_adaptor(4gx) — MySQL/MariaDB connector for user metadata and authentication

• timer_agent(4gx) — Service plugin for deferred command execution with timer(8gx)

Language bindings

• mapi(4gx) — PHP module providing MAPI functions

Configuration files

Program configuration files reside within /etc/gromox. The format for .cfg files is: one "key=value" pair per line. Empty lines are ignored, as are lines beginning with a '#' character. Lines can have a maximum length of 1024. Each key=value line is logically split at the equals sign, and whitespace is trimmed around key and value. Comments at the end of a value are not supported. Escape sequences are not supported.

The format for .ini files is: one "key=value" pair per line. Empty lines are ignored, as are lines beginning with a ';' character.

Many programs have a config_file_path directive with which the search path for further config files can be specified. For example, http(8gx) defaults to config_file_path=/etc/gromox/http:/etc/gromox, so the mysql_adaptor(4gx) plugin as loaded by http will first try /etc/gromox/http/mysql_adaptor.cfg, then /etc/gromox/mysql_adaptor.cfg. This allows having one file that is shared between multiple programs as well as being able to override on a per program-basis.

Listing of config files per component

A list of components and the config files they potentially use.

• alias_resolve(4gx) inside delivery(8gx): /etc/gromox/alias_resolve.cfg, /etc/gromox/mysql_adaptor.cfg

• authmgr(4gx) inside delivery(8gx), delivery-queue(8gx), http(8gx), imap(8gx), midb(8gx), pam_gromox(4gx), pop3(8gx), zcore(8gx): /etc/gromox/authmgr.cfg

• autodiscover(4gx) inside php-fpm(8): /etc/gromox/autodiscover.ini, /etc/gromox/mysql_adaptor.cfg

• delivery(8gx): /etc/gromox/alias_resolve.cfg, /etc/gromox/exmdb_local.cfg, /etc/gromox/ldap_adaptor.cfg, /etc/gromox/mlist_expand.cfg, /etc/gromox/mysql_adaptor.cfg

• delivery-queue(8gx): /etc/gromox/authmgr.cfg, /etc/gromox/midb_agent.cfg, /etc/gromox/ldap_adaptor.cfg, /etc/gromox/mysql_adaptor.cfg

• event(8gx): /etc/gromox/event.cfg

• exchange_emsmdb(4gx) inside http(8gx): /etc/gromox/exchange_emsmdb.cfg

• exchange_nsp(4gx) inside http(8gx): /etc/gromox/exchange_nsp.cfg

• exchange_rfr(4gx) inside http(8gx): no config file

• exmdb_provider(4gx) inside http(8gx): /etc/gromox/exmdb_provider.cfg

• http(8gx): /etc/gromox/cache.txt, /etc/gromox/exchange_emsmdb.cfg, /etc/gromox/exchange_nsp.cfg, etc/gromox/exmdb_provider.cfg, /etc/gromox/fastcgi.txt, /etc/gromox/rewrite.txt

• imap(8gx): /etc/gromox/authmgr.cfg, /etc/gromox/event_proxy.cfg, /etc/gromox/event_stub.cfg, /etc/gromox/imap.cfg, /etc/gromox/ldap_adaptor.cfg, /etc/gromox/mysql_adaptor.cfg

• midb_agent(4gx) inside delivery-queue(8gx), imap(8gx), pop3(8gx): /etc/gromox/midb_agent.cfg

• mlist_expand(4gx) inside delivery(8gx): /etc/gromox/mlist_expand.cfg

• mod_cache(4gx) inside http(8gx): /etc/gromox/http.cfg, /etc/gromox/cache.txt

• mod_fastcgi(4gx) inside http(8gx): /etc/gromox/http.cfg, /etc/gromox/fastcgi.txt

• mod_rewrite(4gx) inside http(8gx): /etc/gromox/http.cfg, /etc/gromox/rewrite.txt

• mh_emsmdb(4gx) inside http(8gx): no config file

• mh_nsp(4gx) inside http(8gx): no config file

• pop3(8gx): /etc/gromox/authmgr.cfg, /etc/gromox/event_proxy.cfg, /etc/gromox/imap.cfg, /etc/gromox/ldap_adaptor.cfg, /etc/gromox/mysql_adaptor.cfg

• timer(8gx): /etc/gromox/timer.cfg

• timer_agent(4gx) inside http(8gx), zcore(8gx): /etc/gromox/timer_agent.cfg

• user_filter(4gx) inside http(8gx), imap(8gx), pop3(8gx): /etc/gromox/gromox.cfg

• zcore(8gx): /etc/gromox/authmgr.cfg, /etc/gromox/zcore.cfg, /etc/gromox/ldap_adaptor.cfg, /etc/gromox/mysql_adaptor.cfg, /etc/gromox/timer_agent.cfg

Listing of components per config file

• /etc/gromox/alias_resolve.cfg: used by the alias_resolve(4gx) plugin, accessed process-wise by the delivery(8gx) process.

• /etc/gromox/authmgr.cfg: used by the authmgr(4gx) and pam_gromox(4gx) plugin, accessed process-wise by delivery(8gx), delivery-queue(8gx), http(8gx), imap(8gx), midb(8gx), pop3(8gx), zcore(8gx), and arbitrary PAM applications.

• /etc/gromox/autodiscover.ini: used by the autodiscover(4gx) component, accessed process-wise by php-fpm(8).

• /etc/gromox/event.cfg: used by the event(8gx) process.

• /etc/gromox/event_proxy.cfg: used by the event_proxy(4gx) plugin, accessed process-wise by imap(8gx), midb(8gx), pop3(8gx).

• /etc/gromox/event_stub.cfg: used by the event_stub(4gx) plugin, accessed process-wise by imap(8gx).

• /etc/gromox/exchange_emsmdb.cfg: used by the exchange_emsmdb(4gx) plugin, accessed process-wise by http(8gx).

• /etc/gromox/exchange_nsp.cfg: used by the exchange_nsp(4gx) plugin, accessed process-wise by http(8gx).

• /etc/gromox/exmdb_local.cfg: used by the exmdb_local(4gx) plugin, accessed process-wise by delivery(8gx).

• /etc/gromox/exmdb_provider.cfg: used by the exmdb_provider(4gx) plugin, accessed process-wise by http(8gx).

• /etc/gromox/gromox.cfg: An effort to consolidate all the invididual .cfg files you see around here. This is a work-in-progress. See the gromox.cfg(5) manpage.

• /etc/gromox/http.cfg: used by the mod_cache(4gx), mod_fastcgi(4gx), mod_rewrite(4gx) plugins, and the http(8gx) process.

• /etc/gromox/imap.cfg: used by the imap(8gx) process.

• /etc/gromox/ldap_adaptor.cfg: used by the ldap_adaptor(4gx) plugin, accessed process-wise by delivery(8gx), delivery-queue(8gx), http(8gx), imap(8gx), midb(8gx), pop3(8gx), zcore(8gx), and arbitrary PAM applications.

• /etc/gromox/midb_agent.cfg: used by the midb_agent(4gx) plugin, accessed process-wise by delivery-queue(8gx), imap(8gx), pop3(8gx).

• /etc/gromox/mlist_expand.cfg: used by the mlist_expand(4gx) plugin, accessed process-wise by delivery(8gx).

• /etc/gromox/mysql_adaptor.cfg: used by the alias_resolve(4gx), mysql_adaptor(4gx) plugins, accessed process-wise by delivery(8gx), delivery-queue(8gx), http(8gx), imap(8gx), midb(8gx), pop3(8gx), zcore(8gx), and arbitrary PAM applications.

• /etc/gromox/midb.cfg: used by the midb(8gx) process.

• /etc/gromox/mod_cache.txt: used by the mod_cache(4gx) plugin, accessed process-wise by http(8gx).

• /etc/gromox/mod_fastcgi.txt: used by the mod_fastcgi(4gx) plugin, accessed process-wise by http(8gx).

• /etc/gromox/mod_rewrite.txt: used by the mod_rewrite(4gx) plugin, accessed process-wise by http(8gx).

• /etc/gromox/mysql_adaptor.cfg: used by the autodiscover(4gx) plugin, http(8gx), imap(8gx), pop3(8gx), zcore(8gx) processes.

• /etc/gromox/pam.cfg: used by the pam_gromox(4gx) plugin, accessed process-wise by arbitrary PAM applications.

• /etc/gromox/pop3.cfg: used by the pop3(8gx) process.

• /etc/gromox/timer.cfg: used by the timer(8gx) process.

• /etc/gromox/timer_agent.cfg: used by the timer_agent(4gx) plugin, accessed process-wise by http(8gx), zcore(8gx).

• /etc/gromox/zcore.cfg: used by the zcore(8gx) process.

Databases

• User information is held in a MariaDB/MySQL database. This database can be accessed by multiple Gromox servers, and so enables distributed Gromox operation. The MariaDB system itself provides the necessary utilities for distributing or replicating this database.

• Per-user e-mail messages are stored in a SQLite database (e.g. /var/lib/gromox/user/m1/1/1/exchange.sqlite3), as is a message index (e.g. /var/lib/gromox/user/m1/1/1/midb.sqlite3). These are normally only used by one system, but can be shared through network filesystems provided that file locking is properly implemented in the filesystem driver. Normal file mechanisms can be used to backup or transfer the database to another Gromox host.

Host addresses

Gromox exclusively uses the AF_INET6 socket family with the Berkeley/BSD/POSIX socket API. What this means is that, whenever an IP address is logged, it will have the form specified in RFC 4291 §2.2/§2.5.5. Furthermore, whenever some configuration file directive (field) requires an IP address (i.e. you cannot or do not want to use a hostname), the RFC 4291 form must be used for both IPv6 and IPv4.

Listening sockets

• /run/gromox/zcore.sock — zcore(8gx)

• *:24 — delivery-queue(8gx) LMTP/SMTP service (when Postfix is on 25)

• :*25 — Normally, your own MTA (postfix(1), exim(8), whatever the case may be). delivery-queue(8gx) will only be on 25 in developer setups that wish to cut and skip Postfix/etc. to get a simpler test setup.

• *:80 — http(8gx) HTTP service

• *:110 — pop3(8gx) POP3 service

• *:143 — imap(8gx) IMAP service

• *:443 — http(8gx) HTTP over implicit TLS

• *:993 — imap(8gx) IMAP over implicit TLS

• *:995 — pop3(8gx) POP3 over implicit TLS

• [::1]:5000 — exmdb_provider(4gx) plugin inside http(8gx)

• [::1]:5555 — midb(8gx) service

• [::1]:6666 — timer(8gx) service

• [::1]:33333 — event(8gx) service

Files

The exact paths depend on the options used when Gromox's build was configured. Especially the path for libraries, represented in this documentation as /usr/lib/gromox, may for example actually be /usr/lib64/gromox or /usr/lib/riscv64-linux-gnu, depending on the platform.

• /usr/lib/gromox/libgxh_*.so: HTTP processing plugins for http(8gx)

• /usr/lib/gromox/libgxm_*.so: hook plugins for delivery(8gx)

• /usr/lib/gromox/libgxp_*.so: PDU processing plugins for http(8gx)

• /usr/lib/gromox/libgxs_*.so: service plugins

• /usr/share/gromox/cpid.txt: mapping between character set IDs and names

• /usr/share/gromox/folder_names.txt: Translations for essential folders in a message store.

• /usr/share/gromox/lang_charset.txt: mapping from language code to character set

• /usr/share/gromox/lcid.txt: mapping between locale IDs and names

• /usr/share/gromox/mime_extension.txt: mapping between file extensiosn and MIME types

• /var/lib/gromox: basic root directory of all variadic data for Gromox

• /var/lib/gromox/queue: directory for delivery-queue(8) temporary files

• /var/lib/gromox/user: default directory hierarchy for private mailboxes

• /var/lib/gromox/domain: default directory hierarchy for public mailboxes (public folders)

  Additional hierarchies for private and public can be added to exmdb_list.txt.
• .../user/account@domain: individual mailbox container

  The directory name/path has only few requirements. It needs to be within one of the exmdb_list.txt-specified hierarchies for private/public hierarchies, and the users.maildir column in MySQL needs to reflect that location. Some user management tools generate extra directory levels, e.g. /user/m1/1/0.

• .../a@d/exmdb/exchange.sqlite3: mail store with almost everything (no mail bodies)

• .../a@d/cid/: attachments and message bodies (PR_BODY, PR_HTML, PR_RTF_COMPRESSED).

• .../a@d/eml/mid_string: RFC5322 representation for a message.

  mid_string has no required form. Typically, there is timestamp.seqid.hostname which represents EMLs captured by delivery(8gx) on ingestion, and timestamp.seqid.midb for EMLs generated by midb(8gx) out of MAPI messages.
• .../a@d/ext/mid_string: Digest for the RFC5322 file.

  This JSON-encoded file contains e.g. indexing information for individual MIME parts of the RFC5322 representation. Generated by midb(8gx).

fail2ban integration

Daemons emit a mostly consistent log messages on authentication failures that can be matched with (PCRE):

/rhost=\[(\S+)?\]\S* user=(\S+) .*(auth|login.*|logon) rejected:/

Operation texts can be "HTTP auth rejected" (http), "zs_logon rejected" (zcore), "zs_logon_token rejected" (zcore), "LOGIN phase0 rejected" (imap), "LOGIN phase1 rejected" (imap), "LOGIN phase2 rejeceted" (imap), "login rejected" (pop3).

Duration specifications

Duration strings must be of the form:

duration := quantum [ quantum ]*

quantum := number unit

# "number" can be whatever strtol(3) accepts. If a period is detected, parsing switches to strtod(3).

unit := "ns" | "nsec" | "µs" | "µsec" | "ms" | "msec" | "s" | "sec" | "second" | "seconds" | "min" | "minute" | "minutes" | "h" | "hour" | "hours" | "d" | "day" | "days" | "week" | "weeks" | "month" | "months" | "y" | "year" | "years"

Whitespace is ignored whereever it appears (so use as much as you need). Quanta with the same unit may be used; they are simply added together. Per this syntax, numbers can be positive or negative, integral or fractional (be mindful of precision limits of computers' floating-point math).

Examples: 1d1h1m1s, 3.5 hours, 1 hour 1 hour (2 hours), 1 hour 60 minutes (2 hours)

Note that Gromox may impose additional restrictions on specific configuration directives after the basic parse to enforce certain minimum and maximum values.