Saving sent mails in aerc + notmuch + mailbox.org

Update:

See part 2 for a better solution.

Original post:

I recently moved to using aerc for email after using mutt (and then neomutt) for some number of years. Today I decided to go beyond a simple maildir for organizing mails locally, and use the notmuch backend for aerc. I use neomutt + notmuch for $dayjob, so I'm no stranger to creating tag 'rules' and searching, etc. What I was unprepared for is that aerc does not care about sent mails after pushing them to smtp. Contrast this to mutt and it's record setting that will save sent mails to some configured location.

The "official stance" on this seems to be either use a custom send script that saves off the mail to a sent dir, or configure something on your mail provider to store mails in the imap Sent folder. I have been using aerc as the smtp client here, but use msmtp elsewhere. Neither support any mechanism for storing sent mails. I decided that option 2 must be the "easier" route in this case.

And for once, it was! (granted, I didn't even try the first option, but I wasn't finding anything promising in the short amount of time I looked into it..)

My mail provider, mailbox.org, features a really nice filter interface for creating rules to do all sorts of things to mail on arrival. So I (ab)used this to make the following rule:

## Conditions
Address         Header  From
                Part    All     Contains        <my email address here>

To                              Contains not    <my email address here>

Cc                              Contains not    <my email address here>


## Actions
Mark mail as                    seen

File into                       Sent

In case it's not clear above:

Move emails from my email address, but not if my email is also in the To: or Cc: header, to the Sent folder and mark them as 'read'.

The exception for having my email address in To: or Cc: is so that I can still send myself emails (for archival purposes) or Cc myself, and still receive the mail in my Inbox.

Great, so now I just have to Bcc: myself on mails in aerc, and they'll get filed into the Sent folder on the imap server. But manually adding Bcc: <me> is tedious. One really (really) cool feature of aerc is message templates (see man aerc-templates). The first step is to configure a user template directory in aerc.config:

[templates]
template-dirs=/usr/share/aerc/templates/:~/.config/aerc/templates/

Then create a custom template, I saved this one to ~/.config/aerc/templates/new.

Bcc: <my email here>


-Clayton

The new-lines in the template are important, one new-line separates the header from the message body, and I added another just so I would have room to start typing the message immediately before my 'signature'

And finally modified the key binding (in bind.conf) for composing a new mail to also include the template:

[Messages]
C = :compose -T new<Enter>

Now mails I sent from aerc are automatically filed into the Sent imap folder on the mail server. When mbsync runs, the sent mails are pulled down locally. The downside to this approach is that extra round trip to the imap server before the sent mail is stored locally on the system that sent it.... But, for now, this will do.

Alpine Linux adventures: running a thing only on system shutdown

One interesting change from Purism recently was to enable something called "ship mode" on the Librem 5's USB charge controller chip. Ship mode causes the chip to cut all power to the rest of the phone, solving the problem of the phone completely draining its battery even when it is "off." This mode is enabled by setting some bits in certain registers on the chip, and should only be enabled on shutdown and not on system reboot. This is key, since their script enables ship mode with a delay... if it's enabled on reboot then power to the phone will be suddenly cut off when the phone is booting back up.

Purism's implementation for setting this up involves a script using i2cset to set these registers. This script is installed to /usr/lib/systemd/system-shutdown/, so systemd will call it on shutdown/reboot. The script knows whether or not the system is going down for a shutdown vs. reboot, because systemd will pass something like poweroff to the script on shutdown. Easy peasy lemon squeezy.

Porting this thing to postmarketOS has been anything but...

Being based on Alpine Linux, pmOS uses openrc to manage daemons. Openrc's openrc-run manpage states that there's an RC_REBOOT variable set in the environment when it runs scripts in the shutdown runlevel. Well, that's exciting, this should be easy too! And wrong I was. After observing that RC_REBOOT is not set on reboot, or shutdown, or at any time, I popped into the #openrc IRC channel for advice. The reason why this wasn't working was because I completely forgot that Alpine Linux uses busybox for init, and does not use openrc-init. RC_REBOOT is only set by openrc-init.

One idea was put forth: just run openrc-shutdown --reboot in the inittab on reboot, which should be straight forward to do since the inittab file allows for specifying a runlevel in the second column on each line.

After that didn't work at all, I came across this gem in the busybox inittab example:

# : The runlevels field is completely ignored.

womp womp.

Next idea: replace busybox init with openrc-init! This was mostly straight forward to do, but required adding some getty setup in /etc/init.d since openrc does not use inittab for spawning tty. Somewhat unsurprisingly, this resulted in RC_REBOOT being set when my ship mode script ran in the shutdown runlevel. I didn't like this solution though, since I did not want the Librem 5 to be the only device in pmOS that required a different init, and replacing the init just for this 1 feature didn't seem worth any future maintenance cost associated with having a different init than the rest of Alpine Linux and pmOS.

Feeling somewhat stuck, I mostly gave up on the idea of using some incantation of busybox and/or openrc to solve this, and started wondering what exactly happens when the Linux kernel does a shutdown or reboot. Let's grep the Linux source!

Searching file names in the tree for 'reboot' yields a number of possibilities, many of them are architecture-specific (e.g. under 'x86' and so on), but one file in particular seemed like a good place to start: kernel/reboot.c The shutdown routine, at least at this level, is pretty straight forward, and can be found under kernel_power_off:

void kernel_power_off(void)
{
    kernel_shutdown_prepare(SYSTEM_POWER_OFF);
    if (pm_power_off_prepare)
        pm_power_off_prepare();
    migrate_to_reboot_cpu();
    syscore_shutdown();
    pr_emerg("Power down\n");
    kmsg_dump(KMSG_DUMP_SHUTDOWN);
    machine_power_off();
}

The important part here is the last line, machine_power_off, which sounds like some pointer to some machine-specific shutdown routine. Grepping for this results in a direct hit in arch/arm64/kernel/process.c. That's relevant since the Librem 5 is an arm64 platform.

machine_power_off in arch/arm64/kernel/process.c is really short and sweet:

/*
* Power-off simply requires that the secondary CPUs stop performing any
* activity (executing tasks, handling interrupts). smp_send_stop()
* achieves this. When the system power is turned off, it will take all CPUs
* with it.
*/
void machine_power_off(void)
{
    local_irq_disable();
    smp_send_stop();
    if (pm_power_off)
        pm_power_off();
}

pm_power_off is a function pointer that can apparently be set by anything (drivers!) to have the "final say" in what actually powers off. pm_power_off is not called on reboot. Now we're getting somewhere!

New idea: modify the bq25890-charger driver to hook into pm_power_off, and set appropriate ship mode registers on shutdown. There are lots (and lots) of examples in the kernel tree of drivers using pm_power_off to do similar things, so it wasn't too bad coming up with a functional POC.

One problem I ran into early with this approach had to do with one condition where we do not want to enable ship mode: when shutting down and the device is charging. Since I was in the charge controller driver, it's trivial to determine the charging state (it's figured out elsewhere in the driver), so I naively added a very simple condition at the start of my pm_power_off function:

/* Don't enable ship mode if charging */
if (bq25890_dev->state.online) {
    dev_info(bq25890_dev->dev, "Charging, not entering ship mode");
    return;
}

In theory, this is what we want... bail out early if charging so that ship mode is not enabled. The important part I was not understanding at the time was that the kernel seems to expect that pm_power_off will actually end with the system powered off. Returning here without doing that results in the system being in some on-but-kinda-off state where it's sucking power but there's nothing alive. Oops! The "fix" was to save the previous value of pm_power_off when this driver loads, then call that when putting the device into ship mode is aborted. I'm sure there's a better way to do this.

In any case, I've compiled a patch and submitted it to Purism's kernel fork. With this I'm able to, as far as I can tell, reliably put the device into ship mode by powering it off while not charging, and not put it into ship mode in all other cases. I hope to have this patch, or something like it, upstreamed to the mainline kernel. At the very least, I now have a way to enable ship mode on the Librem 5 in postmarketOS.

To be continued...

Note: all kernel snippets are from 5.11.4.