Deb.News

Steinar H. Gunderson: Decompilation patterns, part 4: range comparison

Wed, 07 Oct 2026 15:18:43 +0000

Today, we're looking at a rewrite that rarely differs for matching, but can mean quite a lot for understanding the code. Say that you have a signed variable and the compiler has suddenly decided to make an unsigned comparison:

    if ((u32) (x - 4) < 2) {

What's happening is that it uses a controlled overflow to do two signed comparisons using one unsigned comparison. How does this work?

Since the comparison is unsigned, this is trivially equivalent to the first expression:

    if ((u32) (x - 4) >= 0 && (u32) (x - 4) < 2) {

and since negative numbers compare larger than 6 in unsigned comparisons, nothing prevents us from adding 4 on each side of the inequality signs:

    if ((u32) (x) >= 4 && (u32) (x) < 6) {

which is, for the same reason, equivalent to:

    if (x >= 4 && x < 6) {

which is what you probably should be writing.

Bits from Debian: Looking for artwork for the next Debian release: Forky

Wed, 07 Oct 2026 09:00:00 +0000

Each release of Debian has a shiny new theme which is visible on the installer, the boot screen, the login screen and, most prominently, on the desktop wallpaper. It's a very important part of a Debian release as it is usually the first thing new users see when installing or booting the system for the first time. Not only that, but also the first thing appearing when debianites share their screen or connect to an external device when giving talks around the world.

As the most enthusiastic users will know, the forky - yes, that is the name of the next stable version, Debian 14 - release cycle is rapidly approaching its latter stages. This means we need to select the artwork shipping with forky soon, really soon!

And as with everything else in Debian, collecting artwork is a collaborative effort which Debian shares with its community. So, if you would like (or know someone who would like) to create a desktop look and feel that will be seen by trillions of people around the world - and in space! - be sure to send in your artwork ASAP!

The deadline for submissions is: 2026-11-26.

For the most up to date details please refer to the Debian wiki.

At the same time, we would like to thank Elise Couper for creating the Ceratopsian theme for our last trixie release.

And for the interested or the curious ones, the artwork is usually picked based on which theme looks the most:

''Debian'': admittedly not the most defined concept, since everyone has their own take on what Debian means to them. Though, usually they all agree when something looks like Debian.
''Plausible to integrate without patching core software'': as much as we love some of the insanely looking themes, some would require heavy GTK+ theming and patching GDM/GNOME.
''Clean and well designed'': without becoming something that gets annoying to look at a year, or ten, down the road. Examples of good themes include Emerald, Homeworld, Joy, Lines, softWaves and futurePrototype

If you'd like more information or details, please post to the Debian Desktop mailing list.

Steinar H. Gunderson: Decompilation patterns, part 3: do-while-while

Tue, 06 Oct 2026 21:35:00 +0000

Today's decompilation pattern is a variation on yesterday's loop pattern, because loops are important and tend to induce a lot of transformations by the compiler. So assume you have code like this (perhaps after converting a goto):

if (var_s0 > var_s1) {
  // Perhaps some other initialization stuff here
  do {
    ...
  } while (var_s0 > var_s1);
}

Assuming there are no side effects from the initializations, a natural candidate would be that the programmer instead wrote:

// Initialization stuff moved up here
while (var_s0 > var_s1) {
  ...
}

Remember to remove the old while so that it does not say while (x) { ... } while (x); it won't tell you with a syntax error, but rather leave an infinite loop that will mess up the generated code.

Russell Coker: A Power Case for a FOSS Phone

Tue, 06 Oct 2026 08:08:04 +0000

The Problem

I want to use a FOSS phone running Debian as my daily driver and not use a non-free OS on my phone. GrapheneOS [1] is a FOSS rebuild of Android, it’s a great project and I commend them for what they are doing. But I want a system where I have access to all the bits and where I can ssh in to it and manage it in the same way as all other Linux systems. Android is free software but it’s a closed design and the phones are expected to be appliances not full peers on the network. The Android native terminal emulator (AKA Linux development environment) is a nice feature, a VM with a default of 1GB of RAM but it’s still separate from the main OS.

KDE Connect [2] is a system for KDE talking to phones, it is a great project and allows convenient interaction between a Linux PC and a phone. But in addition to that I’d like the option to ssh to my phone to send an SMS or have one sent from a cron job. I blogged about using the basic functions of Kdeconnect from the command-line [3].

I have had ongoing issues with the battery not lasting long enough on the PinePhonePro (PPP) and the Librem 5 (L5) which makes them unusable for my purposes. For a phone running Droidian I can have an open ssh session (with keep-alive packets) running overnight on Wifi without any problems while for the PPP or L5 an hour of reading an ebook (the least demanding use of a phone) will use most of the battery.

I sent my FuriPhone back for warranty repair and it’s been almost a month with no follow up, so I need to have another option.

The Aim

The aim of this post is to develop a rough design for a battery case for either a L5 or a PPP, I have both phones and they are both capable of doing what I want apart from battery life. Also designing a case that basically works for both phones and has two variations of the CAD file for 3D printing is good to allow collaboration with people who use either phone. Protecting the phone from damage when dropped is also a core feature.

I investigated commercial options. There are some reports of cases for other phones working, someone reported a PPP working well with a battery case for a Samsung S20 Ultra. The OtterBox uniVERSE Case for Samsung Galaxy XCover Pro is reported to fit the PPP so probably battery cases for the XCover Pro will also work. There are also universal power cases which have spring clips to fit a wide range of phone sizes, but they are much larger than other options and probably increase the risk of damage if the phone is dropped.

At this stage I’m planning out the rough specs of a case that can be 3D printed to protect a phone while being easy to change the cells. The process of changing cells should be quick, easy, and not risk breaking anything. The hardware required should be affordable ($100AU plus some 3D printing seems reasonable) and should not require significant skills to assemble. After recent experiments with Thinkpad repair I’ve determined that I am not good at electronics by today’s standards so I plan to avoid anything difficult in that regard.

Battery Options

There are phone cases with batteries built in for the more common phones. A quick search on AliExpress turned up options for recent Pixel phones. They aren’t as common as they used to be as Android phones and iPhones generally have good battery life nowadays. Of course there aren’t such options for less common phones like the L5 and PPP.

It is cheap and easy to buy a portable battery for charging phones, but that’s a separate bulky device and unless the connector is inside a protective case it leaves the phone vulnerable to catastrophic damage if dropped.

So can a suitable battery case be designed and 3D printed?

Cheap Batteries

According to Wikipedia the 18650 LiIon cell is the most commonly used LiIon battery, that is 18mm in diameter and 65mm in length. For a phone case thinner batteries would be more convenient but economies of scale make the 18650 cheap to buy new and commonly available on the second hand market. The current prices on AliExpress (in Australian dollars) are about $8 per cell when buying 10 at a time. Most of the cells on AliExpress don’t have wires attached so they can be swapped in a carrier the way AA batteries are used, but the cells are symmetrical (no bump for positive) so the user has to take care of polarity. Chargers are around $22 for a 4 cell charger or $52 for a 12 cell charger.

I’ve seen prices as low as $1 per cell on the second hand market, I haven’t investigated the amount of usable capacity in such cells. Probably the prices for chargers I’ve found aren’t the best available but they are good enough to start the design. 10 cells and a 4 cell charger is about $100. That’s about 130Wh while a 74Wh USB-C battery pack costs about $50. So the price per Wh is OK.

I have read about an attempt to do this for the L5 with two of the batteries designed for the L5 in a case attached to it, so you have 3*L5 batteries. It’s an interesting approach but those batteries have poor value for money when compared to 18650 ($29US for 4500mAh compared to $8 for 3500mAh) and they also don’t ship them outside the US at this time. The 18650 batteries are available everywhere cheaply and have multiple uses.

How They Fit

According to my measurements the PPP is 76mm wide and the L5 is 73mm wide. That’s wide enough for 3*18mm or 4*18mm cells side by side arranged parallel to the long side of the phone or one 65mm long cell going across the phone with the necessary spring and wires.

The L5 is about 151mm long and the PPP is about 5mm longer. So that means that for just the cells we could have 2 cells in a row in parallel to the long side of the phone giving a 2*3 or 2*4 array or we could have a row of 8 cells parallel to the short side of the phone.

A quick search for controllers for a battery of LiIon cells that outputs USB-PD turned up one of the smaller ones as 40*28mm, here is the AliExpress page [4]. So that means we could have a maximum of about 4 cells and the controller while leaving space for the camera. Looking at other similar items it seems that most of them take a large portion of the 73mm width available and take a bit less than 30mm height. The price range for DC-DC voltage converters ranges from about $3 to $20, the one I linked to is currently $8.

The mass of 18650 cells is around 45g so 4 of them would be about 180g before counting the weight of the case and electronics. A L5 weighs 262g and a PPP weighs 220g so 180g of battery will make a significant difference, not an impossible difference but holding a PPP while reading an ebook is already annoying. Maybe the case could be designed to be easier to hold, a loop of wool attached to the top could be used to suspend the phone from one finger while the rest of the hand just keeps it steady.

In a default configuration the L5 has a 4500mAh 3.8V battery and the PPP has a 3000mAh 3.8V battery. The 18650 cells have up to 4000mAh with 3500mAh being common. So 4 such cells could multiply the battery life of a PPP by a factor of 5 or a L5 by 4. That would not be enough to last for a whole day without charging so they need to be easy to swap.

A 3 cell battery seems like a good option, 135g of batteries so the project overall wouldn’t even double the weight of the phone with battery case. As it doesn’t seem possible to put in enough cells to last a day of reasonable use (running a web browser, checking email, reading wikipedia, and having some IM systems checking for notifications) there doesn’t seem to be a benefit in trying to stuff the maximum number of cells in. Going to 4 or 5 cells doesn’t provide much benefit over 3 while increasing the weight and making the design more difficult. The modules for DC Voltage conversion often have configuration for the number of cells to be used so it wouldn’t be difficult to print a new case and reconfigure the Voltage converter.

Design

A case needs to provide protection as well as hold the batteries. The lack of cases for the L5 and PPP is a problem even without the extra weight of a battery pack increasing the kinetic energy of falling phone. Ideally a drop from 1M on to concrete would not cause any damage to the phone, destruction of the case is OK as it would be 3D printed and can be rebuilt easily.

I think that a case where the phone slides in from the top would be best. To change cells one would slide the phone out and the cells would be immediately accessible with no divider between the phone and the cells. Sliding the phone in would hold the cells against the back of the case and as there is a gap of about 3mm on each side of the PPP between the display area and the edge of the phone there’s plenty of space for the case to wrap around it at the sides and bottom. At the top there would have to be some sort of plastic clip. The PPP only has buttons on the top right so the case can be snug on all sides apart from the top right. The L5 has switches at the top left and buttons at the top right so whatever clips on to the top would really need some strength to cover the fall flat on face case.

The USB C plug would need to be held in place well enough to allow the phone to easily slide on to it but also be weak enough that it would move if a drop forced the phone out of the case.

Thingiverse has a design for a L5 case [5] (with several derivatives) and a design for a PPP case [6]. Those cases could be used as starting points and then changed to store batteries and the voltage change circuit board. I have just learned that it’s possible to use rubberised material in a 3D printer which is apparently what those designs are for. If a case was printed with rubberised material the top could just stick in place after being pushed in.

Potential for Excessive Excitement

Having an exploding device in your pocket would be the exciting in a bad way. My initial idea was to have multiple 18650 cells in series. The problem is that if the cells have different capacity levels then one can run out before the rest and get to a deep discharge state which at a minimum causes damage to the cell.

One way of minimising risk is to have only a single cell in use at one time, here is a converter for a single cell to 5V [7]. It is possible to run multiple cells in parallel by simply wiring them together, but it’s recommended to make sure that the voltage difference between cells is less than 0.2V to avoid high current when connecting.

The arrangement of cells is often referred to as nS or nP to refer to n cells in series or parallel. Aliexpress has a range of 1S to 6S devices. Some of the 2S devices have a connector for the mid point which permits separate voltage monitoring for both cells but there doesn’t seem to be anything equivalent for 3S or more. They also don’t appear to have anything to specifically manage 2P or 3P arrangements at the moment (they had a 3P board on sale when I looked into this last year).

Conclusion

This is a bigger project than I expected when I first started investigating it. I’ve played with FreeCAD which seems OK but will take a lot of learning and practice. Then I need to do more investigation into the DC-DC converters to find one that works well and is unlikely to start a fire.

I will look into other phone options as well and keep chasing Furilabs people about my phone replacement.

Russell Coker: Kdeconnect DBUS

Mon, 05 Oct 2026 22:22:49 +0000

I’ve tested the DBUS interfaces for Kdeconnect (the system for connecting a KDE desktop to a phone or another computer). Unlike most DBUS interfaces it has a significant tree of interfaces so the busctl command to get the tree is important. Here’s the useful things I discovered:

# list basic interfaces for kdeconnect
qdbus6 org.kde.kdeconnect.daemon /modules/kdeconnect
# get tree of interfaces for kdeconnect
busctl --user tree org.kde.kdeconnect.daemon
# get list of devices
qdbus6 --literal org.kde.kdeconnect.daemon /modules/kdeconnect org.kde.kdeconnect.daemon.deviceNames
# list interfaces for a device (DEVID is a hex string from the above command)
qdbus6 org.kde.kdeconnect.daemon /modules/kdeconnect/devices/$DEVID
# get battery charge
qdbus6 org.kde.kdeconnect.daemon /modules/kdeconnect/devices/$DEVID/battery org.kde.kdeconnect.device.battery.charge

Sending an SMS apparently requires using the QVariantList type which is more pain than I wanted so I tried the kdeconnect-cli command. Here’s a quick summary of how to use it:

# list devices
kdeconnect-cli -l
# send a SMS
kdeconnect-cli -d $DEVID --send-sms "the message" --destination $NUMBER
# find device (ring)
kdeconnect-cli --ring -d $DEVID
# send a notification message
kdeconnect-cli --ping-msg "this is the message" -d $DEVID
# unlock and lock screen (only works on Debian devices for me not Android devices)
kdeconnect-cli --unlock -d $DEVID
kdeconnect-cli --lock -d $DEVID

There doesn’t seem to be support for making phone calls via kdeconnect which is a significant omission. It’s a very common situation to want to call a number that’s listed on a web site or in some other data source that’s easier to access on a PC than on a phone. The clipboard could be used but that’s needless pain.

Steinar H. Gunderson: Decompilation patterns part 2: do-while

Mon, 05 Oct 2026 15:54:00 +0000

Continuing our journey on decompilation patterns, here's another common one: Let's say m2c outputs code like this:

loop_151:
  ...
  if (var_s0 > var_s1) goto loop_151;

then the natural change is:

do {
  ...
} while (var_s0 > var_s1);

Fewer gotos are nearly always good, and this is a much more likely pattern than the original one.

m2c often manages to convert gotos to do/while, but not always when they are e.g. nested in some other loop; so often, you may need to apply some other transformation before you get to this point.

Tomorrow, we'll look at another variation over this topic.

Vincent Bernat: Hacking the Go compiler to efficiently map IPv4 to IPv6

Sun, 04 Oct 2026 14:12:19 +0000

netip.Addr features an Unmap() method returning the unwrapped IPv4 contained in an IPv4-mapped IPv6 address: from ::ffff:203.0.113.10 or ::ffff:cb00:710a, it returns 203.0.113.10.¹ There is no Map() or To6() method for the reverse direction. Such a method is trivial to implement, but Go maintainers have rejected it on the grounds that users should write netip.AddrFrom16(ip.As16()) and let the compiler optimize it.² Today, this pattern is eight times slower than a native method. How can we teach the compiler to optimize this sequence?

The alternatives

Letâ€™s explore three ways to implement the map semantics for netip.Addr. My favorite is to add it to the Go standard library. Go maintainers prefer a small external helper chaining netip.AddrFrom16() and netip.Addr.As16(), hoping the compiler eventually optimizes it. The unsafe package opens a third path, with the same performance as the first solution.

Modifying the Go standard library

Internally, netip.Addr stores any IP address as a 128-bit value with an extra field z to encode the family and the zone:

type Addr struct {
    addr uint128
    z unique.Handle[addrDetail]
}

type addrDetail struct {
    isV6   bool   // IPv4 is false, IPv6 is true.
    zoneV6 string // != "" only if IsV6 is true.
}

var (
    z0    unique.Handle[addrDetail]
    z4    = unique.Make(addrDetail{})
    z6noz = unique.Make(addrDetail{isV6: true})
)

AddrFrom4() encodes an IPv4 address as an IPv4-mapped IPv6 address and sets z to the unique value z4:

// AddrFrom4 returns the address of the IPv4 address given by the bytes in addr.
func AddrFrom4(addr [4]byte) Addr {
    return Addr{
        addr: uint128{
            0,
            0xffff00000000 |
                uint64(addr[0])<<24 | uint64(addr[1])<<16 |
                uint64(addr[2])<<8 | uint64(addr[3])},
        z: z4,
    }
}

Unmap() turns an IPv4-mapped IPv6 address into an IPv4 address by setting the z field to z4:

func (ip Addr) Unmap() Addr {
    if ip.Is4In6() {
        ip.z = z4
    }
    return ip
}

Implementing the reverse direction inside the Go standard library is trivial: we set the z field to z6noz if the address is IPv4.

// To6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func (ip Addr) To6() Addr {
    if ip.Is4() {
        ip.z = z6noz
    }
    return ip
}

Update (2026-10)

Instead of patching Go, we can use the -overlay flag of go build to hijack any package, including the standard library. In our case, it can add To6() to net/netip. Yet it is cumbersome:³ the flag expects a JSON file mapping absolute paths in GOROOT to replacement files, and you need to add it to every go command.

As a helper

We canâ€™t access the z field from outside the net/netip package. Instead, we build a small helper around the netip.AddrFrom16(ip.As16()) pattern:

// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func AddrTo6(ip netip.Addr) netip.Addr {
    if ip.Is4() {
        ip = netip.AddrFrom16(ip.As16())
    }
    return ip
}

As an unsafe function

Another solution uses the unsafe package to alter the Addr struct through a proxy with the same memory layout:⁴

// addrProxy has the same memory layout as netip.Addr.
type addrProxy struct {
    addr [2]uint64      // netip.uint128
    z    unsafe.Pointer // unique.Handle[netip.addrDetail]
}

var (
    anyIPv6    = netip.IPv6Unspecified()
    netipZ6noz = (*addrProxy)(unsafe.Pointer(&anyIPv6)).z
)

// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an
// IPv6 address unmodified.
func AddrTo6(ip netip.Addr) netip.Addr {
    if !ip.Is4() {
        return ip
    }
    (*addrProxy)(unsafe.Pointer(&ip)).z = netipZ6noz
    return ip
}

Benchmarks

On my computer, with Goâ€¯1.27.1, the standard library solution costs 0.88 ns per operation, while the solution favored by Go maintainers costs 7.14 ns. The unsafe solution matches the performance of the first one.

goos: linux
goarch: amd64
pkg: github.com/vincentbernat/go-netip-addrto6
cpu: AMD Ryzen 5 5600X 6-Core Processor
                   â”‚     sec/op     â”‚
AddrTo6/safe            7.137n Â± 0%
AddrTo6/unsafe         0.8682n Â± 2%
AddrTo6/builtin        0.8775n Â± 2%

Assembly code

Letâ€™s check the assembly code the compiler generates for each solution.⁵ The one built into the standard library looks like this:⁶

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ  net/netipÂ·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE   end                      ; if not, stop here
 MOVQ  net/netipÂ·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

Goâ€™s assembly language is not a direct representation of the underlying machine language: it operates on a semi-abstract instruction set derived from Planâ€¯9â€™s assembler. It has four pseudo-registers: FP (frame pointer for function arguments), PC (program counter), SB (static base pointer for global symbols), and SP (stack pointer). It also has architecture-specific registers like AX, CX, DX, BX, SI, DI, and R8 to R15. Instructions storing data use their last argument as the destination. Instructions can carry an explicit size suffix: MOVB moves a byte, MOVWâ€¯16 bits, MOVLâ€¯32 bits, and MOVQâ€¯64 bits. In the example above, the first instruction compares the 64-bit value z4 with the CX register.

The unsafe solution looks almost the same:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ  net/netipÂ·z4(SB), CX  ; check "z" if this is an IPv4 address
 JNE   end                   ; if not, stop here
 MOVQ  netipZ6noz(SB), CX    ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The helper solution has far more instructions. To understand why, letâ€™s look at the code for As16() and AddrFrom16(). They are short enough for the compiler to inline them.

func (ip Addr) As16() (a16 [16]byte) {
    byteorder.BEPutUint64(a16[:8], ip.addr.hi)
    byteorder.BEPutUint64(a16[8:], ip.addr.lo)
    return a16
}

func AddrFrom16(addr [16]byte) Addr {
    return Addr{
        addr: uint128{
            byteorder.BEUint64(addr[:8]),
            byteorder.BEUint64(addr[8:]),
        },
        z: z6noz,
    }
}

We can already guess the pattern to optimize: the code packs the IP address into an array, copies it, then unpacks it. If we inline the Go code by hand, we get:

func AddrTo6(input netip.Addr) netip.Addr {
    if !input.Is4() {
        return input
    }

    var a16 [16]byte
    byteorder.BEPutUint64(a16[:8], input.addr.hi)
    byteorder.BEPutUint64(a16[8:], input.addr.lo)

    addr := a16

    var output netip.Addr
    output.addr.hi = byteorder.BEUint64(addr[:8])
    output.addr.lo = byteorder.BEUint64(addr[8:])
    output.z = netip.z6noz
    return output
}

As humans, we can mentally derive the optimized form:

func AddrTo6(input netip.Addr) netip.Addr {
    if !input.Is4() {
        return input
    }
    var output netip.Addr
    output.addr.hi = input.addr.hi
    output.addr.lo = input.addr.lo
    output.z = netip.z6noz
    return output
}

Unfortunately, as of Goâ€¯1.26.8, the compiler is not smart enough to do the same:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
; Push the stack (32 bytes):
;    0(SP) addr netip.uint128
;   16(SP) a16 [16]byte
 PUSHQ   BP
 MOVQ    SP, BP
 SUBQ    $32, SP

 CMPQ    net/netipÂ·z4(SB), CX  ; check "z" if this is an IPv4 address
 JNE     end                   ; if not, stop here

; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi)
;       byteorder.BEPutUint64(a16[8:], input.addr.lo)
 MOVBEQ  AX, net/netipÂ·a16+16(SP)
 MOVBEQ  BX, net/netipÂ·a16+24(SP)

; addr = a16, 16 bytes at once through the vector register X0
 MOVUPS  net/netipÂ·a16+16(SP), X0
 MOVUPS  X0, net/netipÂ·addr(SP)

; CX = netip.z6noz
 MOVQ    net/netipÂ·z6noz(SB), CX
; Unpack: output.addr.hi = byteorder.BEUint64(addr[:8])
;         output.addr.lo = byteorder.BEUint64(addr[8:])
 MOVBEQ  net/netipÂ·addr(SP), AX
 MOVBEQ  net/netipÂ·addr+8(SP), BX

end:
 ADDQ    $32, SP
 POPQ    BP
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The compiler does a decent job on the byte shuffling: the eight byte stores of BEPutUint64() become a single MOVBEQ, which stores a register byte-swapped. The eight byte loads of BEUint64() become a single MOVBEQ the other way round.⁷ Three groups of instructions remain: a pack, a copy, and an unpack.

Hacking the Go compiler

The Go compiler has several phases:

Parsing: The compiler tokenizes and parses the source code. It builds a syntax tree for each source file.
Type checking: The compiler maps each identifier to the object it denotes, folds constants, and infers the type of every expression.
IR construction: The compiler converts the syntax tree and its types into its own intermediate representation (IR). This process, called â€œnoding,â€� goes through a serialization format named unified IR.
Middle end: The compiler performs several optimization passes on the IR, such as devirtualization, function call inlining, and escape analysis.
Walk: This phase runs two steps: order of evaluation decomposes complex statements into simpler ones, and desugaring transforms higher-level Go constructs, like switch or channels, into more primitive instructions or calls to the runtime.
Generic SSA: The compiler converts the IR into Static Single Assignment (SSA) form, a lower-level intermediate representation suited for machine-independent optimizations and rewrite rules.
Machine code generation: The compiler rewrites the SSA form into machine-specific variants, allocates registers, and applies more optimization passes. At the end, the assembler turns the generated instructions into machine code.

The hammer

My first idea is to replace occurrences of netip.AddrFrom16(ip.As16()) with netip.Addr{addr: ip.addr, z: netip.z6noz} as early as possible, during the â€œnodingâ€� process. Before that, the type checking phase prevents us from accessing unexported struct fields.

Goâ€¯1.27 introduced a convenient debug option to dump the IR of a function at interesting points during compilation:

$ GOTOOLCHAIN=go1.27.1 GOAMD64=v3 go build -a -gcflags="-d=astdump=AddrTo6Safe" .
Writing text ast output for AddrTo6Safe to AddrTo6Safe.ast
Writing html ast output for AddrTo6Safe to AddrTo6Safe.html
Writing html syntax output for AddrTo6Safe to AddrTo6Safe.syntax.html

In the HTML file, the first column shows the IR as it comes out of noding:

DCLFUNC addrto6.AddrTo6Safe ABI:ABIInternal FUNC-func(netip.Addr) netip.Addr
DCLFUNC-Dcl
. NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. NAME-addrto6.~r0 Class:PPARAMOUT Offset:0 OnStack netip.Addr
DCLFUNC-body
. IF # ipv6_safe.go:11:2
. IF-Cond
. . CALLFUNC bool
. . CALLFUNC-Fun
. . . METHEXPR addrto6.Is4 FUNC-func(netip.Addr) bool
. . . . TYPE netip.Addr Class:PEXTERN Offset:0 type netip.Addr
. . CALLFUNC-Args
. . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. IF-Body
. . AS # ipv6_safe.go:12:6
. . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . . CALLFUNC netip.Addr
. . . CALLFUNC-Fun
. . . . NAME-netip.AddrFrom16 Class:PFUNC Offset:0 Used FUNC-func([16]byte) netip.Addr
. . . CALLFUNC-Args
. . . . CALLFUNC ARRAY-[16]byte
. . . . CALLFUNC-Fun
. . . . . METHEXPR addrto6.As16 FUNC-func(netip.Addr) [16]byte
. . . . . . TYPE netip.Addr Class:PEXTERN Offset:0 type netip.Addr
. . . . CALLFUNC-Args
. . . . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. RETURN # ipv6_safe.go:14:2
. RETURN-Results
. . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr

In the body of the if statement, we spot the calls to the method netip.Addr.As16() and to the function netip.AddrFrom16(). Our goal is to patch them with a struct literal:

IF-Body
. AS # ipv6_safe.go:12:6
. . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . STRUCTLIT netip.Addr
. . STRUCTLIT-List
. . . STRUCTKEY netip.addr
. . . . DOT netip.addr netip.uint128
. . . . . NAME-addrto6.ip Class:PPARAM Offset:0 OnStack Used netip.Addr
. . . STRUCTKEY netip.z
. . . . NAME-netip.z6noz Class:PEXTERN Offset:0 unique.Handle[net/netip.addrDetail]

In noderâ€™s reader.go, the expr() method builds the IR tree for an expression. At the end of the exprCall case, we add a call to a rewriteAddrFrom16As16() function. It takes the current node and returns the struct literal on success, or nil if the rewrite is not possible. First, we check that we have the expected pattern: a call to the netip.AddrFrom16() function with a call to the netip.Addr.As16() method as its only argument:

func rewriteAddrFrom16As16(n ir.Node) ir.Node {
    call, ok := n.(*ir.CallExpr)
    if !ok || call.Op() != ir.OCALLFUNC ||
        len(call.Args) != 1 || len(call.Init()) != 0 ||
        !isNetipFunc(call.Fun, "AddrFrom16") {
        return nil
    }
    inner, ok := call.Args[0].(*ir.CallExpr)
    if !ok || inner.Op() != ir.OCALLFUNC ||
        len(inner.Args) != 1 || len(inner.Init()) != 0 ||
        !isNetipFunc(inner.Fun, "Addr.As16") {
        return nil
    }
    x := inner.Args[0]
    // [...]
}

Then, we fetch netip.z6noz:

z6noz, err := lookupVar(ir.StaticCalleeName(call.Fun).Sym().Pkg, "z6noz")
if err != nil {
    return nil
}

And we build the struct literal:

typ := call.Type()
pos := call.Pos()
var list []ir.Node
for i, f := range typ.Fields() {
    var value ir.Node
    switch f.Sym.Name {
    case "addr":
        value = typecheck.DotField(pos, x, i)
    case "z":
        value = z6noz
    default:
        return nil
    }
    list = append(list, ir.NewStructKeyExpr(pos, f, value))
}
lit := ir.NewCompLitExpr(pos, ir.OSTRUCTLIT, typ, list)
lit.SetTypecheck(1)
return lit

Have a look at the complete patch.⁸ We can test it with the following commands:

$ cd src
$ ./make.bash
Building Go cmd/dist using /usr/lib/go-1.27. (go1.27.1 linux/amd64)
Building Go toolchain1 and bootstrap cmd/go (go_bootstrap) using /usr/lib/go-1.27.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 and commands using go_bootstrap and Go toolchain2.
Checking command staleness for linux/amd64.
---
Installed Go for linux/amd64 in /home/bernat/code/free/go
Installed commands in /home/bernat/code/free/go/bin
*** You need to add /home/bernat/code/free/go/bin to your PATH.
$ export PATH=$PWD/../bin:$PATH
$ go version
go version go1.28-devel_9834516e20 Sat Sep 12 08:23:11 2026 -0700 linux/amd64
$ go test net/netip/...
ok      net/netip   0.224s
$ cd ../../go-netip-addrto6
$ go test .
ok      github.com/vincentbernat/go-netip-addrto6   0.062s

The generated code for the helper is now the shortest possible version!

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ    net/netipÂ·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE     end                      ; if not, stop here
 MOVQ    net/netipÂ·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

Go maintainers are unlikely to accept this change. It relies on the internal structure of net/netip.Addr. Itâ€™s an ugly hack in the noder, whose job is to faithfully translate the type-checked AST into the IR. And itâ€™s harder to maintain than adding a To6() method.

The screwdriver

The right place for such an optimization is the generic SSA phase. One of the last machine-independent passes is memcombine. With the appropriate debug flag, the compiler dumps the SSA form after this pass:⁹

$ GOTOOLCHAIN=go1.26.8 GOAMD64=v3 \
> go build -a -gcflags='-d=ssa/memcombine/dump=AddrTo6Safe' .
$ head -5 AddrTo6Safe_01__memcombine.dump
AddrTo6Safe func(netip.Addr) netip.Addr
  b2:
    (?) v1 = InitMem <mem>
    (?) v2 = SP <uintptr>
    (?) v3 = SB <uintptr>

The result of the memcombine pass follows the same structure as the assembly code for AddrTo6Safe() we looked at earlier: two stores, one move, and two loads we would like to optimize away.

; [â€¦]
  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v466 = ArgIntReg <*netip.addrDetail> {ip+16} [2]  ; input.z
; [â€¦]
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)

  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16

  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v416 = Load <uint64> v285 v286                    ; addr[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi
  v174 = Load <uint64> v415 v286                    ; addr[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo
; [â€¦]

Each line features a value identifier (v442), an operation with its type (Bswap64 <uint64>), and its arguments (v490).¹⁰ Values are the basic building blocks of SSA and are defined exactly once. Square brackets enclose integer parameters ([8]) and curly braces contain auxiliary arguments ({netip.addr}). Operations writing to memory produce a new memory state. Every memory operation takes the current state as its last argument, which keeps them in order.

On paper

Letâ€™s focus on output.addr.lo, aka v39:

  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v174 = Load <uint64> v415 v286                    ; addr[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

To simplify this code, we could apply three rewriting rules:

The first one adds a shortcut when loading through a move: (Load (OffPtr [o] p) (Move p src mem)) => (Load (OffPtr [o] src) mem). This matches v174 with its arguments v415 and v286 and creates a new value v600:

  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Load <uint64> v600 v282                    ; a16[8:]
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

The second one simplifies a load following a store: (Load p (Store p x _)) => x. The load is forwarded: the stored value replaces it and no memory access remains. It matches v174. It notices that v600 and v173 are the same address and replaces v174 with a copy of v442:

  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Copy <uint64> v442                         ; bswap(input.addr.lo)
  v39  = Bswap64 <uint64> v174                      ; output.addr.lo

The last step cancels the two byte swaps: (Bswap64 (Bswap64 x)) => x. v39 becomes a copy of v490:

  v490 = ArgIntReg <uint64> {ip+8} [1]              ; input.addr.lo
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v442 = Bswap64 <uint64> v490                      ; bswap(input.addr.lo)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v415 = OffPtr <*byte> [8] v285                    ; &addr[8]
  v600 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v174 = Copy <uint64> v442                         ; bswap(input.addr.lo)
  v39  = Copy <uint64> v490                         ; output.addr.lo = input.addr.lo

If we ignore the values not needed to compute v39, only this SSA form remains:

  v490 = ArgIntReg <uint64> {ip+8} [1]  ; input.addr.lo
  v39  = Copy <uint64> v490             ; output.addr.lo = input.addr.lo

Letâ€™s switch to output.addr.hi, aka v299:

  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Load <uint64> v285 v286                    ; addr[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

To optimize it away, we also apply three rewriting rules:

The first one also adds a shortcut when loading through a move, but without an offset: (Load p (Move p src mem)) => (Load src mem). This rewrites v416 to use arguments from v286:

  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Load <uint64> v22 v282                     ; a16[:8]
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

The second rule forwards a value stored one step earlier, skipping over a store to another address: (Load p (Store q _ (Store p x _))) => x. This matches v416: x is v542, p is v22 (&a16), q is v173 (&a16[8]), and p and q do not overlap for uint64.

  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Copy <uint64> v542                         ; bswap(input.addr.hi)
  v299 = Bswap64 <uint64> v416                      ; output.addr.hi

The third rule cancels two byte swaps: (Bswap64 (Bswap64 x)) => x. v299 becomes a copy of v502:

  v502 = ArgIntReg <uint64> {ip+0} [0]              ; input.addr.hi
  v22 = LocalAddr <*[16]byte> {netip.a16} v2 v1     ; &a16
  v173 = OffPtr <*byte> [8] v22                     ; &a16[8]
  v542 = Bswap64 <uint64> v502                      ; bswap(input.addr.hi)
  v161 = Store <mem> {uint64} v22 v542 v23          ; a16[:8] = bswap(hi)
  v282 = Store <mem> {uint64} v173 v442 v161        ; a16[8:] = bswap(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
  v416 = Copy <uint64> v542                         ; bswap(input.addr.hi)
  v299 = Copy <uint64> v502                         ; output.addr.hi = input.addr.hi

If we remove the values not used to compute v299, we get this SSA form:

  v502 = ArgIntReg <uint64> {ip+0} [0] ; input.addr.hi
  v299 = Copy <uint64> v502            ; output.addr.hi = input.addr.hi

In practice

Most of these rules already exist in generic.rules. They use conditions to validate their context: ssa.IsSamePtr() for the same address, ssa.Disjoint() for addresses that do not overlap. The rule forwarding a stored value to a load already exists with three variants looking through several other stores. Here are the two we need:

(Load <t1> p1 (Store {t2} p2 x _))
    && ssa.IsSamePtr(p1, p2)
    && copyCompatibleType(t1, x.Type)
    && t1.Size() == t2.Size()
    => x
(Load <t1> p1 (Store {t2} p2 _ (Store {t3} p3 x _)))
    && ssa.IsSamePtr(p1, p3)
    && copyCompatibleType(t1, x.Type)
    && t1.Size() == t3.Size()
    && ssa.Disjoint(p3, t3, p2, t2)
    => x

Goâ€¯1.27 added the rule loading through a move with CLâ€¯748200 to fix issueÂ #77720:

(Load <t1> op1:(OffPtr [o1] p1) move:(Move [n] p2 src mem))
    && o1 >= 0 && o1+t1.Size() <= n && ssa.IsSamePtr(p1, p2)
    && !ssa.IsVolatile(src)
    => @move.Block (Load <t1> (OffPtr <op1.Type> [o1] src) mem)

It lacks a variant without an offset:

(Load <t1> p1 move:(Move [n] p2 src mem))
    && p1.Op != ssaop.OpOffPtr
    && t1.Size() <= n && ssa.IsSamePtr(p1, p2)
    && !ssa.IsVolatile(src)
    => @move.Block (Load <t1> (OffPtr <p1.Type> [0] src) mem)

There is no generic rule to cancel two byte swaps, but the AMD64 lowering pass includes this rule:

(BSWAP(Q|L) (BSWAP(Q|L) p)) => p

After switching to Goâ€™s development branch and adding the missing rule, the generated assembly code is worse than with Goâ€¯1.26.8, even though our additional rule slightly improves the situation at the end:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
; Push the stack (16 bytes):
;    0(SP) a16 [16]byte
 PUSHQ   BP
 MOVQ    SP, BP
 SUBQ    $16, SP

 CMPQ    net/netipÂ·z4(SB), CX       ; check "z" if this is an IPv4 address
 JNE     end                        ; if not, stop here

; The four forwarded bytes: the low half of input.addr.lo is taken apart and
; put back together in registers
 MOVQ    BX, DX                     ; DX = input.addr.lo
 SHRQ    $24, BX                    ; BX = input.addr.lo >> 24
 MOVQ    DX, SI                     ; SI = input.addr.lo
 SHRQ    $16, DX                    ; DX = input.addr.lo >> 16
 MOVQ    SI, DI                     ; DI = input.addr.lo, kept for the pack
 SHRQ    $8, SI                     ; SI = input.addr.lo >> 8
 MOVBLZX DIB, R8                    ; R8 = byte(input.addr.lo)
 MOVBLZX SIB, SI                    ; SI = byte(input.addr.lo >> 8)
 SHLQ    $8, SI
 ORQ     R8, SI                     ; SI = two low bytes of input.addr.lo
 MOVBLZX DL, DX                     ; DX = byte(input.addr.lo >> 16)
 SHLQ    $16, DX
 ORQ     SI, DX
 MOVBLZX BL, BX                     ; BX = byte(input.addr.lo >> 24)
 SHLQ    $24, BX
 ORQ     DX, BX                     ; BX = input.addr.lo & 0xffffffff

; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi)
;       byteorder.BEPutUint64(a16[8:], input.addr.lo)
 MOVBEQ  AX, net/netipÂ·a16(SP)
 MOVBEQ  DI, net/netipÂ·a16+8(SP)

; The four other bytes of input.addr.lo, read one by one from a16
 MOVBLZX net/netipÂ·a16+11(SP), DX   ; a16[11]
 SHLQ    $32, DX
 ORQ     DX, BX
 MOVBLZX net/netipÂ·a16+10(SP), DX   ; a16[10]
 SHLQ    $40, DX
 ORQ     DX, BX
 MOVBLZX net/netipÂ·a16+9(SP), DX    ; a16[9]
 SHLQ    $48, DX
 ORQ     DX, BX
 MOVBLZX net/netipÂ·a16+8(SP), DX    ; a16[8]
 SHLQ    $56, DX

; output.z = netip.z6noz
 MOVQ    net/netipÂ·z6noz(SB), CX
; output.addr.hi = byteorder.BEUint64(a16[:8])
 MOVBEQ  net/netipÂ·a16(SP), AX
; output.addr.lo assembled from the previous steps
 ORQ     DX, BX

end:
 LEAVEQ
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

The rule loading through a move, added in Goâ€¯1.27, introduced this regression.

Out of order

Letâ€™s not give up now! In reality, the rewriting rules run before memcombine, notably in the late opt pass. At this point, the inlined versions of BEPutUint64() and BEUint64() still expand to sixteen byte stores and sixteen byte loads, matching their source code:

func BEUint64(b []byte) uint64 {
    _ = b[7] // bounds check hint to compiler; see golang.org/issue/14808
    return uint64(b[7]) | uint64(b[6])<<8 | uint64(b[5])<<16 | uint64(b[4])<<24 |
        uint64(b[3])<<32 | uint64(b[2])<<40 | uint64(b[1])<<48 | uint64(b[0])<<56
}

Letâ€™s follow two bytes of output.addr.lo: addr[15] and addr[11]. Here is a simplified SSA form before late opt:

  v273 = Trunc64to8 <byte> v490                     ; byte(input.addr.lo)
  v226 = Trunc64to8 <byte> v225                     ; byte(input.addr.lo >> 32)
; [â€¦]
  v235 = Store <mem> {byte} v233 v226 v223          ; a16[11] = byte(lo >> 32)
  v247 = Store <mem> {byte} v245 v238 v235          ; a16[12] = â€¦
  v259 = Store <mem> {byte} v257 v250 v247          ; a16[13] = â€¦
  v271 = Store <mem> {byte} v269 v262 v259          ; a16[14] = â€¦
  v282 = Store <mem> {byte} v280 v273 v271          ; a16[15] = byte(lo)
  v285 = LocalAddr <*[16]byte> {netip.addr} v2 v282 ; &addr
  v286 = Move <mem> {[16]byte} [16] v285 v22 v282   ; addr = a16
; [â€¦]
  v433 = OffPtr <*byte> [15] v285                   ; &addr[15]
  v435 = Load <byte> v433 v286                      ; addr[15]
  v479 = OffPtr <*byte> [11] v285                   ; &addr[11]
  v481 = Load <byte> v479 v286                      ; addr[11]

The first rule loads through the move: (Load (OffPtr [o] p) (Move p src mem)) => (Load (OffPtr [o] src) mem). It matches both loads, which now read a16 with the memory state before the copy:

  v600 = OffPtr <*byte> [15] v22 ; &a16[15]
  v435 = Load <byte> v600 v282   ; a16[15]
  v601 = OffPtr <*byte> [11] v22 ; &a16[11]
  v481 = Load <byte> v601 v282   ; a16[11]

The second rule shortcuts a load following a store: (Load p (Store p x _)) => x. It matches v435, as v282 stores a16[15]. It does not match v481: v235 stores a16[11] four stores earlier in the chain, while the variants of this rule look through three stores at most.

  v435 = Copy <byte> v273        ; byte(input.addr.lo)
  v601 = OffPtr <*byte> [11] v22 ; &a16[11]
  v481 = Load <byte> v601 v282   ; a16[11]

The same happens to the other bytes: the rule forwards the four bytes stored last, a16[12] to a16[15]. The twelve other loads now read a16 instead of addr.

BEUint64() becomes a chain of Or64, each one adding a byte shifted into place. memcombine is a pass written in Go, not a set of rewrite rules. It starts from the last Or64 of the chain and collects up to eight terms. If each term is a byte load, extended to 64 bits and shifted, and if the eight loads read consecutive addresses from the same pointer with the same memory state, it replaces the whole chain with a single 64-bit load and a byte swap. Otherwise, it tries again with four, then two terms, and from each intermediate Or64. Here is the loop checking each term in a simplified version of combineLoads():

for i := int64(0); i < n; i++ {
    v := a[i]
    shift := int64(0)
    if v.Op == shiftOp {
        v, shift = peelShift(v)
    }
    if v.Op != extOp {
        return false
    }
    load := v.Args[0]
    if load.Op != ssaop.OpLoad {
        return false
    }
    if load.Args[1] != mem {
        return false
    }
    p, off := splitPtr(load.Args[0])
    if p != base {
        return false
    }
    r[i] = LoadRecord{load: load, offset: off, shift: shift}
}

For output.addr.hi, the eight loads read a16 with the same memory state v282:

  v13  = Load <byte> v22 v282               ; a16[0]
  v530 = Load <byte> v14 v282               ; a16[1]
  v488 = Load <byte> v504 v282              ; a16[2]
  v405 = Load <byte> v537 v282              ; a16[3]
  v385 = Load <byte> v397 v282              ; a16[4]
  v361 = Load <byte> v373 v282              ; a16[5]
  v196 = Load <byte> v63 v282               ; a16[6]
  v432 = Load <byte> v315 v282              ; a16[7]
  v319 = ZeroExt8to64 <uint64> v432         ; uint64(a16[7])
  v329 = ZeroExt8to64 <uint64> v196         ; uint64(a16[6])
  v330 = Lsh64x64 <uint64> [true] v329 v138 ; uint64(a16[6]) << 8
  v331 = Or64 <uint64> v319 v330            ; a16[7] | a16[6] << 8
; [â€¦] same for a16[5] to a16[1]
  v401 = ZeroExt8to64 <uint64> v13          ; uint64(a16[0])
  v402 = Lsh64x64 <uint64> [true] v401 v55  ; uint64(a16[0]) << 56
  v403 = Or64 <uint64> v402 v391            ; | a16[0] << 56 = output.addr.hi

memcombine merges them into one load and a swap:

  v286 = Load <uint64> v22 v282   ; a16[:8]
  v285 = Bswap64 <uint64> v286    ; output.addr.hi

For output.addr.lo, here is the chain memcombine sees after late opt:

  v436 = ZeroExt8to64 <uint64> v273 ; addr[15], forwarded
  v448 = Or64 <uint64> v436 v447    ; | addr[14] << 8, forwarded
  v460 = Or64 <uint64> v459 v448    ; | addr[13] << 16, forwarded
  v472 = Or64 <uint64> v471 v460    ; | addr[12] << 24, forwarded
  v484 = Or64 <uint64> v483 v472    ; | a16[11] << 32, loaded
  v496 = Or64 <uint64> v495 v484    ; | a16[10] << 40, loaded
  v508 = Or64 <uint64> v507 v496    ; | a16[9] << 48, loaded
  v520 = Or64 <uint64> v519 v508    ; | a16[8] << 56, loaded

From v520, four of the eight terms are forwarded bytes, not loads from memory, and memcombine canâ€™t combine them. It doesnâ€™t merge the four remaining loads either, as they sit on top of the forwarded bytes.

Back in order

In summary, the rewriting rules run too early to be effective. A quick workaround exists: run an earlier round of memcombine before late opt. After this change, the generated code for the helper is back to the shortest possible version:

// AX = input.addr.hi, BX = input.addr.lo, CX = input.z
 CMPQ    net/netipÂ·z4(SB), CX     ; check "z" if this is an IPv4 address
 JNE     end                      ; if not, stop here
 MOVQ    net/netipÂ·z6noz(SB), CX  ; CX = netip.z6noz
end:
 RET
// return value = Addr{hi: AX, lo: BX, z: CX}

And the benchmark confirms it! âœŒï¸�

goos: linux
goarch: amd64
pkg: github.com/vincentbernat/go-netip-addrto6
cpu: AMD Ryzen 5 5600X 6-Core Processor
                   â”‚   Go 1.26.8    â”‚             Our branch              â”‚
                   â”‚     sec/op     â”‚    sec/op     vs base               â”‚
AddrTo6/safe          6.5470n Â±  0%   0.8944n Â± 4%  -86.34% (p=0.002 n=6)
AddrTo6/unsafe        0.9071n Â±  3%   0.8682n Â± 1%   -4.28% (p=0.002 n=6)
AddrTo6/builtin       0.8871n Â±  2%   0.8785n Â± 1%        ~ (p=0.310 n=6)

Next steps

I think Go maintainers would reject this change because of the additional memcombine pass. Instead, I plan to publish this blog post and bring up the subject again as a follow-up to issue #54365. Either the sheer complexity and the Goâ€¯1.27 regression convince the maintainers that adding a To6() method is simpler and more efficient, or they advise me on how to move forward. Either way, digging into this subject taught me a lot about the Go compiler! âš™ï¸�

Update (2026-10)

I opened issue #81994 to propose Addr.To6(). Give it a ğŸ‘� if you want it in Go!

IPv4-mapped IPv6 addresses let you handle only IPv6 addresses in your code and keep conversions at a few well-defined boundaries. I use them in Akvorado.Â â†©
This is not strictly equivalent. The zero value becomes :: while it would make more sense to leave it untouched.Â â†©
I suppose Go maintainers keep it inconvenient on purpose: they donâ€™t want random packages to patch the standard library or other packages.Â â†©
To avoid catastrophic bugs when netip.Addrâ€™s layout changes, you must add tests to detect it. This is more dangerous if you put this code in a package. Either ask users to run the tests themselves, or detect the change at run time and panic. Hiding this optimization behind a build tag would make users aware of this potential trap.Â â†©
I produced the assembly code with GOTOOLCHAIN=go1.26.8 GOAMD64=v3 ./go-asm '\.asm' from the companion repository, then edited it a bit to keep this article from turning into an endless rabbit hole. Due to an unfortunate sequence of events, we switch between versions: Goâ€¯1.27 introduced a regression that blurs the point of this article.Â â†©
The function is short enough for the compiler to inline it, so the assembly code may vary depending on the surrounding code.Â â†©
MOVBEQ requires GOAMD64=v3, matching the x86-64-v3 microarchitecture from 2013. Otherwise, the compiler translates BEPutUint64() to BSWAPQ+MOVQ and BEUint64() to MOVQ+BSWAPQ.Â â†©
When the call is used as a statement, a struct literal alone is invalid. The patch then turns it into _ = netip.Addr{â€¦}.Â â†©

The compiler can also write an HTML file with the result of each pass:

$ GOSSAFUNC=AddrTo6Safe \
> GOTOOLCHAIN=go1.26.8 \
> GOAMD64=v3 go build -a .
dumped SSA for AddrTo6Safe,1 to ./ssa.html

â†©

Source line numbers in parentheses follow value identifiers, but I removed them from the examples. The SSA form also features branches, but we donâ€™t need them to understand our optimizations.Â â†©

Steinar H. Gunderson: Decompilation patterns part 1: Anti-CSE

Sun, 04 Oct 2026 09:16:00 +0000

There's a lot of¹ interest in decompilation these days; taking some old piece of software (usually a game) and try to reconstruct C that does the same thing. (This is often for modding purposes, but also often for understanding, or just as a personal challenge.) An often sought-after property is matching; that the decompiled C produces 100% the same bytes as the original after compilation. (This means you'll need to find the original compiler and build flags, of course.) Typically, one starts with the disassembled code and runs m2c on it, which then spits out some C that is fairly close to the original, but rarely produces a match without further cleanup. (It is usually much closer to a match than what Ghidra or Hex-Rays would output; Ghidra is made for ease of understanding, not matching, so it tends to simplify a lot.)

Cleaning up the code for matching has two main advantages:

It usually makes the source easier to understand, and
Matching is a very strong argument (though not infallible) that the decompilation is actually correct.

Like with pretty much any other field these days, there is a lot of interest in giving this task to some LLM, which means there's now a lot of decompilations around that match (fitting #2 perfectly) and completely fail #1. :-) Anyway, at some point, I learned that there is almost no public documentation on the cleaning-up-for-matching part; it's all folklore and people rediscovering the same patterns. So I thought I'd do something about that².

My general goal is to write up one decompilation pattern a day (focusing on what I know best, namely GCC on MIPS), with some example. Most of them will be very simple; some are much less obvious. I'll keep going until I run out of ideas. None are formally named, so I'll just invent a name. So here goes part one:

  temp_v0_5 = g_state.players[1].unk2;
  if ((temp_v0_5 != 7) && (temp_v0_5 != 0)) {

What happened here is usually that the compiler has done common subexpression elimination; someone wrote the same thing twice and then the compiler decided to just calculate it once and put it into a temporary variable. (Or the programmer did, in which case you would be wise to leave it alone! Getting into the original programmer's head is part of the challenge.) This will often affect register allocation and/or stack layout, so to get a better match, you could try doing CSE in reverse:

  if (g_state.players[1].unk2 != 7 && g_state.players[1].unk2 != 0) {

Nothing is mechanical; it may help or it may not. (It may also appear to hurt at first and then help later, or vice versa.) But anti-CSE is generally very common and you'll want to have it in your arsenal.

¹ Well, all is relative.

² Reluctantly, with the understanding that the LLMs might also learn from it.

Vincent Bernat: Self-hosted HTTP tunnels with SSH and nginx

Sat, 03 Oct 2026 07:23:12 +0000

A friend wants to proofread your work-in-progress blog post, but its preview only runs on localhost:8080. Several tools can help. Some run as a commercial service, like ngrok or Cloudflare Quick Tunnels. Some are self-hostable but require a specific client, like frp or localtunnel. Some only require a plain SSH client but rely on a specific SSH server, like sish. Let’s implement a self-hosted solution with only OpenSSH and nginx!

$ ssh -R 0:localhost:8080 http-over-ssh
Allocated port 41535 for remote forward to localhost:8080
https://6J3jK1WmB15c6WmjW_X-Wg--1789928654@p41535.ssh.luffy.cx/

Basic setup

First, we forward connections from a port on a remote server to your local service:

$ ssh -N -R 0:localhost:8080 web02.luffy.cx
Allocated port 41535 for remote forward to localhost:8080

When you specify 0 as the remote port, the server allocates a free port. Then, we configure nginx to proxy requests from https://p41535.ssh.luffy.cx to http://127.0.0.1:41535:

server {
  listen 0.0.0.0:443 ssl ;
  listen [::0]:443 ssl ;
  server_name ~^p(?<port>\d\d\d\d\d)\.ssh\.luffy\.cx$;
  location / {
    proxy_pass http://127.0.0.1:$port;
  }
}

We also need to add DNS records for *.ssh.luffy.cx and get a wildcard certificate through Let’s Encrypt:

*.ssh.luffy.cx.               CNAME web02.luffy.cx.
ssh.luffy.cx.                 CAA   0 issuewild "letsencrypt.org"
_acme-challenge.ssh.luffy.cx  CNAME ssh.luffy.cx.acme.luffy.cx.

acme.luffy.cx is a zone hosted on Route 53. I use it for ACME DNS-01 challenges, both for wildcard certificates and for domains served by several web servers. In my case, NixOS gets the certificates automatically.

Access control

The port is the only “secret”¹ keeping the content confidential. Other forwarding solutions add a random string to the domain name to prevent an intruder from enumerating the possible values.

Thanks to ngx_http_secure_link_module, we can secure this setup a bit. This module computes a hash² over a set of values, including a secret, and compares it with the hash from the request. The hash is base64-encoded, so we cannot put it in the domain name, which is case-insensitive. Instead, we put it in the URL as a username, along with its expiration timestamp:³

https://6J3jK1WmB15c6WmjW_X-Wg--1789928654@p41535.ssh.luffy.cx/en/blog
        ╰─────────┬──────────╯  ╰───┬────╯  ╰─┬─╯             ╰──┬───╯
                hash             expires    port               path

The client sends the username to the server with HTTP basic authentication. This works with most HTTP clients, including curl. Nginx exposes the username in the $remote_user variable. The module expects the hash and the expiration timestamp separated by a comma. We use a map directive to extract the two parts from $remote_user and join them with a comma.⁴ We also give the module the string to hash. It contains the expiration timestamp, the port, and a secret:

map $remote_user $httpssh_link {
  "~^([-_A-Za-z0-9]{22})--([0-9]+)$" "$1,$2";
}
server {
  # […]
  location / {
    secure_link $httpssh_link;
    secure_link_md5 "$secure_link_expires $port ZuPerS3cr3!";
  }
}

The module returns the status of the check in the $secure_link variable:

empty if the hashes do not match,
"0" if they match but the link has expired, or
"1" otherwise.

If the hash is incorrect or missing, we return a 401 error with a WWW-Authenticate header to ask for credentials. If the link has expired, we return a 410 error. We remove the Authorization header before forwarding the request and add a few directives to proxy WebSocket connections. Here is the complete configuration:⁵

map $remote_user $httpssh_link {
  "~^([-_A-Za-z0-9]{22})--([0-9]+)$" "$1,$2";
}
server {
  listen 0.0.0.0:443 ssl ;
  listen [::0]:443 ssl ;
  server_name ~^p(?<port>\d\d\d\d\d)\.ssh\.luffy\.cx$;
  location / {
    secure_link $httpssh_link;
    secure_link_md5 "$secure_link_expires $port ZuPerS3cr3!";
    if ($secure_link = "") {
      add_header WWW-Authenticate 'Basic realm="tunnel"' always;
      return 401;
    }
    if ($secure_link = "0") {
      return 410;
    }
    proxy_pass http://127.0.0.1:$port;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Authorization "";
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_buffering off;
    proxy_read_timeout 30m;
  }
}

I think you are now asking yourself the obvious question: “How should I generate the hash?” Easy peasy!

$ expires=$(( $(date +%s) + 86400 ))
$ port=41535
$ secret='ZuPerS3cr3!'
$ printf '%s %s %s' "$expires" "$port" "$secret" \
>   | openssl md5 -binary \
>   | openssl base64 \
>   | tr +/ -_ | tr -d =
6J3jK1WmB15c6WmjW_X-Wg

Well, I suppose you are now saying: “Vincent, this is not very convenient! I’ll stick with ngrok if you don’t mind.” Okay, I hear you. Let’s write a helper script.

Helper script

The main difficulty is finding the ephemeral port that OpenSSH allocates, as it does not appear in any environment variable.⁶ To work around this obstacle, we look for the ancestor sshd-session processes:⁷

pids=$(
  pid=$$
  while [ "$pid" -gt 1 ]; do
    line=$(ps -o comm=,pid=,ppid= -p "$pid")
    echo "$line"
    pid=${line##* }
  done | awk '$1 == "sshd-session" { printf "pid=%s,\n", $2 }'
)
if [ -z "$pids" ]; then
  echo "not an ssh session" >&2
  exit 1
fi

Then, we get the listening ports associated with these sshd-session processes:⁸

ports=$(sudo -n ss --listening --numeric --tcp --processes --no-header \
  | grep -F "$pids" \
  | awk '{ print $4 }' | awk -F: '{ print $NF }' \
  | sort -un)
if [ -z "$ports" ]; then
  echo "no forwarded port, use ssh -R 0:localhost:PORT" >&2
  exit 1
fi

Finally, we display the URLs and keep the session open:

lifetime=86400
secret='ZuPerS3cr3!'
expires=$(( $(date +%s) + lifetime ))
for port in $ports; do
  token=$(printf '%s %s %s' "$expires" "$port" "$secret" \
            | openssl md5 -binary \
            | openssl base64 \
            | tr +/ -_ | tr -d =)
  echo "https://$token--$expires@p$port.ssh.luffy.cx/"
done
sleep infinity

I install this script as http-over-ssh on the server and add this entry to my ~/.ssh/config:

Host http-over-ssh
  Hostname web02.luffy.cx
  RemoteCommand http-over-ssh
  ControlPath none

With this solution, I only rely on OpenSSH and nginx, two pieces of software already running on this server. One short command gives me a self-hosted tunnel and a URL to share. To try it, grab the complete helper script, which includes a few minor improvements. If you run NixOS, as any person of taste would, have a look at my http-over-ssh.nix instead. ❄️

Since the kernel allocates it from the local port range, its entropy is low. Moreover, on Linux, we lose one more bit because the kernel favors odd ports when it picks a random free port.
```
$ sysctl -n net.ipv4.ip_local_port_range \
>   | awk '{print $1"—"$2" ≈ "log($2-$1+1)/log(2)" bits"}'
32768—60999 ≈ 14.785 bits
```
↩
The module relies on MD5. Its security is weak but good enough for this use. ↩
We need an expiration date because a later session could reuse the port, and we have no way to detect this case. ↩
The username could use a comma directly instead of a double dash. But some applications do not recognize the URL correctly, making it harder to share. ↩
This configuration exposes any TCP port listening on 127.0.0.1 or 0.0.0.0 to anyone with the secret, bypassing most firewall rules. You could strengthen it by limiting the server_name regex to the ephemeral port range. ↩
One SSH session can contain several tunnels, and the client can add or remove them at any time. This is unlike tun device forwarding (ssh -w), which gets its own SSH_TUNNEL environment variable. ↩
Since OpenSSH 9.8, sshd-session is the name of the helper handling the session. With older versions, look for sshd instead. ↩
We need sudo because the sshd-session process dropped its privileges. The kernel then marks it as not dumpable, and its /proc/PID/fd directory belongs to root. Without access to this directory, ss cannot find which process owns a socket. ↩

Ben Hutchings: FOSS activity in September 2026

Fri, 02 Oct 2026 14:34:15 +0000

Debian packages:
- firmware-nonfree:
  - Bugs:
    - closed #1148284: firmware-amd-graphics: AMD AV1 hardware decoding produces severe video artifacts with firmware 20260810-1
  - Merge requests:
    - opened and merged !154: Update to 20260916; fix some license information
  - Uploads:
    - uploaded version 20260810-1~bpo13+1 to trixie-backports
    - uploaded version 20260916-1 to unstable
- initramfs-tools:
  - Merge requests:
    - closed !202: mkinitramfs: fix path to kernel config in /lib/modules
- kernel-team:
  - Merge requests:
    - merged !14: find-recent-urgent: Sort bugs from linux first
    - opened !15: Add form responses for common problems with bug reports
- linux:
  - Bugs:
    - closed #1146373: linux-libc-dev: sbrk(2) allocates more than memory + swap space
  - Merge requests:
    - closed !2010: [x86] Backport security fixes for KVM (LTS)
    - merged !2083: linux-image: Bump Pre-Depends version on linux-base (>= 4.17~)
    - reviewed and merged !2095: 6.1 stable updates (LTS)
    - merged !2108: [alpha] Reenable CONFIG_COMPACTION
    - merged !2109: Enable BMI323 support as a module
    - merged !2110: riscv64 config update for linux 7.3
  - Uploads:
    - uploaded version 7.1.13-1~bpo13+1 to trixie-backports
    - uploaded version 7.2.6-1~bpo13+1 to trixie-backports
- linux-6.1 (LTS):
  - Bugs:
    - closed #1117674: Please enable udebs for the 6.1 backport
- linux-6.12 (LTS):
  - Uploads:
    - uploaded version 6.12.107-1~deb12u1 to bookworm-security
  - merged changes from trixieâ€™s 6.12.111-1
- linux-base:
  - Uploads:
    - uploaded version 4.17~bpo13+1 to trixie-backports
- nfs-utils:
  - Merge requests:
    - reviewed !37: Install and use sysusers.d/tmpfiles.d config files
- wireless-regdb:
  - Uploads:
    - uploaded version 2026.09.03-1 to unstable
Mailing lists:
- debian-kernel:
  - posted Agenda items for kernel-team meeting on 2026-09-30
  - replied to Meta packages for modules
  - replied to Updating linux-6.12 (bookworm-security) to 6.12.107 (LTS)
- debian-lts-announce:
  - posted [SECURITY] [DLA 4788-1] linux-6.12 security update

Junichi Uekawa: Tokyo has been raining for many days but finally there's some sunny days.

Fri, 02 Oct 2026 07:56:37 +0000

Tokyo has been raining for many days but finally there's some sunny days.

Dirk Eddelbuettel: RcppFastAD 0.0.6 on CRAN: Agentic Fix

Thu, 01 Oct 2026 17:05:00 +0000

A week ago, release 0.0.5 of the RcppFastAD package went to CRAN, and we noted that it provided a stop-gap measure against a gnarly segfault with g++-16 under heavier optimization / inlining. We are happy to now announce that a new and fixed version 0.0.6 of RcppFastAD is now on CRAN, and has been built for r2u as well as at r-universe.

This release takes advantage of some large-language model inference access I now get via Debian. The Deepseek v4 flash-0731 model, driven in a container via the pi.dev harness, took just about no time finding a small workaround for the issue described last week where we could see segfaults under heavier optimization and inlining. The fix was to prohibit inlining the local bind() reimplementation. We have also sent this upstream as a patch. It is nice to have an option of querying something a little more powerful than the local models I experimented with before. We suppress more compiler warnings in local builds via another option, and did some standard maintenance.

The NEWS file for this release follows.

Changes in version 0.0.6 (2026-10-01)

The expression-template view rebinding is protected against g++ -O3 (which led to wrong Eigen::Map dimensions, an Eigen::Map resize assertion, and a segfault.

The bind() helpers are marked non-inline and are also protected from operating on a nullptr.

Both items co-written with DeepSeek V4 Flash, and have been sent upstream in a PR.

Suppress another compiler warning when local build detected.

Add r-universe badge to README.md.

Courtesy of my CRANberries, there is also a diffstat report for the most recent release. More information is available at the repository or the package page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Daniel Pocock: Reform UK, Nigel Farageâ€™s party/company, admits missing [redacted]

Thu, 01 Oct 2026 10:00:00 +0000

In my other blog today, I explored how the other Reform UK Party Limited party/company was previously subject to a judgment and insolvency proceedings to try and extract payment.

Very few companies are ever listed in the official Gazette.

After that painful lesson, as they progress towards being in a position to run the UK Government, wouldn't you expect them to improve their procedures and run their party headquarters competently?

When Sarah Cooper-Lesadd recently resigned and defected from Reform UK, she told the BBC that Reform UK is chaotic.

In the Scotsman's report about the VMS Enterprises vs Brexit Party dispute, the party was described as "beyond chaotic". They can't seem to learn how to tie their shoelaces but they want to run the United Kingdom. If Nigel Farage does ever become Prime Minister, I hope his party remembers to pass on the messages when Argentina makes their next attempt to take the Falkland Islands.

It looks like Sarah Cooper-Lesadd is right. They admit despite having one of the most serious legal documents, a [redacted], served on their party headquarters, they have no idea where it went:

Fortunately, the process server provide video evidence so I can prove it was served correctly and the incompetence is not on my side.

In fact, the case only reached this point because of a litany of errors on the part of Reform UK themselves. Errors they should never have made if they learnt the lesson from the somebody who brought their party into insolvency proceedings in 2022.

Look at the cost. Their lawyers, Aston Bond, have invoiced over £25,000 before the first hearing. Most of that is for investigating what went wrong inside their own organisation and how the dispute was leaked by the MouseInTheCourt before the [redacted] ever arrived on the doorstep of party HQ.

Why is Nigel Farage so so afraid of a candidate who only got sixteen votes?

Daniel Pocock: Creditors beware: VMS Enterprises Ltd vs Brexit Party (Reform UK Party Ltd)

Thu, 01 Oct 2026 07:30:00 +0000

As Reform UK tries to [redacted], they have complained to the court that [redacted] other failed candidates, Reform UK defectors, independent contractors and small business owners how to [redacted].

If they block more people [redacted]

It is not my intention to publish the forms unless the conduct and behaviour of these people forces me to do so.

Judgment: VMS Enterprises Ltd vs The Brexit Party Ltd ([2021] SC GLA 49)

The Brexit Party Ltd became Reform UK Party Ltd, company number 11694875. This subsequently got merged into Reform 2025 Ltd to become a different Reform UK Party Limited company no. 16260766. An inconvenient consequence of this name change and merger is that people searching for judgments against the new company number 16260766 don't see the history of the old company name and company number.

The full judgment is available here for download as an Acrobat document:

VMS Enterprises Ltd won the case but it took them over a year to get paid after the judgment. The records about VMS Enterprises Ltd (SC476412) at Companies House show us they are no longer active. It looks like beating Brexit Party / Reform UK in court was only a phyrric victory.

Not only was there a judgment ([2021] SC GLA 49), there were also some scholarly research articles published about the dispute:

On 10 September 2021, Scottish Legal News published a detailed summary "Glasgow sheriff orders Brexit Party to pay £22k in unpaid invoices to advertising firm".

On 15 November 2021, the case appeared again in a Law Society of Scotland article about case management review.

Despite the Sheriff making a judgment against the Reform people in July 2021, they still didn't pay the bill. On 19 July 2022, The Scotsman reported "Reform UK face winding up petition over £22,000 debt owed to Scottish firm".

Quoting the article:

The political party, formerly led by Nigel Farage as the Brexit Party during the 2019 general election, faces a hearing in the High Court in London as early as September. Victor Shields, director of VMS Enterprises, was forced to take the party – since rebranded and led by Richard Tice – to court over the bill from the 2019 general election. Despite Glasgow Sheriff Court agreeing the party should pay up, it has consistently refused to answer letters or pay the sum owed to the businessman, forcing him to take further legal action in England. It is understood a winding up petition was served on the political party last week, with a hearing set for September 7 or soon thereafter. Reform UK has consistently rejected the claim, now worth more than £26,000 after interest and before court expenses are taken into account. A winding up order, if granted by the court, would see the party face liquidation proceedings to pay the outstanding amount and could include an investigation into director conduct. The initial Scottish court case arose after a dispute between the party and VMS Enterprises around payment for election billboards in London during the general election campaign. The court stated the party was liable for the bill for advertising services after some invoices were paid by Brexit Party staff and others were not questioned. Several invoices had gone unpaid amid claims a volunteer did not have the authority to charge the party for the billboards. However, the court stated the failure to question the payment of invoices incurred by candidates meant the party was liable for the full costs. Speaking to The Scotsman last year, Mr Shields attacked Reform UK’s “hypocrisy” in failing to pay. He said: “I was quite happy to keep quiet after the judgment because I assumed they would respect the Scottish courts and duly pay us. "They don't care. They feel they are above the law. "Richard Tice is positioning himself to the right of Boris [Johnson] and trying to capture the Conservative voters, Scottish voters too and here we are being forced to go to English courts, for me being forced to spend further money, to try and recover a debt that is legally due to us." The court case heard witnesses describe the organisation of the Brexit Party, then led by former UKIP leader Nigel Farage, as "shambolic" by a party official as it struggled to cope with its popularity and decision to pull hundreds of candidates from the ballot paper ahead of polling day. Witnesses told the court the party’s organisational structure "visibly started to fall apart", was "very unprofessional" and "beyond chaotic" ahead of the 2019 election. Reform UK’s latest accounts state that in 2020 it received nearly £3 million in donations, with a further £17m in 2019. Reform UK has been contacted for comment by The Scotsman.

Eighteen months after the judgment, almost two years after the judicial procedure began, in December 2022, VMS Enterprises Ltd used the High Court insolvency procedure to publish an insolvency notice for Reform UK Party Limited.

As a consequence, a listing was placed in the official Gazette with their old company number 11694875 and the High Court case number CR-2022-003973.

Diagram demonstrating evolution of the Reform UK companies

Searching for information about the names and company numbers is tedious and time consuming. I created a colour-coded diagram so people can see and understand the relationship between the companies in the history of Reform UK at a glance.

Here is what you see if you search for Reform UK at Companies House. Which Reform UK is which?

Looking through the filing history of each company, you can see many minor changes, including some name changes. I put the filing history for both companies side-by-side:

Drilling down on the name changes, it is fascinating to put the two forms side-by-side like this:

Here is the full diagram I created to help people understanding the judgment and insolvency court history of Reform UK and its leaders at a glance:

Direct links to search the Gazette for old company number 11694875 and for the new company number 16260766.

People who have a legitimate claim against any company are encouraged to contact their solicitor for advice and share their contact details with other creditors to resolve the dispute more expediently. See my previous blog about how to join a creditors petition in the UK.

Daniel Pocock: Reform UK last minute accounts filing

Wed, 30 Sep 2026 22:15:00 +0000

On 25 June 2026, Reform UK, the political party, filed accounts for the 2025 calendar year with the Electoral Commission.

Reform UK is the only major political party in the UK that is also registered as a company at Companies House.

Consequently, each year, they must file a set of accounts, with an auditor's report, at Companies House. The deadline was today, 30 September 2026. Why did they file the accounts at the very last minute and not at the same time they filed with the Electoral Commission?

Click here to see the filing history and access the other accounts.

As explained in my previous blog on the subject, the reports given to Companies House are slightly different from the reports given to the Electoral Commission. The Electoral Commission is trying to show the impact of money on democracy. The Companies House report aims to give members, investors, trading partners and employees an image of the organisation's strength or weakness.

The document uploaded to the Electoral Commission in June is a text-searchable PDF. The PDF uploaded to Companies House is not text-searchable. However, Companies House also has another copy in iXBRL, which is an XHTML file that is text-searchable in a web browser.

At a glance, we can see the numbers are not exactly the same. For example, the membership revenue in the report to the Electoral Commission appears to be a bit higher because they used to collect the money through a different company and then they subsequently merged the two companies during the 2025 reporting year.

Like the report submitted to the Electoral Commission, the report submitted to Companies House does not mention the potential impact of the new, backdated restrictions on donations from overseas electors. If those restrictions are implemented soon, they will have a very severe impact on Reform UK. It seems odd to me they can completely omit that from the statement about the company being a going concern for the next 12 months.

One of the very significant differences in the Companies House report submitted today is on the very last page. It is the declaration about how they merged the two former companies and who really has control over the company, which is acting as a political party but is really a business:

19. Related party transactions

On 19 February 2025, the company acquired the entire issued share capital of Reform 2025 Ltd for consideration of £15.

...

20. Controlling party

The company is limited by guarantee and has no share capital. The company has two members, who have equal voting rights. Neither member has individual control of the company and, accordingly, there is no ultimate controlling party.

...

Legal form of entity and country of incorporation

...

The company is limited by guarantee and has no share capital. The liability of each member is limited to £1, being the amount that each member undertakes to contribute to the assets of the company in the event of its winding up. There were two members at the year end.

Looking at the full list of documents on the Companies House web site, we can find the first document filed when the company was created. They tell us the two real members of the company are Nigel Farage and Zia Yusuf.

Companies limited by guarantee in the UK are a special type of company that is normally used for non-profits and charities. In some non-profits, like sporting clubs, every member is really a member of the company and invited to the annual general meeting.

The accounts that Reform UK has just filed show us they have a two-tier membership structure, similar to the scandal of the FSFE misfits.

People who "join" Reform UK and pay £25 per year are not really members like Nigel Farage and Zia Yusuf are members in the accounts.

In UK law, the real members, Nigel Farage and Zia Yusuf, can choose to change it into a normal shareholder company at any time they want and then they can sell the shares or pass them to their children, as long as the articles of association permit that.

In June 2025, Zia Yusuf resigned from Reform UK and came back two days later. The accounts show us that he resigned from his position on the board but they do not tell us if he transferred his position as one of the two controlling members to somebody else.

Reform UK claims to have more members than any other political party in the UK. These accounts tell us their company only has two members, making it one of the smallest political parties at the same time as being one of the biggest.

If the members accept this two-tier structure, does that mean it really is a cult, as we were told by Rupert Lowe?

When they changed the structure, they told people they were making it more democratic. What they really did is they reduced their risk. If it fails, they only lose their guarantee, which is £1 each. If it succeeds, these two people can choose to pivot back to a shareholder company form, issue shares and sell them on the stock market.

The other thing they did through this reoganisation is obfuscating the history of disputes and judgments against the former company. It looks like at least one law suit is still progressing against the old company and it is not clear if the new company structure will be obliged to pay for it if they lose.

The UK permits directors, company secretaries and shareholders to be nominees, in other words, proxies contracted to obey the person who really pays them to perform that role. While Nigel Farage and Zia Yusuf are listed publicly as the members who founded the new entity in February 2025, there is no obligation for them to publicly identify any transfers of these memberships to other people. The only thing we can deduce from the report is that there are currently two members, the same number of members they had when they created the entity but not necessarily the same two people.

Dirk Eddelbuettel: RcppFastFloat 0.0.6 on CRAN: New Upstream

Wed, 30 Sep 2026 21:17:00 +0000

A new release of RcppFastFloat just arrived on CRAN, and has been built for r2u. The package wraps fast_float, another nice library by Daniel Lemire. For details, see the arXiv preprint or published paper showing that one can convert character representations of ‘numbers’ into floating point at rates at or exceeding one gigabyte per second.

This release updates the underlying fast_float library version to the current version 8.3.0, and also updates a number of the standard packaging details.

Changes in version 0.0.6 (2026-09-29)

Update to upstream release 8.3.0 (Daniel Lemire in #6)

Standard package maintenace adding README badge, updating URL, updating continuous integration, and adding an aspell dictionary

Completed help page for as_double example with @return tag

Courtesy of my CRANberries, there is also a diffstat report for this release. For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Daniel Pocock: High Court victory: Nigel Farage is ‘the Company’s own candidate’

Wed, 30 Sep 2026 19:45:00 +0000

In the sentencing of the late Cardinal George Pell, it was his defence lawyer Robert Richter KC who argued the Cardinal should receive a lighter punishment because it was a "plain vanilla" case of abuse. In fact, when somebody in such a senior position of trust commits an offence of that nature, it is a lifelong case of psychological torture for the victims. That is what makes institutional abuse, such as the abuse by jurists and the abuse by the Catholic Church so much worse than other types of abuse.

The Cardinal was later acquitted on appeal. The day I dared to walk into a Carabinieri post, in Italy and explained the similarities between the way the snobby set collaborated to protect paedophiles in both the Church & Debian, the Cardinal died four hours later.

In my case against Reform UK Party Limited, I meekly asked them to pay £5,000, barely 0.1 percent of the controversial £5 million donation, based on Farage's promise to pay for the election. Instead, I received this priceless gem from Farage's own lawyers. Adam Richardson from 4-5 Gray's Inn Square has admitted Nigel Farage is "the Company's own candidate". This is one of those gaffs that may well stand the test of time, like the "plain vanilla" defense. I've never seen a confession like that in democracy before. So much for the promise the Clacton-on-Sea by-election was an opportunity to vote against the Establishment.

Dirk Eddelbuettel: myman 0.1.0 on CRAN: New (fabulous !!) Package

Wed, 30 Sep 2026 15:44:00 +0000

Thrilled to share that our new package myman reached CRAN a few days ago. The package offers sixteen hundred ninety four “My man …” posts by Kevin Kruse made on bsky during the summer of 2026. Each wave picked at one particular public persona. This package wrapse these up in the style of packages like fortunes or gaussfacts.

A sample usage illustration shows how to extract by pattern:

> library(myman)
> myman("maitre")
My man looks like he's inquiring with the maitre'd about the house curly fries.
     -- about Howard Lutnick on 2026-08-28

My man looks like a maitre'd who deeply doubts you have a reservation.
     -- about Scott Bessent on 2026-08-31

My man looks like he's asked the maitre'd to remove a party of four he finds visually
unpleasant.
     -- about Scott Bessent on 2026-08-31

My man looks like the maitre'd at a very exclusive restaurant called The Berghof.
     -- about JD Vance on 2026-09-13

>

One can also subset by ‘target’, or simple sample randomly (which is the default).

As CRAN review for new package takes a little longer these days, the GitHub repo was updated once more with one (eighth !!) wave of posts since the submission was made. CRAN version 0.1.0 corresponds to the stock of messages from waves one to seven. We will update the CRAN version shortly but if you would like more posts right now consider installing from the repo.

We include the full NEWS file since the package was started, this also includes an entry for the eights wave we will bring to CRAN soon.

Changes in version 0.1.0 (2026-09-27)

Initial CRAN version (minus the 0.0.8 changes which were made while CRAN reviewed the package)

Add an aspell dictionary to aid DESCRIPTION text

Changes in version 0.0.8 (2026-09-18)

Data set now comprises 1694 posts aiming at eight different 'men'

The number of 'sample()' object returned can not be set with 'posts'

Changes in version 0.0.7 (2026-09-14)

Data set now comprises 1494 posts aiming at seven different 'men'

Changes in version 0.0.6 (2026-09-07)

Data set now comprises 1394 posts aiming at six different 'men'

Changes in version 0.0.5 (2026-09-01)

Data set now comprises 1194 posts aiming at five different 'men'

The myman() function can now now subset by target permitting more focussed randlom sampling

Changes in version 0.0.4 (2026-08-28)

Data set now comprises 1040 posts aiming at four different 'men'

Changes in version 0.0.3 (2026-08-19)

Data set now comprises 916 posts aiming at three different 'men'

Csv file, processing and display extended to three columns also showing post creating date (in UTC) and 'man'

Changes in version 0.0.2 (2026-07-30)

Updated with full week of posts leading to a corpus of 698 posts

Added support for simple regular-expression lookup ability

Changes in version 0.0.1 (2026-07-19)

Initial release

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Gunnar Wolf: The Origins of GPU Computing

Tue, 29 Sep 2026 18:55:02 +0000

This post is an unpublished review for The Origins of GPU Computing

This article appears in the “Communications of the ACM” magazine as part of a series devoting to showcasing relevant examples of technologies where federal funding of academic research have resulted in widely impacting, clearly identifiable technological advances that have visibly improved –or, at least, changed– life far beyond the technological arena.

The road leading from the first parallel computing technologies built completely for research, through a phase that was preeminently directed at gaming, and back to leading current GPUs, that are basically massively parallel supercomputers used for LLM training and inference and other IA tasks, is far from trivial, and the authors make a good job outlining it in few pages. The article is written in a clearly divulgatory way, staying as clear of technical language as possible.

The authors present three main enabling technologies for current GPU technologies: Parallel computing, parallel graphics systems, and stream processing.

Coming from a technical formation myself, although clearly better grounded in the software side of Computer Engineering than on the electronics side, the most interesting “nugget of information” I take is that parallel computing was born due to the fact that transistor switching consumes very little energy, whereas communication along wires consumes much more — so the more simultaneous operations can be carried out on-chip at once, the less power will be dissipated as heat (and the less time that will be lost due to data stores and loads).

Graphics systems are a natural match for parallel systems, as rendering realistic images basically requires repeating the same blocks of instructions to a large array of items. Early GPU implementations were built on fixed-function graphics pipelines, but by the turn of the century, when nVidia spinned off the research labs at Stanford and started marketing their product, introducing along the way the Graphical Processing Unit term, product, they enabled it to implement the full OpenGL graphics pipeline on a single chip.

Stream processing is the arrangement of in-chip electronics so that they can better exploit producer-consumer locality between stages, in order to achieve greater arithmetic intensity (this is, a greater computation-to-bandwidth ratio), decreasing the total number of access to memory accesses.

The authors explain in broad terms the described technologies and how they fit together, and how the technology was transferred from the universities to the companies that became the most dynamic companies behind today’s top trending technologies.

I felt the article to be engaging and easy to read. As a technologist, I did miss more details — dates when specific advances were made, how some of the advances joined into further inventions… But I understand the aim of the “Federal Funding of Academic Research” series’ aim is more towards outsiders. And, as such, this work very well explains how trillion-dollar technologies are only the result of decades of funding research ideas.

Dima Kogan: Lunar terminator paradox

Sun, 27 Sep 2026 21:04:00 +0000

There's an optical illusion known by names like the "lunar terminator paradox". This has much discussion and descriptions online, but none made sense in my head, so I wrote this program to figure out what was going on. The paradox:

The moon is illuminated by the sun
After sunset, the sun is below, so we would expect the illuminated portion of the moon to point down, towards where the sun is
This isn't what always happens!

I was camping in the mountains with a friend recently. We were looking up at the sky, just after the sun has set. The moon was clearly pointing up.

After playing with this program, I have clarity: we're looking up at the moon from below. The most pathological case is a full moon. Let's say the moon is on the horizon: elevation=0. If we're looking at this low moon, and the moon is full, the sun is directly behind us. The moon is fully illuminated, and we see 100% of the moon disc lit up.

Now, what happens if the moon is not at the horizon, but higher up at the sky? Effectively, the sun is infinitely far away, so the same part of the moon is lit up. But we're now looking at the moon from below, and we would see some of the dark side on the bottom edge. I.e. the moon appears bright at the top, but has a dark slice missing at the bottom: it points up! Even at sunset.

This is the most clearly weird case. A half-moon has no paradoxical behavior, and the more full the moon, the more clearly we can observe the bright moon pointing up despite the sun being down.

I produce two plots. At 3/4-full moon at sunset we look at the moon as it rises from the horizon all the way up to the zenith:

If the moon is at the horizon (elevation = 0), we see the expected 3/4-lit disc. As the moon rises, we see more and more of its bottom, and thus more and more dark areas show up: It clearly points up at sunset!

We can also move the sun around. Let's say the moon is 60deg up; let's move the sun from new-moon to a full-moon:

In the darkest configuration, we can still see a lit crescent on the bottom: the illumination is all on the opposite side, but we can see a slice of the illuminated back side. As the sun swings around, we see more and more of the moon. At azimuth = 90deg, we have a half-lit moon. Past that, more and more of the moon is visible, always pointing up.

This was also a super interesting study of the current state of AI tools. As I was writing this and debugging, I would talk to Claude and Gemini about how this worked and what I should be seeing. They were both completely unable to comprehend this, and would make infinite circular arguments. Clearly they read some incomplete explanations on the internet (as I have), and lack the reasoning capability necessary to distill it into something resembling "understanding". AGI is not here yet.

Dirk Eddelbuettel: #060: Using bubblewrap for R sandboxing

Sun, 27 Sep 2026 16:02:00 +0000

Welcome to post 60 in the R⁴ series.

bubblewrap is a great tool and very suitable for using with an agent harness. It is a very compelling—and lightweight—alternative to using a full-blown docker container as it offers low-level unprivileged sandboxing on Linux hosts.

In a nutshell, bubblewrap can ‘turn everything off’ (see unshare-all below) and allow access only to selected services and directories (as shown below). This makes it a very useful tool be used on a main workstation as it can provide a lower-risk deployment quite easily while taking advantage of the already installed software stack. I continue to get a lot of value out of docker, especially as r2u makes installing R package dependencies so trivial. Its slogan ‘easy, fast, reliable: pick all three’ clearly holds for r2u. But sometimes bubblewrap is compelling, for example to launch opencode (documented for example at the debian-inference site).

When using R we need add more directories to the example to provide /etc/alternatives which is governing inter alia the LAPACK / BLAS resolution on Debian / Ubuntu systems; this is now on debian-inference site but wasn’t when I first tried it a few days ago ;-) as well as /etc/R for config files and possibly ~/.R/Makevars for compiler settings. With that my current wrapper is

#!/bin/bash
#
# cf https://inference.debian.net/doc
#    and 'curl' command to set bearer code
bwrap \
  --ro-bind /usr /usr \
  --symlink usr/bin /bin \
  --symlink usr/lib /lib \
  --symlink usr/lib64 /lib64 \
  --ro-bind /etc/ssl /etc/ssl \
  --ro-bind /etc/resolv.conf /etc/resolv.conf \
  --ro-bind $HOME/.gitconfig $HOME/.gitconfig \
  --bind $HOME/.config/opencode $HOME/.config/opencode \
  --bind $HOME/.cache/opencode $HOME/.cache/opencode \
  --bind $HOME/.opencode $HOME/.opencode \
  --bind $HOME/.local/share/opencode $HOME/.local/share/opencode \
  --bind $HOME/.local/state/opencode $HOME/.local/state/opencode \
  --ro-bind $HOME/.R/Makevars $HOME/.R/Makevars \
  --ro-bind /etc/R/ /etc/R \
  --ro-bind /etc/alternatives/ /etc/alternatives \
  --proc /proc \
  --dev /dev \
  --tmpfs /tmp \
  --bind $(pwd) $(pwd) \
  --chdir $(pwd) \
  --unshare-all \
  --share-net \
  --die-with-parent \
  /usr/local/bin/opencode "$@"

Launched that way we can have agents use R to check packages sources, builds, and triage for bugs. this works equally well with local models served via ollama (and with that a quick shoutout to packages.lingfish.net for providing an apt service for ollama) as it does with the various cloud-based offerings. So harness away with R!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Daniel Pocock: Notorious Gem of Tanzania (Wrekin Ruby) & Nigel Farageâ€™s mega-donations

Sun, 27 Sep 2026 15:00:00 +0000

Many people have seen the recent news about Reform UK party receiving two donations of £36 million each. Therefore, it is quite a shock to see the party simultaneously being subject to a [redacted debt collection].

UK law doesn't require a contract to be in writing. Nigel Farage presents himself as somebody you can have a pint with. If somebody agrees to pay for a round of drinks you don't ask them to provide a written contract. Likewise, when Nigel Farage and other leaders in his party made promises to pay election expenses, I felt that was like offering to buy a round of drinks. It was a verbal contract.

The failure to make those payments provided a justification to begin [redacted]. Does that mean Nigel Farage can simply pay me the money and exit the [redacted] procedure? Maybe not. [redacted].

We can find old reports from other Reform UK legal entities online. Here are the 2021 accounts. This is the relevant statement where the auditor contemplates whether money loaned to the party by Richard Tice could be a risk to solvency:

Material uncertainty related to going concern

We draw attention to the Balance Sheet which indicates that as at 31 December 2021, the Party had net liabilities of £849,456. As stated in the going concern accounting policy note, these liabilities consist mainly of directors loans from Richard Tice and we have received suitable reassurances from him about the intention for these to help grow the party in the medium term.

When the auditor writes their report for the next set of accounts, they will have to mention a whole range of scandals much bigger than that loan.

Remember that most volunteers and donors put in their time and money and they can not get their time and money back later. Yet Richard Tice put in his money as a loan, in other words, when other people contribute money, he gets his money back.

The existence of loans like this helps understand directors' willingness to accept donations that other parties might not touch with a ten foot pole.

Notice that Reform UK announced the mega donations just two weeks before the deadline for the accounts and auditor's report. Hence my curiosity about whether the publicity about these donations was meant to either sway the auditor or distract the public from concerns the auditor will express in the published report.

We saw the same thing with the notorious Gem of Tanzania. By coincidence, in November 2025, shortly before the end of Reform UK's first accounting year, the International Gem Society published a detailed history of the scandal. News reports claim the gem was worth little more than £100. Nonetheless, Wrekin Construction listed it on their balance sheet as an asset worth £11 million.

When I saw the news about these mega-donations, I felt déjà-vu. If it sounds too good to be true, maybe it is.

Sub-contractors and suppliers to Wrekin Construction felt they could give the company favorable credit terms because the balance sheet filed at Companies House looked extremely healthy. Some of the suppliers purchased trade credit insurance and when Wrekin Construction went bust, the insurance companies had to reimburse the suppliers who were not paid.

When people see these figures in the accounting documents and news stories about Reform UK, they may be fooled to be overoptimistic, in the short term, about Reform UK's future prospects.

The new legislation going through the UK parliament tells us these donations will be converted to loans. The party will be obliged to repay the money within 30 days.

The directors have known about this new reality since 25 March 2026 but their words and actions suggest they are being stubborn and behaving like petulant teenagers.

Russ Allbery: Review: Ode to the Half-Broken

Sun, 27 Sep 2026 02:57:00 +0000

Review: Ode to the Half-Broken, by Suzanne Palmer

Publisher:	DAW
Copyright:	May 2026
ISBN:	0-7564-2013-X
Format:	Kindle
Pages:	441

Ode to the Half-Broken is a standalone post-apocalyptic (sort of) science fiction novel.

As this novel opens, the unnamed first-person protagonist is waking up in a bathtub in an abandoned building. They are covered in electromagnets and missing one leg.

Waking up involved a secure reboot from a protected core kernel, so it is immediately obvious that the protagonist is a robot. That also explains the electromagnets, which interfere with their ability to control their body. Or they would, if the protagonist didn't have long-dormant combat routines available to deploy EM shielding and free themselves from the bathtub. There are no immediate answers to why someone embedded a virus in the data attached to a scientific paper on ants, an act so precisely targeted that it had to be personal. Or what happened to their leg.

"There are a number of electromagnets in the next room," I say. I point. "Behind that wall with all the holes in it."

"Those are big holes. And fresh, too," the dog says. "I wonder what made them."

"Electromagnets experiencing sudden velocity."

The dog gives another of its short barks, and I am more convinced now it is laughing, though I was not attempting to be funny.

Ode to the Half-Broken has a classic science fiction beginning: in medias res in a future science fiction version of Earth where nothing is immediately explained, leaving the reader to work out the date and the setting. Helping us out is the protagonist, who is neither chatty nor very sociable with other characters but keeps up an analytical monologue about the world as they narrate events.

We are some decades into a grim future. The United States has largely collapsed. Large parts of New York City are dangerous to humans due to disease and radiation. Neither of those hazards bother the large number of sentient robots; their scarcity, and therefore economy, is instead based on power. Robots that control solar power stations act as low-level mob bosses, trading power for information or labor.

Our protagonist doesn't want to care about any of this. They retreated from the world long ago for a solitary life in the former New York Botanical Gardens studying ants. But they need a new leg and the best robot repair person nearby is a human woman outside of the city, so they reluctantly head in that direction, accompanied somewhat surprisingly by a cyborg dog.

The typical science fiction apocalypse is caused by something specific. Classically, that event is a nuclear war, but writers found various alternatives before, during, and after the Cold War era: alien invasions, meteor showers, plagues, sea level rise, and peak oil and general environmental collapse, among many others. Less common is a post-apocalyptic novel where the collapse comes from multiple overlapping crises and accumulating failures, from a general failure to meet what Adam Tooze and others call polycrisis. The first time I encountered that style of post-apocalyptic story was in Octavia Butler's Parable of the Sower. It was one of the choices that made (and still makes) that book feel so chillingly realistic.

The world background in Ode to the Half-Broken is shaped by that sort of polycrisis collapse. There is no one reason why New York City is a dangerous, irradiated ruin and society has collapsed into isolated and sometimes warring factions. It is not runaway artificial intelligence, or at least not directly; sentient robots were a late-stage innovation of the collapsing US civilization and a tool and accelerator of its many wars, but they were not the cause and may be a critical ingredient in building some functional replacement. Instead, society fell apart because we failed to address any of the growing problems and then turned on each other in the resulting ruins. The reader gets an idiosyncratic view of that collapse in flashback chapters intermixed with the main narrative of the initially unnamed robot protagonist, flashbacks told from the perspective of the man who invented the key component for robot sentience. As you would expect, those two narratives are linked. The reader will not discover how they are linked until late in the story.

This book could have been extremely depressing. It is not, which says a great deal about Palmer's skill in telling it. The clinical distance of the protagonist in the early chapters helps considerably: They have rejected involvement in this world and therefore can describe it in detached, factual terms, aided by the tendency of most robots in this story to be slightly more precise, logical, and verbose than the human characters. But as the protagonist's clinical distance fades, as it must to allow the story's emotional stakes to rise, Palmer fills this book with moments of beauty and hope. A former nuclear submarine that has become the mind of a subway car is a key early ally. A collective has turned the Hudson Bridge into a thriving if chaotic community. Self-regulating enclaves of humans and robots are working together to try to build a livable world. Robots oversee pollinator swarms that are attempting to restore plant life, for no reason other than that they like order and stability and see this as a way of restoring both. This is a grim, post-apocalyptic world that does not dwell on either the grimness or the apocalypse. It is instead full of small moments of hope, collaboration, and personal connection.

This is a story with villains, danger, pain, and risk. Despite the similar keywords and cover aesthetics, it bears little resemblance to Becky Chambers's philosophical Monk and Robot series. It is not, in any definition of the term, cozy; it's a post-apocalyptic adventure story about a robot and a dog making their way through a ruined civilization and fighting people who are trying to build something even worse on the rubble. But it's also charming and oddly happy. There are strong found-family vibes in the way that a party coalesces around the protagonist and eventually gives them a name, there is the quiet happiness of seeing a grumpy misanthrope slowly come out of their protective shell, and the rail mechs are an absolute delight.

Like a lot of stories of happy small-group anarchism, I didn't entirely believe the politics and have some qualms about the realism of the ending, although perhaps I should strive to be as optimistic as Palmer writes. The somewhat dry and fussily precise first-person narration will also not be to everyone's taste; it worked for me by the end of the book, but it took a bit of reading to get used to it. But those quibbles aside, this was quietly lovely. It's a satisfying science fiction adventure story in a world with disturbing parallels to the one we're living in, but which uses those parallels to focus on small-scale collective efforts to build something better. And all the small details of the world-building are delightful: the different tiers of robots and their odd behaviors, the machine sense of purpose, even the mix of suspicion and generosity. The plot has large-scale implications, but it was the human-level details that charmed me.

I still want another Fergus novel, but this was a highly successful foray outside of that series. Palmer has won two Hugos for her novellas, so I can't really say she's overlooked, but her novels aren't getting anywhere near the attention they deserve. Highly recommended.

There is plenty of world-building room here for a sequel, but Ode to the Half-Broken reaches a satisfying standalone conclusion. So far as I know, no sequel is currently planned.

Rating: 8 out of 10

Dirk Eddelbuettel: RcppFastAD 0.0.5 on CRAN: Maintenance

Thu, 24 Sep 2026 19:13:00 +0000

A new release 0.0.5 of the RcppFastAD package is now on CRAN, has been built for r2u, and updated at r-universe. This comes (to the day) two years after the preceding 0.0.4 release.

RcppFastAD wraps the FastAD header-only C++ library by James which provides a C++ implementation of both forward and reverse mode of automatic differentiation. It offers an easy-to-use header library that is both lightweight and performant. With a little of bit of Rcpp glue, it is also easy to use from R in simple C++ applications. This release updates the continuous integration setup as one does, adds a local configuration helper to quieten compilation (described also in this blog post). It also adds a defensive setting for g++: Under recent g++ versions and optimisation at least the -O3 level, segfaults are seen as something is not quite right with (temporary) “views” of Eigen objects in code generated by this versions. Others are fine, as is clang++. We have not gotten to the bottom of it, but setting -g0 seems to ensure that builds generally work.

The NEWS file for this release follows.

Changes in version 0.0.5 (2026-09-24)

Several routine updates to continuous integration have been made

Local builds (where a .git/ directory is seen) now append silencing compiler option that CRAN would object to

Given recent issues with g++ under optimization, debugging is turned off by default.

Courtesy of my CRANberries, there is also a diffstat report for the most recent release. More information is available at the repository or the package page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

Russell Coker: Links September 2026

Thu, 24 Sep 2026 10:33:22 +0000

Bjorn Stahl of the Arcan project wrote an insightful blog post The Day of a new Command-Line Interface: Shell [1]. He does a really good job of identifying problems and strategies for dealing with them, the UI of the results isn’t suitable to what I do though.

Using sqlite as an executable format (replacement for ELF) is one of the wildest ideas I’ve seen in a while, it might actually have some good use cases too [2].

Andrew Dana Hudson wrote an interesting LongNow article about SciFi’s focus on space travel when it seems unlikely [3].

TheDriven has an informative article about the “Hands Off Our Fuel” astroturf campaign that uses farmers as the face of an attempt to give tax exemptions to mining companies [4].

The Conversation has an interesting article about the new Chinese regulations restricting access to AI systems to prevent emotional dependence and prevent minors from having virtual partners or relatives [5].

Elana Hashman wrote an informative blog post about Python virtualenvs, has useful information for people who aren’t Python experts and just need to get stuff done [6].

We need more systems in the NTP pool, I would consider doing this if I had a server with an IP address that won’t change and no risk of bandwidth throttling [7].

Cory Doctorow has some interesting thoughts on the CEO fetish for “AI” even though it is almost never doing any good [8].

Steve Rose wrote an insightful article for The Guardian about the tech fascists of Silicon Valley [9].

Jacqui Lambie calls Pauline Hanson an “angry little woman” and a “bloody coward”, One Nation hate the rule of law [10].

The Conversation has an interesting article about Regime Change, the book about Trump by Maggie Haberman and Jonathan Swan [11].

Elvira Barry wrote an insightful article about the many failures of the Russian legal system, it’s interesting to note that ALL those failures match things that the Trump regime is doing [12].

Pyotr Kurzin – Geopolitics YouTube channel has an interesting interview about “continental” vs “maritime” powers in the context of what the US and Russia are doing [13].

Maxinomica has an interesting video about how China obtained a monopoly on rare earth metals and what happens next [14].

Anarcat wrote an insightful blog post The people vs the AI overlords about the problems that “AI” software development is bringing [15].

John Goerzen wrote an interesting blog post discussing the issues related to LLM use in FOSS projects [16].

Cybernews has an article about an exploit for Samsung, Xiaomi, Oppo, and OnePlus phones due to bad OEM kernel drivers and bad OEM API proxies, no company should release devices with proprietry drivers [17].

Scott Santens wrote an insightful article about how retraining fails laid-off workers and why UBI is needed when workers get replaced by LLMs [18].

libroot has some information on the Snowden archive and why so little of it has been used by journalists, we need more answers about this [19].

Drew Devault’s list of Weird little guys of FOSS is interesting history about some people who are best avoided [20].

[1] https://tinyurl.com/ydye29ta
[2] https://tinyurl.com/26223e79
[3] https://longnow.org/ideas/space-dead/
[4] https://tinyurl.com/287rktuz
[5] https://tinyurl.com/25a9mag3
[6] https://hashman.ca/managing-venvs/
[7] https://dreamstation.systems/personal/ntppool.html
[8] https://tinyurl.com/2bzwm5d6
[9] https://tinyurl.com/25bj88ss
[10] https://tinyurl.com/2ywkupwv
[11] https://tinyurl.com/257umlg6
[12] https://tinyurl.com/2cjyev9l
[13] https://www.youtube.com/watch?v=y50uWvxGb-A
[14] https://www.youtube.com/watch?v=BXLGV0Sj0n8
[15] https://anarc.at/blog/2026-08-18-people-vs-ai-overlords/
[16] https://tinyurl.com/2a8pzj8j
[17] https://tinyurl.com/249oafcy
[18] https://tinyurl.com/28f5gdjv
[19] https://tinyurl.com/2cv3v9e7
[20] https://drewdevault.com/weird-guys/

Mike Gabriel: Looking for Golang + Fullstack Developer with interest in Civic Tech

Wed, 23 Sep 2026 20:16:37 +0000

Who?

Fre(i)e Software GmbH is looking for a senior Golang + fullstack developer who can handwrite code and act as meticulous code review partner for another senior developer in the company.

What?

Most of your work would result in Open Source contributions to the Voxit project [1], an online civic tech tool for digital participation.

How and Where?

The work can either be delivered on project agreement base via freelancing or via employment (option only available to developers resident in Germany, Austria or Poland).

What next?

If you are interested, please get in touch and provide your hourly rate to us and your average hours of availability per week.

[1] https://gitlab.com/voxit/

Vincent Bernat: Bot-free self-hosted analytics with GoatCounter on NixOS

Sun, 20 Sep 2026 19:44:50 +0000

Inâ€¯2016, I removed Google Analytics from this blog to avoid being complicit in feeding the biggest machine for harvesting personal data. Instead, I relied on GoAccess to analyze my server logs.¹ For the past couple of years, the statistics have made no sense, despite my attempts to filter bots: AI scrapers inflate the number of visitors to around 2,000 per day. Eventually, I settled on GoatCounter, an open-source, privacy-friendly web analytics platform. I replaced the JavaScript client to filter bots more aggressively and added a CSS fallback. To improve reliability, I implemented a local proxy running on each of the five web servers serving this blog. The rest of this post details how these pieces fit together and how I deploy them on NixOS. â�„ï¸�

Why GoatCounter?

GoatCounter does not collect personal data: instead of storing the readerâ€™s IP address or relying on cookies, it creates a session identifier valid for 8 hours from the user agent and the IP address. Its feature set is modest but sufficient for a blog. If you want to look at the interface, GoatCounterâ€™s author runs a public instance for his site. A hosted version lets you try it before running your own instance. With a single binary and an SQLite database, GoatCounter is one of the lightest self-hosted solutions. Privacy-friendly alternatives, in increasing order of complexity, include Umami, Plausible, and Rybbit.

Custom JavaScript client

GoatCounter includes a small JavaScript clientâ€”2,189 bytes minified and gzipped. It ships some features I donâ€™t use: a visitor counter, tracking clicks, configurable settings, etc. I replace it with this function to register a hit:

const count = ({ event, title } = {}) => {
  const params = new URLSearchParams({
    p: event || location.pathname,
    t: title || document.title,
    r: document.referrer,
    q: location.search,
    s: document.documentElement.clientWidth,
    e: !!event,
    rnd: Math.random().toString(36).slice(2, 7),
  });
  fetch(`/count?${params}`, { keepalive: true }).catch(() => {});
};

To filter bots,² I go the extra mile by requiring a user interactionâ€”an idea I stole from Bear Blog.

let sendHit = () => (sendHit = () => {}, count());
["touchmove", "mousemove", "keydown", "pointerdown"].forEach((eventName) =>
  document.addEventListener(eventName, sendHit, {
    once: true,
    passive: true,
  }),
);

If a reader has disabled JavaScript in their browser, I record the hit using a CSS image. The :hover pseudo-class loads it only after an interaction, another trick stolen from Bear Blog. Aboutâ€¯2% of my visitors fit into this bucket.³

<!DOCTYPE html>
<html lang="en" class="nojs">
  <head>
    <script>
      // The JavaScript code for this blog requires ES6
      if ("noModule" in HTMLScriptElement.prototype)
        document.documentElement.classList.remove("nojs");
    </script>
  </head>
  <body>
  <!-- ... -->
    <style>
      .nojs body:hover {
        border-width: 0;
        border-image: url('/count?p=/en/blog/2026-kpi-goodhart&t=Building...&r=NoJS&e=false');
      }
    </style>
  </body>
</html>

Where GoAccess reported around 2,000 visitors a day, GoatCounter counts fewer than 200 humans.⁴ I assume AI scrapers use a low-effort approach: if the content is available without barriers, as on this blog, they donâ€™t spawn a complex mechanized browser that could trigger a page view. Even crawlers running JavaScript, like Googlebot with its headless Chromium, do not interact with the page and never trigger the events I listen to. The interaction-based â€œproof of humanityâ€� I use is likely to keep working.⁵

Local proxy

Five servers ~~across the world~~ in Europe and in North America serve the content of this website, but GoatCounter runs on only one of them. To avoid losing track of visitors when GoatCounter is down, I run a local proxy listening on the same /count endpoint. On each server, it stores the hits in memory with a buffer large enough to survive several days of downtime. It sends them in batches to the upstream backend using the /api/v0/count authenticated endpoint.

🖼 Servers on a map. web02 is in Paris, web03 in Helsinki, web04 in Nuremberg, web05 in Ashburn, web06 in Chicago.

I proposed the code for the proxy in pull request #909. GoatCounterâ€™s maintainer declined to maintain so much code for such a niche use case. As a fellow open-source developer, I often hold the same position for my own projects: a one-time contributor effort may translate into a long-term maintainer commitment.

I expose the endpoint for the proxy on the domain of this website to evade ad blockers. This sounds like I donâ€™t respect the readerâ€™s choice, but as GoatCounter is privacy-friendly, I find it acceptable.

location = /count {
  access_log off;
  proxy_pass http://127.0.0.3:8087/count;
  proxy_pass_request_headers off;
  proxy_set_header Accept-Language $http_accept_language;
  proxy_set_header User-Agent $http_user_agent;
  proxy_set_header X-Real-Ip $remote_addr;
}

Deploying on NixOS

My web servers run NixOS, a declarative Linux distribution with built-in configuration management. I manage this small fleet with Colmena, a stateless deployment tool for NixOS. My configuration is available on GitHub.

Deploying applications in containers

For better isolation, each application runs inside an ephemeral lightweight container, powered by systemd-nspawn. Each container runs a stripped-down NixOS instance. A module wraps NixOSâ€™s containers options to avoid repeating the same options for each application.⁶ The containers share their network namespace with the host: the additional isolation is not worth the increased complexity. For a smaller footprint, I also disable a few non-essential services.

{ config, lib, ... }:
let
  cfg = config.luffy.containers;
in
{
  # User-configurable settings for our custom module
  options.luffy.containers = lib.mkOption {
    default = { };
    description = "Ephemeral containers sharing the host network.";
    type = lib.types.attrsOf (lib.types.submodule {
      options = {
        config = lib.mkOption {
          type = lib.types.deferredModule;
          default = { };
          description = "NixOS configuration of the container.";
        };
      };
    });
  };

  # Translate our options to NixOS containers
  config = {
    containers = lib.mapAttrs
      (name: container: {
        ephemeral = true;
        autoStart = true;
        privateNetwork = false;
        extraFlags = [ "--resolv-conf=replace-host" ];
        config = {
          imports = [ container.config ];
          networking.firewall.enable = false;
          system.stateVersion = config.system.stateVersion;
          systemd.services = {
            console-getty.enable = false;
            systemd-logind.enable = false;
            systemd-oomd.enable = false;
          };
        };
      })
      cfg;
  };
}

To configure a GoatCounter instance running in a container and listening on 127.0.0.4:8088, we import the module⁷ and declare the container in the config.luffy.containers attribute set:

{ pkgs, config, ... }: {
  imports = [ ./modules/container.nix ];
  config.luffy.containers.goatcounter = {
    config = {
      services.goatcounter = {
        enable = true;
        address = "127.0.0.4";
        port = 8088;
        proxy = true;
      };
    };
  };
}

As the containers are ephemeral, we need to keep persistent data in directories on the host. We add a mounts option and ask NixOSâ€™s containers to expose the configured directories through the bindMounts option.

{ config, lib, ... }:
let
  cfg = config.luffy.containers;
in
{
  options.luffy.containers = lib.mkOption {
    type = lib.types.attrsOf (lib.types.submodule {
      options = {
        mounts = lib.mkOption {
          type = lib.types.listOf lib.types.str;
          default = [ ];
          description = "Host directories mounted read-write at the same place.";
        };
      };
    });
  };

  config = {
    containers = lib.mapAttrs
      (name: container: {
        bindMounts =
          lib.genAttrs container.mounts (path: { hostPath = path; isReadOnly = false; });
      })
      cfg;
  };
}

For example, to persist GoatCounterâ€™s database in the /var/db/goatcounter directory on the host, we add the directory to the mounts option and alter the service definition to tell GoatCounter where the database is.

{ config, ... }:
let
  databaseDirectory = "/var/db/goatcounter";
in {
  config.luffy.containers.goatcounter = {
    mounts = [ databaseDirectory ];
    config = {
      services.goatcounter = {
        extraArgs = [ "-db=sqlite+${databaseDirectory}/db.sqlite" ];
      };
    };
  };
}

A container may also need some secrets. Colmena can upload secrets without storing them in the Nix store. We add a keys option to our containers. It takes an attribute set mapping secret names to the commands to populate them. Then, the module declares the required secrets to Colmena in the deployment.keys option, makes the container depend on the presence of the secrets, and exposes them to the container.

{ config, lib, ... }:
let
  cfg = config.luffy.containers;
in
{
  options.luffy.containers = lib.mkOption {
    type = lib.types.attrsOf (lib.types.submodule {
      options = {
        keys = lib.mkOption {
          type = lib.types.attrsOf (lib.types.listOf lib.types.str);
          default = { };
          description = "Secrets, as a command to run locally. They are mounted in /etc.";
        };
      };
    });
  };

  config = {
    # Colmena uploads each secret in `/var/keys` and make them available
    # to the group "keys".
    deployment.keys = lib.concatMapAttrs
      (_: container: lib.mapAttrs
        (_: keyCommand: {
          inherit keyCommand;
          group = "keys";
          permissions = "0640";
          destDir = "/var/keys";
        })
        container.keys)
      cfg;

    # The container can only start if the required secrets are available.
    systemd.services = lib.mapAttrs'
      (name: container:
        let
          units = map (key: "${key}-key.service") (lib.attrNames container.keys);
        in
        lib.nameValuePair "container@${name}" {
          requires = units;
          after = units;
        })
      cfg;

    # Mount each secret inside the container.
    containers = lib.mapAttrs
      (name: container: {
        bindMounts = lib.mapAttrs'
          (key: _: lib.nameValuePair "/etc/${key}" {
            hostPath = "/var/keys/${key}";
            isReadOnly = true;
          })
          container.keys;
      })
      cfg;
  };
}

For example, GoatCounter needs credentials to download the GeoIP database. I provide a local command to fetch the secret from my password manager and expose it inside the container through the /etc/goatcounter.env environment file.

{ pkgs, config, ... }: 
let
  keyCommand = variable: [
    "${pkgs.runtimeShell}"
    "-c"
    "pass show personal/nixops/secrets | grep '^${variable}='"
  ];
in {
  config.luffy.containers.goatcounter = {
    keys."goatcounter.env" = keyCommand "GOATCOUNTER_GEODB";
    config = {
      systemd.services.goatcounter.serviceConfig = {
        EnvironmentFile = "/etc/goatcounter.env";
        SupplementaryGroups = [ "keys" ];
      };
    };
  };
}

GoatCounter server

Nixpkgs already packages GoatCounter. By overriding the src and vendorHash attributes, I reuse its definition for my custom version with the proxy:

{ goatcounter, fetchFromGitHub }:
goatcounter.overrideAttrs (_: {
  src = fetchFromGitHub {
    owner = "vincentbernat";
    repo = "goatcounter";
    rev = "feature/proxy";
    hash = "sha256-dJRlQlFu3tjcEgabT1LEbyFrasJlhmYu4L/T7EkoNcY=";
  };
  vendorHash = "sha256-c9Q5OrbZR+q6pD3SgPPWe8JUzcZco1AVUKGaV61k5DE=";
})

I wrote a NixOS module to encapsulate GoatCounter: the container definition, the service definition, and the secrets. The module accepts the following options: package, serve.enable, serve.listenAddress, serve.port, and serve.databaseFile. I already detailed the container configuration in the previous section. In the end, I chose not to reuse the GoatCounter module from NixOS: itâ€™s small, so itâ€™s better to insulate my module from unexpected future changes.

{ config, pkgs, lib, ... }:
let
  cfg = config.luffy.goatcounter;
  databaseDirectory = builtins.dirOf cfg.serve.databaseFile;
  chown = "${pkgs.coreutils}/bin/chown -R";
in {
  config.luffy.containers.goatcounter = {
    config.systemd.services.goatcounter = {
      description = "GoatCounter Web Analytics";
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        EnvironmentFile = "/etc/goatcounter.env";
        SupplementaryGroups = [ "keys" ];
        DynamicUser = true;
        Restart = "always";
        ExecStart = lib.escapeShellArgs [
          (lib.getExe cfg.package)
          "serve"
          "-listen=${cfg.serve.listenAddress}:${toString cfg.serve.port}"
          "-tls=none"
          "-db=sqlite+${cfg.serve.databaseFile}"
          "-automigrate"
        ];
        # Transfer database ownership to dynamically assigned user "goatcounter".
        ExecStartPre = "+${chown} goatcounter:goatcounter ${databaseDirectory}";
        ReadWritePaths = databaseDirectory;
      };
    };
  };
}

The following snippet configures GoatCounter to listen on 127.0.0.4:8088:

{
  luffy.goatcounter = {
    serve = {
      enable = true;
      listenAddress = "127.0.0.4";
      port = 8088;
    };
  };
}

The last step is to configure nginx to expose GoatCounter on the Internet. I disable the /count endpoint as the local proxy handles it.

{ config, ... }:
let
  cfg = config.luffy.goatcounter.serve;
in
{
  services.nginx.virtualHosts."goatcounter.luffy.cx" = {
    forceSSL = true;
    locations = {
      "/" = {
        proxyPass = "http://${cfg.listenAddress}:${toString cfg.port}";
      };
      "= /count".extraConfig = ''
        return 404;
      '';
    };
  };
}

GoatCounter proxy

The same NixOS module configures the local proxy, with the following options: proxy.enable, proxy.listenAddress, proxy.port, and proxy.siteâ€”the site receiving the batches of page views. The local proxy has no persistent data, but it needs the API key to authenticate to the main GoatCounter instance: its container uses the keys option but not the mounts option.

{ config, pkgs, lib, ... }:
let
  cfg = config.luffy.goatcounter;
  keyCommand = _: [ "â€¦" ];
in
{
  config.luffy.containers.goatcounter-proxy = {
    keys."goatcounter-proxy.env" = keyCommand "GOATCOUNTER_API_KEY";
    config.systemd.services.goatcounter = {
      description = "GoatCounter Proxy.";
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        EnvironmentFile = "/etc/goatcounter-proxy.env";
        SupplementaryGroups = [ "keys" ];
        DynamicUser = true;
        Restart = "always";
        ExecStart = lib.escapeShellArgs [
          (lib.getExe cfg.package)
          "proxy"
          "-site=${cfg.proxy.site}"
          "-listen=${cfg.proxy.listenAddress}:${toString cfg.proxy.port}"
          "-ratelimit=10/1"  # 10 requests per second per IP
        ];
      };
    };
  };
}

For each server, I enable the local proxy with the following snippet. The nginx configuration shown earlier exposes the /count endpoint under the same domain as my blog.

{
  luffy.goatcounter = {
    proxy = {
      enable = true;
      site = "goatcounter.luffy.cx";
      listenAddress = "127.0.0.3";
      port = 8087;
    };
  };
}

Backup of the SQLite database with Litestream

Litestream is a streaming replication tool for SQLite databases. It compresses the changes committed to the write-ahead log (WAL) next to the database and sends them to a remote destination. I encapsulate its configuration in a NixOS module, which takes an attribute set databases mapping a name to the path of the database to back up.

Litestream also runs in a container. I mount the databases to replicate, as well as the secrets to push the backups to a Hetzner storage box using SFTP:

{ config, pkgs, lib, ... }:
let
  cfg = config.luffy.litestream;
  databaseDirs = lib.unique (map builtins.dirOf (builtins.attrValues cfg.databases));
in
{
  config = lib.mkIf (cfg.databases != { }) {
    luffy.containers.litestream = {
      mounts = databaseDirs;
      keys."litestream.env" = [
        "${pkgs.runtimeShell}"
        "-c"
        "pass show personal/nixops/secrets | grep '^SQLITE_BACKUP_'"
      ];
    };
  };
}

Inside the container, I configure Litestream through NixOSâ€™s services.litestream options:

full snapshots every day, kept for 15 days,
three levels of compaction for transaction files: 5 minutes, 30 minutes, and 3 hours,
auto-recovery,⁸
replica stored in a directory matching the host name, and
credentials read from /etc/litestream.env and exposed through variable expansion.

{ config, pkgs, lib, ... }:
let
  cfg = config.luffy.litestream;
in
{
  config.luffy.containers.litestream = {
    config = {
      # The databases belong to dynamically allocated users, whose UID is
      # not known here, so Litestream runs as root.
      systemd.services.litestream.serviceConfig = {
        User = lib.mkForce "root";
        Group = lib.mkForce "root";
      };
      # Use NixOS service.
      services.litestream = {
        enable = true;
        environmentFile = "/etc/litestream.env";
        settings = {
          auto-recover = true;
          snapshot = {
            interval = "24h";
            retention = "360h";
          };
          levels = [
            { interval = "5m"; }
            { interval = "30m"; }
            { interval = "3h"; }
          ];
          dbs = lib.mapAttrsToList
            (name: path: {
              inherit path;
              replica = {
                type = "sftp";
                host = "\${SQLITE_BACKUP_HOST}";
                user = "\${SQLITE_BACKUP_USER}";
                password = "\${SQLITE_BACKUP_PASSWORD}";
                host-key = "\${SQLITE_BACKUP_HOSTKEY}";
                path = "${config.networking.hostName}/${name}";
              };
            })
            cfg.databases;
        };
      };
    };
  };
}

To back up GoatCounterâ€™s database, I declare a goatcounter attribute in luffy.litestream.databases, set to the database path:

{ config, ... }:
let
  cfg = config.luffy.goatcounter.serve;
in
{
  luffy.litestream.databases.goatcounter = cfg.databaseFile;
}

On the SFTP server, we can inspect Litestreamâ€™s work, with the compacted transactions and the full snapshots:

â�¯ ls web02/goatcounter/ltx
web02/goatcounter/ltx/0
web02/goatcounter/ltx/1
web02/goatcounter/ltx/2
web02/goatcounter/ltx/3
web02/goatcounter/ltx/9
â�¯ ls -lh web02/goatcounter/ltx/1
29.1K Sep  5 01:25 0000000000003f2a-0000000000003f2b.ltx
72.4K Sep  5 02:03 0000000000003f2c-0000000000003f2d.ltx
63.3K Sep  5 02:24 0000000000003f2e-0000000000003f2f.ltx
[â€¦]
â�¯ ls -lh web02/goatcounter/ltx/9
 8.5M Sep  5 02:00 0000000000000001-0000000000003f2b.ltx
 8.5M Sep  6 02:03 0000000000000001-0000000000004008.ltx
 8.6M Sep  7 02:03 0000000000000001-00000000000043a8.ltx
[â€¦]

We can restore the database from the backup with a few shell commands. First, we stop the containers. Then, we move the damaged database away, invoke litestream restore from the right environment, and restart the containers.⁹

# systemctl stop container@goatcounter container@litestream
# mv /var/db/goatcounter/db.sqlite{,.old}
# ( . /etc/nixos-containers/litestream.conf ; 
>   set -a ; . /var/keys/litestream.env ; set +a ;
>   $SYSTEM_PATH/sw/bin/litestream \
>     restore -config $SYSTEM_PATH/etc/litestream.yml /var/db/goatcounter/db.sqlite)
# ls -lh /var/db/goatcounter/db.sqlite
-rw-r--r-- 1 root root 20M Sep 20 07:33 /var/db/goatcounter/db.sqlite
# systemctl start container@goatcounter container@litestream

Ten years after removing Google Analytics, JavaScript-based analytics is back on this blog, but without storing cookies or IP addresses, and without involving a third party. I still write for myself first, notably because it lets me dig into a topic and refer back to it years later. But knowing a bit more about my fellow human readers is a nice bonus, even the ones disabling JavaScript. ğŸ��

Nginx scrambles IP addresses before storing them, thanks to the ipscrub nginx module.Â â†©
GoatCounter already filters some bots based on the user agent or the IP address. But AI scrapers lie about their user agent and hide behind residential proxies.Â â†©
Without JavaScript, I cannot send the real referrer. I insert â€œNoJSâ€� instead.Â â†©
I am missing the humans reading the RSS feed. I am not comfortable adding a tracking pixel, and bots are likely to fetch it, compromising the statistics. Iâ€™ll live without counting these readers.Â â†©
In Februaryâ€¯2025, I tested another CSS-only approach without the interaction trick: each page loaded an empty image as a background. Over a month, it recorded up to ten times fewer page views than the nginx logs, even after removing user agents identifying as bots. Compared to this experiment, GoatCounter counts about three times fewer views, 18 months later. As the time ranges differ, this is not an apples-to-apples comparison, but it gives a hint about how efficient bot filtering is.Â â†©
A NixOS module is a function receiving the configuration of the whole system as config and returning three attributes:
- imports adds other modules to import,
- options declares user-configurable settings, each with a type and a default value, and
- config sets values for options declared by any module, like containers from NixOS.
If a module does not need to declare options, you can return the config attribute set directly. When a module does not require any argument, you can define it as an attribute set instead.Â â†©
Most of the time, you donâ€™t need an explicit import. NixOS automatically imports the modules shipped with Nixpkgs. For my own modules, some machinery also imports them automatically.Â â†©
Litestream warns against using the auto-recover option as it can cause data loss. But I donâ€™t properly monitor my servers and I prefer uninterrupted backups to a slight chance of losing the last few records.Â â†©
In my case, the process is slow: around 20 minutes for a 20â€¯MiB database. You can test by restoring to a copy with the -o option, but you still need to stop the Litestream container.Â â†©

Andrew Cater: Debian 11 is at end of life from Long Term Support

noreply@blogger.com (Andrew Cater) — Sun, 20 Sep 2026 15:04:40 +0000

Lots of posts in the debian-user mailing list complaining about updates with Debian 11.11 suddenly failing.

See Debian 11 Long Term Support reaches end-of-life

August 31st, 2026

The Debian Long Term Support (LTS) Team hereby announces that Debian 11 bullseye support has reached its end-of-life today, 31 August 2026, five years after its initial release on 14 August 2021.

Starting in September, Debian will not provide further security updates for Debian 11. A subset of bullseye packages will be supported by external parties. Detailed information can be found at Extended LTS.

The Debian LTS Team is currently providing security support for Debian 12 bookworm, the current oldstable release. Thanks to the combined efforts of different teams including the Security Team, the Release Team, and the LTS Team, the Debian 12 life cycle encompasses five years. Debian 12 will receive Long Term Support until 30 June 2028. The supported architectures in Debian 12 LTS are amd64, i386, arm64, armhf and ppc64el.

For further information about using bookworm LTS and upgrading from bullseye LTS, please refer to LTS/Using.

Debian and its LTS Team would like to thank all contributing users, developers, sponsors and other Debian teams who are making it possible to extend the life of previous stable releases, and who have made Bullseye LTS a success.

If you rely on Debian LTS, please consider joining the team, providing patches, testing or funding the efforts.

Paul Tagliamonte: Why Write?

Fri, 18 Sep 2026 17:05:00 +0000

As a graduate of a liberal arts university, I wound up, unsurprisingly, taking a lot of classes in every possible academic discipline. Thinking back to the person that I was going into university, I don’t think I would have chosen to take them – after all, my degree was in the sciences; I’d have been stoked to do nothing more than wall-to-wall computer science until I ran out of coursework and filled the rest of my hours with research and independent studies with my professors (which, I guess I did actually do, just not as much as I would have otherwise).

Even this blog’s CSS theme (now 18 years old and starting to look it) was something I wrote after receiving and repeatedly re-reading a tattered third-hand copy of “The Laws of Thought” by George Boole. It was gifted to me by a college friend majoring in philosophy while I was crashing at his rental on the beach. He gave it to me because he knew I “liked that shit” and, while it was definitely computer science in nature, I would not have read it otherwise. I have vivid memories of sleeping on his la-z-boy surrounded by towers of books he was working through. He went on to do incredible work, getting his PhD, doing research, brilliant writing – his death in 2023 has robbed humanity of more time with him. “The Laws of Thought” sits behind me in my office, and is one of my most valued possessions.

My friends mean more to me than I could ever express, and I definitely don’t show it enough. Each of my classes made me a more thoughtful person. My professors made me a better, more well-rounded person – and a person who attempts to live up to the oft repeated credo of being “men and women for others”. I did the best I could, even though I was never a particularly good student. Without liberal arts, I don’t think I would, dispositionally, have been capable of pushing myself – to this day – to continue to learn on nights and weekends, for no reason other than wanting to learn. I refuse to stop expanding my perspective, and do my best to approach new problems with as much humility and curiosity as I can muster.

Every few days for the last year or so, I have been thinking back to a reading assignment from my junior year that, at the time, I thought was a borderline throwaway filler assignment for the class. The reading is an essay from 1947 by the french existentialist philosopher and libertarian marxist, Jean-Paul Sartre (as some of the more erudite may have now already worked out given the blog’s title), “Why Write?”. I re-read it this week. It is not filler. I remember this essay better than some of the classwork I considered more important at the time.

Why Write?

“Why Write?” starts off by describing the ways in which writing – trying to communicate your thoughts, opinions, or feelings to others – is an act of projection. The writer will only ever draw from a place of their “own subjectivity” – writing is taking your person and putting it on display. You’re choosing what words to use, how to use them, pulling from knowledge you’ve accumulated (based on how you’ve chosen to spend your time). Spending any amount of your finite existence in order to convey a thought is, itself, announcing to the world that you believe it to be a thought worth sharing.

Meanwhile, the act of reading is not simply turning letters into words. Reading is engaging with the work, and understanding the work in a process that looks more like what Sartre terms “re-invention” or “discovery” – “the literary object, though realized through language, is never given in language”. It is not enough to read the words in a book end-to-end; you must, as a reader, actively engage with the work to understand what is being communicated by those words. Reading is to take the words on the page, and “exceed” the mere words through what he calls “directed creation”. The reader is “re-inventing”/“discovering” the writer’s thoughts by following their “landmarks in the void”.

And this all makes pretty good sense to me – I know if someone is being sarcastic because I know the writer; I have read their words and have read into their words, allowing myself to be directed by the writer into “discovering” the thought they have left me in their work. I understand that their words are humorous, not irate. Knowing who the author of a work is can completely change the point of a sentence. The fact I’m writing about Sartre at all, or that this very writing has been constrained for presentation in this blog’s CSS is only happening because of who I am, what things I’ve experienced in life, and what has made me, me. Sartre argues that this dyadic coupling between writer and writing means that I, as a writer, can never truly read my own writing. I will never be capable of reading my own work and getting something out of it – the work is already an extension of my own person. I can discover no thought in my own works.

Of course, Sartre, as an existentialist, is honor bound to go one step further here – the reader, by participating in reading a work, is asserting their own freedom. Every time a writer writes “[…] the writer appeals to the reader’s freedom to collaborate in the production of [their] work”. The creative act may only be complete if both writer and reader have recognized one another and made a choice to do so. No one is compelled to complete the creative act. The things you read matter. Reading something fundamentally alters you as a person – you can not simply un-read a thought. Everything you read changes your universe. I have a hunch Sartre would find things like the (original early 2010s era) “tl;dr” reply to obviously terrible writing absolutely hilarious as a reader’s expression of freedom (not to be confused with modern 2020s era usage as shorthand for “summary section”). Using your freedom to choose to complete the writer’s creative act (or not) is inherently asserting your humanity.

I’m a bit fuzzy on the specifics of this quote, but I think it was sj who told me at one year’s Mystery Hunt that “A puzzle is a contract between puzzle author and puzzle solver, the author promises that the puzzle is solvable if you’re clever enough”. The puzzle author and puzzle solver are engaging in a collaborative production of the work that is only possible by recognizing one another. Similarly, Sartre – “whatever connections [the reader] may establish among the different parts of the book among the chapters or the words [the reader] has a guarantee, namely, that they have been expressly willed”.

Wall Drawing #123

It follows, Sartre argues, that the act of writing, as a means to convey a thought to a reader, may only be completed through the act of reading. Reading can only be done by others; writing, therefore, only exists to be read – it can serve no other purpose. Writing and reading are two halves of the same creative, collaborative act, only satisfied when both writer and reader acknowledge one another. Combined, writing and reading are the act of recognizing one another’s humanity, thoughts, experiences, consciousness – the act of co-creation of thought is the critical aspect of writing and reading – reconstructing the author’s perspective, thought, intent, point. The co-creation is the point of both reading and writing.

Writing without anyone to read the work leaves the writer’s bid for co-creation unsatisfied and humanity unrecognized. No thought has been conveyed to any reader, no one has understood the reason behind the “landmarks in the void” you’ve carefully placed, leaving you to either “[…] put down [your] pen or despair”. Writing is an appeal to the reader’s freedom, and the nature of that freedom is the writer can not control it – never being understood is always a possibility any time anyone sets out to write.

Nearly 12 years ago, I attempted to read the text on the git manpage generator for the first time – I remember the exact feeling of my brain going into “git manpage parsing mode” where every word was dragging git plumbing megaliths through the sand from their far-flung homes. I can still feel the surface of my desk as I instinctually started to trace out logical connections between git internals referenced as I read along. It took me a good 20 seconds to realize what I was reading made no sense. This website broke Sartre’s writer-reader agreement – I was attempting to read this website, but there was nothing there. You can “mechanically” “read” the git-man-page-generator, but you can never read it – it is not possible to read it. It was funny – unsettling. Each grasp at a real-looking image in front of me misses, my hand coming back empty. I couldn’t get enough of it. I must have tried to read a dozen generations in a row. I had never had something quite that broken pass that far into my consciousness before. Although I kept trying, it never did feel quite the same as that first time; I think fundamentally, I couldn’t forget that it was random.

While never having anyone read your writing drives them to “put down your pen or despair”, Sartre never had to contend with the opposite problem – being asked to read that which was not written. Reading that which was not written does not merely leave someone unrecognized when a reader makes a choice; instead, it is an inherent violation of the contract between reader and writer. Not only is no humanity being recognized through attempting to read words which were not written, but the reader, not the writer, is the one who bears the burden of unexpected apophenia. The reader, while engaging in co-creation, must now contort themselves to uphold their end of the Sartrean agreement, scrutinizing text to ascribe consciousness, thought and intent only to come back up with pareidolia-fueled echos of one’s own self and disfigured half-thoughts of others. The reality is, the modern reader must now “mechanically” “read” most written work they come across, scrutinizing text for any signs of thought before truly attempting to read the work. Failing to do this correctly changes our person – each time it happens, a small part of us is irrevocably altered. Choosing to engage in this ballet because you wish to do so is one thing – this is your right and freedom as a reader – but passing words which were not written as your own in an attempt to wrestle a reader’s freedom of co-creation from them is another entirely.

If you didn’t write it, I don’t want to read it (dw;dr). Send me your prompt instead.

Louis-Philippe Véronneau: Fake Cash: The World Trade Game

Fri, 18 Sep 2026 04:00:00 +0000

This is a new series¹ I'm starting on the tools I have been crafting and buying to play pedagogical games in my Economics classes.

These games, aside from being just plain fun for the students, often aim to teach them a lesson via interactive play. Unsurprisingly, it is much easier for them to grok a particular concept this way. Moreover, many students told me these activities were the best part of the classes I teach.

The World Trade Game

I found out about The World Trade Game in The Handbook of Pluralist Economics Education², which also contains many other fun pedagogical games. It was first developed by Action Aid, but has since been refined by John Sloman and then by Irene van Staveren, who wrote about her use of the game in the aforementioned book.

The goal of this blog entry is not to describe the game and teach you how to play it. If that interests you, either buy the book I mentioned or read about Sloman's version here.

No, our focus point here is the tools to play this game: fake cash! Fake cash can be used in many economics pedagogical games and in my opinion, is a worthwhile "investment" of your time and energy. You could indeed play with regular printed sheets of paper listing a value (I sure did!), but using Play Money (a more appropriate term, since we're not trying to defraud anyone) will up your game and create a more immersive atmosphere in your classes.

At first, I tried to buy play money. Surely, some commercial offering aiming to create realistic props for movies and TV productions existed? At least in Canada, it turns out the answer was no. The only thing I could find were shady websites and weird expensive listings on AliExpress. Not being tenured yet, I did not try to convince my employer to take a 50-50 chance on supporting criminal endeavours.

As such, the next best thing I found was printable play money designed by the Bank of Canada Museum. Their designs were quite nice and met my needs, but their PDF layout made it overly complicated to print large quantities of bills. As such, I played around with GIMP a little and came out with these files. A commercial printer shouldn't have issues printing them on glossy cardstock paper and cutting them up for you:

Once printed and cut, you'll unfortunately have to fold each bill by hand. To secure both halves, I ended up using a strip of transparent tape on each long edge.

Here is the result, with a real Canadian 20$ bill as a comparison:

Don't hold your breath though, as I do not currently have a release schedule. ↩
Reardon, J. (2009). The Handbook of Pluralist Economic Education. New York : Routledge. ↩

Russell Coker: Nheko DBUS

Thu, 17 Sep 2026 01:40:38 +0000

Nheko is my current favourite client for the Matrix IM system, which is my favourite IM system.

Matrix is an open system with end to end encryption and Nheko is free software and runs well on Linux desktops and phones.

The Nheko client allows interaction with dbus which could be good for automating things, EG you could change the status message when unlocking the screen. I’m documenting the most useful ones here because they don’t seem to be documented anywhere else. I have filed a Debian bug about the activate room option not working. The qdbus6 program is the QT6 version of the dbus command-line query program, there are a range of other programs which work in much the same way.

# list all interfaces
qdbus6 im.nheko.Nheko / 
# get the version of Nheko
qdbus6 im.nheko.Nheko / im.nheko.Nheko.nhekoVersion
# list rooms in a dump of the data structures (pity it's not json or something)
qdbus6 --literal im.nheko.Nheko / im.nheko.Nheko.rooms|less
# join a room
qdbus6 im.nheko.Nheko / im.nheko.Nheko.joinRoom "#flounder-random:luv.asn.au"
# supposed to activate a room but doesn't
qdbus6 im.nheko.Nheko / im.nheko.Nheko.activateRoom "#flounder-random:luv.asn.au"
# set the status
qdbus6 im.nheko.Nheko / im.nheko.Nheko.setStatusMessage "whatever"
# get the status
qdbus6 im.nheko.Nheko / im.nheko.Nheko.statusMessage

Here are a couple of examples of using other dbus clients to get similar results. Note that the difference between the Debian version of Nheko (and maybe other recent versions) and what LLMs return for usage examples is that Debian has “/” as the path while the examples have “/im/nheko/Nheko”.

# list rooms via gdbus
gdbus call --session --dest im.nheko.Nheko --object-path / --method im.nheko.Nheko.rooms
# get status via dbus-send
dbus-send --session --print-reply --type=method_call --dest=im.nheko.Nheko / im.nheko.Nheko.statusMessage

Paul Tagliamonte: DESFire EV3

Tue, 15 Sep 2026 13:25:00 +0000

I’ve long been interested in hardware key material storage devices. I’ve been a fan of yubikeys (I still remember when my fancy new NEO-N showed up), PIV (and its associated smattering of additional fields), SaaS HSMs, the kernel keyring, some tooling I’ve fairly satisfied with the design of at prior companies, and of course, our dear friend, the TPM. All that is not even to mention the scores of exotic hardware security modules one generally comes across from time to time when you’re keeping a sharp eye out that you wind up playing with.

The concept of storing private key material on a disk, or even having it in RAM has always skeeved me out, so I have a natural inclination to hardware modules, and how shifting keying material around can change your risks and threat model(s) in interesting ways.

I don’t remember when I first came across the MIFARE DESFire EV3, but a few weeks ago I did a deep-dive into the state of the art of authentication schemes using ID cards. My complete overview of what tradeoffs exist is pretty extensive (and likely not interesting to the vast majority of the world), but the tl;dr wound up being one of “use PIV” or “use MIFARE DESFire EV3”. I wound up picking DESFire for a recent project, and figured it’s worth talking a bit about what I learned, share some thoughts, and some code. That code is published on crates.io/desox, and docs, as is our custom, may be found at docs.rs/desox

DESFire supports DES (I’m sure most readers saw that one coming), 3DES (I didn’t bother playing with 3DES at all) or AES-128 (AFAICT always use this?) keying material. It’s worth noting that the DESFire only supports symmetric keys and is not designed for public key cryptography, and operates exclusively using shared symmetric key material. The DESFire EV series use those keys and related authentication schemes to interact with “files” stored on the on-chip EEPROM (2k, 4k, 8k, and 16k versions exist), or “applications” (groups of files and authentication keys).

Talking to a DESFire EV3

Interactions with the card are done over NFC (ISO/IEC 14443 Type A), and commands to/from the card may be in the usual ISO/IEC 7816-4 APDU format, or “unencapsulated” bytes sent to/from the card are sent using a fixed instruction set and return code structure – saving a few bytes per message. I’ve opted to use their undocumented and proprietary format – I found it easier to work with and with a maximum message of 60 bytes, the savings matter a lot.

While powered via NFC, the card maintains a small amount of state about the connection between the reader and the card in its RAM, including if the session is authenticated or unauthenticated. I’ll dig into how authentication happens later, but it’s worth knowing that sessions can become authenticated using one of the symmetric keys shared by the card and the reader. The vast majority of the DESFire commands I know about tend to work while either authenticated or unauthenticated, with a few exceptions (GetUid, ChangeKey, and ChangeKeySettings for example).

In general, I found working with this card particularly pleasant. There is a fair amount of backwards-compatible behavior and multiple methods of communication that confuse things a bit, but overall, it was better than average to integrate with. Kudos to the NXP team. If the docs on this chip were public, things would be orders of magnitude easier – it’s not entirely clear to my why they’re keeping so much of the interface documentation under NDA, but it’s the largest knock against the chip, by far.

Authentication

I found a lot of really great resources outlining how the handshake and protocol works for a DESFire EV3, especially from Ridrix, some public datasheets ThrRealRevK and posts from AndroidCrypto.

The gist here is that, because the DESFire only does symmetric key operations, the key exchange (a type of SKA – Symmetric Key Agreement) uses symmetric keys to establish a unique session key which is used to sign or encrypt data exchanged between the reader and the card. I’m not going to get too in-depth here, since there’s a ton of other resources out there to dig into – but I will do a quick high-level description to keep this post mostly self-contained.

The authentication protocol serves two main functions – to verify that both parties know the same shared secret, as well as to act as a SKA to construct a new session shared secret key. Here’s a quick overview of how a shared session key is derived between the reader and the card using our symmetric keys (AES-128 in the case below).

the reader requests to start authentication with the card (something like AA 00 to start an AES Authentication handshake with keyslot 0x00).
The card will then reply with AF (a status code that indicates more data is to follow), followed by 16 bytes (in the case of AES-128) of encrypted (using CBC) data.
The hosts then decrypts this block with the symmetric key from keyslot 0, returning the card’s session nonce.
The host generates 16 bytes (usually random) for its session nonce.
The host sends an instruction of AF (indicating a continuation of the previous command), followed by 32 bytes of encrypted data. When decrypted, the first 16 bytes are our nonce generated in step #4, followed by the 16 bytes provided by the card, decrypted in step #3, except where every byte is shifted to the left by one place (the 0th byte is copied to the end).
The card will reply with 00 indicating a successful operation, followed by 16 bytes, which when decrypted, is our session nonce from step #4, shifted to the left by one byte in the same way that we did in step #5 with the card’s nonce.
At this point, both the reader and card have confirmed the other party has the same symmetric secret key. The session is now “authenticated” and a “session key” is derived using the two nonce blocks. Two hashing keys (K1 and K2) are derived from this key, which is used to maintain an ongoing CMAC hash of the messages coming and going to/from the card.

From here on out, the session is “authenticated”, and responses from the card which were previously “plain” will now contain a 8-byte CMAC signature, which can be used to ensure that the replies in question come from the active session.

In my implementation of the handshake I opted to encode the handshake state into rust types, just so I wouldn’t make any mistakes. The Handshake type contains the session internals (session nonce values, keying state, to include IV, etc). This means the authentication flow (from within my code) uses the Handshake struct to generate the commands to send to the card in order:

/// Create a new `Handshake`, and return the
/// start auth command (something like `AA 00`)
fn Handshake::<Initial>::begin(
 output: &mut [u8],
 key: [u8; 16],
 key_id: u8,
) -> (Self, &[u8]);

After we get a reply back from the card (the encrypted version of the card’s session nonce, sometimes called Rnd_B in code I’ve seen), we transition states from Initial into HalfOpen.

/// Given the card's encrypted response, generate
/// our session nonce and generate a reply
/// (something that starts with `AF` followed by
/// 32 bytes of encrypted data).
fn Handshake::<Initial>::rnd_b(
 self,
 output: &mut [u8],
 input: &[u8]
) -> (Handshake::<HalfOpen>, &[u8]);

Now that we’re “HalfOpen”, we’re waiting to hear back from the card to ensure that it, too, can byte-shift our provided nonce. Once we have the card’s reply, we can check it using our complete helper, transitioning from HalfOpen to Successful.

/// Check to ensure that the card replied with
/// our nonce byte-shifted by one place, indicating
/// that they know the symmetric secret in
/// this key slot.
fn Handshake::<HalfOpen>::complete(
 self,
 input: &[u8]
) -> Handshake::<Successful>;

Once the Handshake is successful, the only thing left to do is consume the Handshake struct and turn it into the shared session key by running it through the key derivation function.

/// Consume the `Handshake` struct and return the
/// new shared session secret key.
fn Handshake::<Successful>::into_key(self) -> [u8; 16];

From here on out we can use this session key for the remainder of our interactions with the card – signing messages from (and sometimes to!) the card, or encrypted messages to and from the card. This key is used in CBC block mode, where the session IV is updated with the last block of the encrypted data.

Unit Testing

A nice proprietary of the SKA scheme we’re using as part of DESFire is that the derived session key is actually deterministic if you control your nonce RNG (ok, actually, pretty true for most key agreements, but anyway), which means it is possible to capture traffic over the NFC interface, and “replay” the NFC I/O with cooked RNGs and ensure byte-identical messages and keys are generated. Within desox-rs this is called replay (I’m creative), and I’ve got a few replay sessions checked into VCS, which exercise a signficant amount fo the API surface. All were derived from an actual session with a real DESFire card, and can be updated with a live card and a --cfg flag.

Each replay file is a set of lines (request-response transactions), each containing two space-delimited hex encoded NFC messages. For instance, here’s an authentication handshake in replay format:

1a00 afc7bbd82ff8fefae8
afc6dab54df2278d2952d560821be7e4c3 007d9abe94a9b14748

The code that generated that exchange came from the test stored adjacent to that file – a handshake with the default DES key (all zeros), and an RndA value hardcoded to 32c28fdafd3960de.

let mut card = card
 .authenticate_with_rnd_a(
 0x00,
 Key::Des([0; 8]),
 Key::Des(hex_literal::hex!("32 c2 8f da fd 39 60 de")),
 )
 .await
 .unwrap();

Since the card’s RndB is similarly unchanging (I’m replaying this file every time), this will always derive the same session key, which means messages (including encrypted ones or CMAC signed responses) will be identical, as well. If you’re playing with the DESFire yourself, feel free to grab my replay files if you need a “known good” baseline.

By default this will run using the MockBackend, replaying each file – expecting a byte-identical request, and responding with the harcoded customary reply. If the code (or test!) needs to change, updating the tests is done by swapping the MockBackend out for a real one. Since I had to do this a bunch during development, running cargo test with RUSTFLAGS="--cfg desox_replay_rw" will, on run, overwrite the replay file(s) for the executed test(s), ensuring all line-protocol changes are explicitly caught and reviewed.

Observations

Most commands, even ones which require authentication, are transmitted without CMAC signature(s) or encryption. CMAC signatures from the reader to the card are not really used (except for writes to a file which specifies communication must be CMAC signed), ditto for encryption (although that one is used for key change operations, in addition to file writes on files that specify encrypted communication must be used). The vast majority of commands take a “plain” request from the reader, and return a CMAC signed response.

By my eye, this means that a malicious reader, or something otherwise capable of holding the card online after communication with an authentic reader is complete are able to execute privilaged commands (since one can simply ignore the CMAC signatures on responses), so long as the command doesn’t require the reader to provide CMAC signatures (or encryption), or allow the card to power down.

Fun with DESFire

I’ve played around a bit with ways to use the DESFire cards in interesting configurations, given what they’re capable of. Here’s some half-baked thoughts I had while mucking around with the cards – these are all poorly thought out sketches of some things we can do given the specific tradeoffs I see with the DESFire card. It’s also worth noting that I don’t have any of the actual documentation, and am not a cryptographic grown-up, so take these sketches with a massive grain of salt.

The first thing that came to mind when implementing this is how the authentication scheme can shift the boundary of what is and is not trusted (assuming good secure keying, and provided the key slots and card/application permissions are configured correctly). Rather than push the key material out to the machine connected to the NFC reader (“reader machine”), I instead tried turning the NFC reader and computer into something psuedo-untrusted by “merely” having it pass messages from the card to a trusted remote system (“remote machine”). This means that the “reader machine” is exchanging NFC data with the card, but that data is being decrypted, encrypted and processed by the trusted “remote machine” – the reader is unable to derive the session key.

For each of these, I wind up needing to authenticate – so there’s still a few latent risks, but these can mostly be mitigated by asking for a readbacks of any changed file(s), setting key permissions carefully, and requesting the card’s UID via the encrypted channel – all of which would require the symmetric secrets (which undermine the whole security model if comprimised).

This general construction is also subject to a hostile takeover of the untrusted “reader machine”, since most commands (including destructive ones!) are sent in “PLAIN” mode – the reader machine can wait until authentication is complete and then inject commands into the card and “simply” ignore the CMAC signatures on responses, severing ties with the remote machine. As such, we also need to take steps to ensure that the key being used is not one that allows any access beyond what is allowed. Here were some ideas I sketched out off the back of this theory.

The “second-factor”

Given some established (and authenticated) connection, part of the initial authentication flow may use the DESFire card to prove physical control over it as part of a handshake. This can serve as a second factor during some authentication flow, requiring physical card presence at a reader to fully initialize a connection. This does have one glaring downside, however – it’s phishable. To use this “for real”, we’d need to take some steps to prevent obvious MITM flows (XOR the NFC messages with the URI as seen by the client?), but maybe there’s something interesting there.

This also has a second interesting attribute – when used as part of a physical system authentication flow, this becomes a logical place to inject access control, being able to determine if some person is permitted to operate some device at that particular time (Is “Joe” current on his Laser Cutter certifications?) I think of the ideas I landed on, while conceptually interesting (using an employee id card as a 2FA token, it’s very fast), this one is the least likely to turn into something real.

This construction, when paired with an encrypted DESFire file, allows the “remote machine” to read/write an ’encrypted cookie’ to the card – storing small amount of encrypted data that the “remote machine” can read/write, but not the “reader machine”, since this uses an encrypted and authenticated channel from the “remote machine” directly to the DESFire card, without any intermediate hosts needing to be fully trusted. I keep calling this the “encrypted cookie” in my head because it feels conceptually similar to how Ruby on Rails and Laravel handles cookies.

We’d need to take a few extra steps here (for instance, ensure that you read the cookie back over the encrypted channel after writing to prevent a malicious reader from dropping writes) to secure the system, but it feels like the structure of this is definitely decent.

The “takeover”

This time, let’s say the computer attached to the NFC reader (“reader machine”) is semi-trusted. For this scheme, our trusted “remote machine” and the “reader machine” pass messages over the network to handle authentication to the card (as above), where the handshake data is being decrypted, encrypted and processed by the trusted “remote machine” as usual. However, once the authentication handshake is complete and a session key has been derived, the “remote system” return the session key to the “reader machine”, giving it a one-time-use key and authenticated session to the card.

We need to be careful about global/application permissions and key access control to files – but in this construction, we can allow the “reader machine” to take over privileged actions using a scope-limited DESFire key without handing over the card’s true keying material (preventing cloning of the card). This can be helpful to ensure messages to/from the card are truely from the card (verifying CMAC signatures), enables the “reader machine” to directly read/write to/from encrypted file(s), but allows the symmetric key material to remain in as few places as possible – which is critical given compromising that secret will undermine the security of the entire system.

Yves-Alexis Perez: IKEv1 protocol disabled in strongSwan package for Debian unstable

corsac@debian.org (Yves-Alexis) — Tue, 15 Sep 2026 07:28:41 +0000

Heads up, Debian IKE/IPsec users.

Starting with strongSwan 6.1.0-1 (currently in Debian unstable and targeted at Debian 14 Forky), the IKEv1 protocol has been disabled. This is aligned with upstream decision. Considering IKEv2 is already nearly old enough to drink in the USA (RFC 4306 will turn 21 next December) and IKEv1 has weaknesses, the disabling is long overdue amd will permit upstream to remove some code in the upcoming years.

At this point there is no good reason not to migrate to IKEv2 and exposing IKEv1 code in all Debian installation is no longer relevant. All IKEv1 users using Debian 13 Trixie (either site to site, gateway or roadwarrior client) should investigate IKEv2 protocol (or other options).

Note that some plugins have also been disabled upstream for security/maintenance reasons and we followed suite in Debian. The Debian relevant ones are: af-alg, led, padlock.

Reproducible Builds: Supporter spotlight: Jochen Sprickerhof on ... Reproducible Builds!

Tue, 15 Sep 2026 03:54:00 +0000

The Reproducible Builds project relies on several projects, supporters and sponsors for financial support, but they are also valued as ambassadors who spread the word about our project and the work that we do.

This is the ninth installment in a series featuring the projects, companies and individuals who support the Reproducible Builds project. We started this series by featuring the Civil Infrastructure Platform project, and followed this up with a post about the Ford Foundation as well as recent ones about ARDC, the Google Open Source Security Team (GOSST), Bootstrappable Builds, the F-Droid project, David A. Wheeler, Simon Butler and Kees Cook.

Today, however, we will be talking with Jochen Sprickerhof, one of the newer members of the Reproducible Builds project core team.

Vagrant Cascadian: Could you tell me a bit about yourself? What sort of things do you work on?

Jochen Sprickerhof: I am a freelance programmer working on Open Source. Mainly doing Debian, F-Droid and some smaller software projects. In general I made it a habit to look into every software I use and try to fix bugs or add features I need. In Debian, I maintain about 180 packages with topics covering home banking, build systems and robotics. Most of my time, I currently work on reproduce.debian.net, where we try to bit-for-bit reproduce the packages distributed by Debian.

Vagrant: Could you describe the path that lead you to working on reproducible builds?

Jochen: I started my Debian journey as a teenager, converting my school to Debian and serving as its system administrator for 13 years. After studying Applied System Science, I joined the university’s robotics labs, where I worked on the Robot Operating System (ROS) and the Point Cloud Library (PCL). In the end, I enjoyed programming more than writing papers, so I eventually left academia for a robotics startup. Some years ago, I realized that the open source work I was doing in my spare time was actually the work I cared most about. Nowadays I am really grateful that I can spend my days working on things I find important and have lots of fun with.

Vagrant: What projects did you recently make big progress on?

Jochen: A recent example is metasnap.debian.net. It is a ‘meta archive’ of snapshot.debian.org which is itself archive of all packages in Debian. But let me explain it the other way round: with reproduce.debian.net, we try to reproduce the packages as they are distributed by the Debian archive. For that, we need the same build environment (compilers, libraries, build tools, etc) that was used by Debian back when the original package was compiled. Luckily, snapshot.debian.org has all those packages, but they are not easily accessible via apt, Debian’s package manager. So, metasnap provides a mapping from a package name and version pair to the APT repo on snapshot.debian.org needed to download it from. It was created by josch some time ago, and it’s awesome work. But when we tried to reproduce more and more packages on reproduce.debian.net, we found that some were missing packages from the build environment — even though they where visible on snapshot.debian.org. We found that metasnap excluded some archive areas because they where not expected to be needed. Reimporting all the data took more than two months and surfaced a couple more flaws.

With this fixed, we were able to build more packages, only to find out that metasnap also needs better support for version numbers. Luckily we were able to rewrite the data in a day instead of starting the import again.

Vagrant: You have been working on infrastructure to support reproducible builds for a while. Has recent adoption of reproduce.debian.net into the Debian release tooling changed the focus of your work?

Jochen: Quite a bit. When we started reproduce.debian.net in 2024, only around 33% of the packages could be reproduced successfully. Today we are above 98%. Most were not bugs in the packages themselves but in the infrastructure. Similar to the metasnap issue I reference above, packages just needed a rebuild because something else, like the toolchain, was fixed in the meantime. In May, people from the Debian release team and the Reproducible Builds project sat together and decided that the overall state is good enough, and now packages that regress on reproducibility are blocked from entering the next Debian release. But that does not mean all the work is on the shoulders of Debian package maintainers. Since then I have been constantly looking at the migration tooling to spot regressions and provide fixes. Furthermore, a couple of maintainers reached out to us for help and I hope more will do so in future.

Vagrant: What is one small thing you (or others) have not yet gotten to that you would really like to see?

Jochen: The central tool to reproduce Debian packages is debrebuild, also written by josch. Currently it has two ways to retrieve the build dependencies of a package. Either it uses metasnap.debian.net (as explained above), or it can access the Debian unstable APT repository directly. This allows to test packages locally before everything is indexed on metasnap by compiling against Debian unstable. But actually there are many other APT repositories to query, like Debian stable or even derivatives. Adding support for an optional list of APT repositories in debrebuild would be great. That would also be a big step to support reproducing other Debian based distributions.

Vagrant: … and one big thing?

Jochen: It would be great to integrate metasnap.debian.net into snapshot.debian.org. There is some discussion on it already in Debian bug #650783.

Vagrant: What are the tools you use the most?

Jochen: According to my fish shell history:

$ history | cut -d' ' -f1 | sort | uniq -c | sort -nr | head -10
git
vi
rm
cd
ls
apt
grep
mv
dpkg
cp

Vagrant: So, is the fish shell reproducible? I remember it did not used to be…

Jochen: You can check for yourself — it was last time I checked. But looking through the other commands, neovim sadly is not. I hope we can fix that in future.

Vagrant: Oh, that’s a nice URL to check for reproducible package… you can just pass the source package name to check the current results?

Jochen: Yes. Another one is udd.debian.org/reproducibility/, where you can list all packages of a Debian maintainer. It also lists source reproducibility and has nice filters as well.

Vagrant: What tools do you use specifically working on reproducible builds?

Jochen: I don’t have statistics for that, but I would say sbuild to build the package, debrebuild to reproduce it, and diffoscope to analyze the differences. Obviously I also need run apt source <package> or use git-buildpackage to get the sources and all the tools I mentioned above.

Vagrant: So how many packages are left to build reproducibly, and once those are finished, what is next?

Jochen: Right now, reproduce.debian.net shows over 98% reproducibility, though there are still over 650 package left and some will probably need a lot of work. But actually I think making packages reproducible is just the first step. For me, this is a project to build confidence in the system. To reproduce a package we have two parts: the source of the package and the build environment. Fixing the packages means gaining confidence in the first part but we still rely on the individual build environments for each package as we need to use the same compiler that was used when the package was build initially. Because of this, we have to keep around every historical version of all toolchain packages. I really would like to remove this extra archive, which means we would have to rebuild all of Debian around release time. I am dreaming of a Debian release where you could bit-for-bit reproduce every package just from the released versions. Due to how Debian works, however, this is not a trivial rebuild and it would need some work on the infrastructure. By the way, initially there was a third component to pay attention to: any connection to the outside world during the build. Luckily we fixed the Debian build daemons to not allow network connections during the build some time ago.

Vagrant: Thanks for all that work, and taking the time to tell us a bit about yourself!

Jochen: Thanks a lot for the interview!

For more information about the Reproducible Builds project, please see our website at reproducible-builds.org. If you are interested in ensuring the ongoing security of the software that underpins our civilisation and wish to sponsor the Reproducible Builds project, please reach out to the project by emailing contact@reproducible-builds.org.

Reproducible Builds: Reproducible Builds in August 2026

Fri, 11 Sep 2026 10:12:50 +0000

Welcome to the August 2026 report from the Reproducible Builds project!

In our reports, we try to outline the most important things that we have been up to over the past month. As a quick recap about what problem our project intends to solve, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries. The motivation behind the reproducible builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised or not.

In this month’s report, we cover:

New updated SBOM specification from CISA.gov

CISA, the Cybersecurity and Infrastructure Security Agency of the U.S. government published some the joint guidance entitled Minimum Elements for a Software Bill of Materials (SBOM), which updates and supersedes the baseline 2021 version covered in previous editions of these reports.

Whilst the PDF is worth skimming, the interesting changes include that the specification now mandates standard cryptographic hashes: unlike earlier standards that allowed hash omission or manifest-only parsing, hashes must be computed from the output. This is is important for reproducible builds, as it ensures the recording of the metadata required to demonstrate the shipped software matches the build output precisely where applicable. In addition, where the top-level only dependency limitation that was present in the 2021 version has been removed in favour of complete coverage with no minimum depth. That is, SBOMs are expected to reflect all linked libraries, vendored dependencies and other build-time inclusions.

LWN on Bootstrappable builds at FOSSY 2026

In the “Toolchains and Other Development Tools” track at Software Freedom Conservancy’s FOSSY 2026 in British Columbia, Canada, Timothy Sample gave a presentation on bootstrappable builds. This presentation was then covered in a Linux Weekly News article by Jake Edge entitled Bootstrappable builds: how and why, which serves as an excellent introduction to the concept:

The basic idea behind bootstrappable builds is to create a system that can be built without relying on pre-built artifacts. “Can we go from zero to the modern day without having to just assume the existence of these already-built-for-us artifacts?” The classic recipe for yogurt requires some yogurt to start the process, which is like how we normally build a C compiler today—we start with an existing C compiler binary. You might think about making sourdough bread with your grandmother’s starter brought over from the old country; “we’re basically making C compilers with Dennis Ritchie’s starter carried over from Bell Labs”.

The article, which goes on to cover GNU Mes and other projects that overlap with Reproducible Builds, also has a number of thought-provoking comments.

”What’s missing to have reproducible builds on PyPI?”

Core Python developer, Brett Cannon wrote an interesting blog post this month addressing What’s missing to have reproducible builds on PyPI, the official public repository for third-party Python software packages:

The reason I like the idea of making reproducible builds work is that I think it can be done in such a way as to not require any work on the part of the producer of a distribution (which is a technical term for sdists or wheels, i.e., the people who upload stuff to PyPI), and thus make reproducible builds very low-friction for people to opt into supporting. […]

Brett goes on to outline “What’s missing from the specs” and how reproducibility might be visible on PyPI to consumers:

Assuming all of this comes to pass and we record the where the source code is that went into a distribution and the software used to make the distribution, how do we make it useful to people? Does every person who cares about having a secure supply chain have to rebuild everything they use themselves? Is there some way for even people who don’t care about this stuff to benefit? […]

Distribution work

In Debian this month, 23 reviews of Debian packages were added, 28 were updated and 27 were removed this month adding to our knowledge about identified issues. A number of issue types have been updated as well, such as the addition of a new toolchain issue related to python-traitlets […], and the note for an existing issue related to texi2html was updated as well […].

Lastly, Bernhard M. Wiedemann posted another openSUSE monthly update for their reproducibility work there.

Unreproducible builds under EROFS filesystem fixed

Martin Pitt reported on Fosstodon that they had identified an issue where the mkfs (“make filesystem”) command for the EROFS (Enhanced Read-Only File System) subsystem of the Linux kernel did not have sorted extended file attributes, leading to reproducible builds.

Thankfully, Martin also reported that they had fixed this in a commit to the kernel which “order[s] each inode’s xattrs by name so that images stay reproducible”. […]

Tool and documentation development

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes, including preparing and uploading versions 327, 328 and 329 to Debian. In particular, he ensured that diffoscope did not require python3-guestfs in the autopkgtests on 32-bit architectures in order to fix Debian bug (#1144372) […]. Colin Watson made an additional change, handling a potentially missing openssh-client package when running the autopkgtests […], and Jochen Sprickerhof made a similar change to cope with missing cpio and qemu-img functionality […] whilst also updating the XML comparator to be considered when comparing SVG images […].

Yet again, there were a number of improvements made to our website this month as well. For example:

Chris Lamb added draft for a Gothenburg summit-related news article. […][…]
Holger Levsen then published the same article. […][…]
Lastly, a large number of commits were pushed comprising an interview with Reproducible Builds developer Jochen Sprickerhof to be published within the next week. […][…][…][…][…][…][…]

Six new scholarly papers

Jens Dietrich, Spencer Sun, Tim W. White and Behnaz Hassanshahi (the result of a collaboration between Victoria University of Wellington and Oracle Australia published a paper this month entitled No Snake Oil: Verifying Python Package Builds. Drawing on the metaphor of “snake oil”, that is, a fake or ineffective medicine or solution sold with exaggerated claims of curing or fixing everything, the authors write that

Two tools that are designed to automate [PyPI] rebuilds and run them at scale are macaron and oss-rebuild. We study 12,180 popular releases from PyPI and find that the byte-for-byte equivalence rate is generally low. We analyse the reasons why they produce different wheels, and find that equivalence between the original and rebuilt wheels can often still be established, preserving most of the guarantees users expect from rebuildable releases. We present and evaluate daleq4py, a tool to establish the equivalence of Python wheels through the kernel of a normalisation function that is based on provenance-preserving datalog rules. Experimental results show that daleq4py substantially expands the set of rebuilds that can be accepted as equivalent.

The full PDF of their paper can be viewed online, and Jens Dietrich to our mailing list to announce the availability of both the paper and the daleq4py tool itself.

Dimitri Kokkonis, Michaël Marcozzi and Stefano Zacchiroli published an article this month titled Not In My Git Yard: Catching Backdoors at Commit and Release Time on the topic of “code-level backdoors” — that is, “stealthy code changes that grant hidden privileges via secret triggers”. These issues:

… pose a persistent threat to opensource software. Known attempts to inject such backdoors into widely used projects through malicious commits, tampered release packages, or compromised third-party dependencies, were stopped only by luck and manual review. Existing Continuous Integration (CI) pipelines cannot detect these attacks, and downstream binary analysis tools require substantial manual effort. In this work, we present Lily, an automated approach that strengthens open-source development and release processes against backdoor injection. Lily integrates a backdoor detection mechanism into (1) CI pipelines to block malicious commits, and (2) release vetting workflows to prevent tampered releases or compromised dependencies from entering large ecosystems, such as Linux distributions.

The full PDF can be read online.

Ranindya Paramitha and Laurie Williams of North Carolina State University along with Christian Kästner of Carnegie Mellon University published a paper this month with the title of The Software Supply Chain as a Market for Lemons: A Multivocal Review of Trust Signal Collapse. (A “lemon” in American English, is a vehicle that “turns out to have several manufacturing defects”.) Their abstract is as follows:

Practitioners evaluating open-source dependencies rely on cheap trust signals, e.g., stars, download counts, and contributor activity, as substitutes for direct code inspection, assuming those signals reflect genuine trustworthiness. Prior work has documented individual signal gaming, but the landscape of collapses across all dependency-adoption signals, as well as the ecosystem’s response, remains unexplored. The goal of this study is to aid software practitioners in understanding the reliability of dependency adoption trust signals, such as download counts and contributor activity, by conducting a multivocal review of 252 Google Search sources and 870 Reddit threads.

Worryingly, after their review, the authors conclude that “cheap trust signals collapse under three simultaneous forces: adversarial manipulation, gaming techniques indistinguishable from legitimate behavior, and non-adversarial AI-driven inflation.”

The full PDF of the paper is available online.

Julien Malka, Aman Sharma, Martin Monperrus, Stefano Zacchiroli and Théo Zimmermann published a paper this month on Trusting-Trust Attack against an Entire Linux Distribution through Binary Manipulation:

Ken Thompson’s trusting-trust attack, in which a compromised compiler backdoors the programs it builds and reproduces the backdoor in subsequent rebuilds of itself, is widely regarded as a threat specific to compilers. We show that it is not. We construct a complete trusting-trust attack around GNU strip, an ordinary build utility that neither inspects nor generates source code, using only manipulations of finished ELF files.

Scarily, in the authors’ example, “a single tampered strip in the binary seed implants a payload that propagates from one generation of strip to the next and survives into the final standard environment after the seed leaves the dependency closure […] without failures and backdoors”.

A full PDF of the paper is available for download online.

Mehdi Keshanimm, Amirhossein Rahmati, Mohammad Hossein Aref and Abbas Heydarnoori published a paper that is currently under review at Emperical Software Engineering titled AROMA+: A Study of Factors Affecting Reproducible Builds in the Maven Ecosystem. (Maven is a/the build automation tool used for Java projects.) In their paper, the authors note that

[…] reusing external software in a project presents a security risk when the source of the component is unknown or the consistency of a component cannot be verified. The SolarWinds attack serves as a popular example in which the injection of malicious code into a library affected thousands of customers and caused a loss of billions of dollars. […] Our research aims to support [reproducibility] efforts in the Maven ecosystem through automation. We investigate the feasibility of automatically finding the source code of a library from its Maven release and recovering information about the original release environment. Our tool, AROMA+, can obtain this critical information from the artifact and the source repository through several heuristics and we use the results for reproduction attempts of packages on Maven Central.

The full PDF of their article can be downloaded online.

Lastly, Oreofe Solarin, Kelechi Kalu, James C. Davis and Paschal Amusuo published a paper this month titled Reproducibility is Not Enough: Artifact Verifiability in Decentralized-Build Package Ecosystems:

[A]rtifact verification requires more than deterministic builds: a verifier must also recover the source state, build environment, dependencies, and build instructions that produced the artifact. Decentralized-build ecosystems make this difficult because artifacts are produced through heterogeneous tools, maintainer-controlled workflows, and fragmented metadata. As a result, it remains unclear how often artifacts in these ecosystems can be independently verified. This paper studies artifact verifiability across four popular decentralized-build package ecosystems. We define an independent verifier model that relies only on registry-derivable metadata and an artifact comparison model with tiered equivalence levels. We implement these models in an Artifact Verification Pipeline and use it to measure artifact verifiability across the target ecosystems.

The authors conclude that “beyond build determinism, verifiability is limited by missing source and build metadata, implicit release transformations, and unconventional build practices”.

A PDF of their paper can be reviewed online.

Patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where applicable or possible. This month, we wrote a large number of such patches, including:

Bernhard M. Wiedemann:
- Botan
- Rivet
- boost
- ceph
- cosmic/xdgen
- cpio
- dmd
- dvgt
- efl
- firefox-esr
- fritzing
- gcc/esbuild
- git-annex
- go1.27/esbuild
- gputils/sdcc
- grass
- gri
- java-21-openj9
- java-25-openj9
- ksh
- llvm22
- llvm
- llvm
- lutgen
- maxima
- mingw64-filesystem
- neochat
- obs
- python-PyMuPDF
- python-gguf
- python-langgraph
- python-pook
- python-pyzmq
- python-xgrammar
- python-xlsx2csv
- qt6-tools
- sbcl
- scons
- virtualbox
- xpenguins
- zabbix
Chris Lamb:
- #1143615 filed against keychain.
- #1143616 filed against python-nameparser.
- #1143617 filed against lcov.
- #1144065 filed against displaycal-py3.
- #1144066 filed against redmine.
- #1144407 filed against django-htmx.
- #1144409 filed against pybdsf.
- #1145914 filed against golang-github-adamkorcz-go-fuzz-headers-1.
- #1145915 filed against googletest.
Jochen Sprickerhof:
- #1145443 filed against cross-toolchain-base.
- #1146088 filed against ferret-vis.
Robin Candau:
- bashburn
- LMMS
Werner Fink:
- bash

If you are interested in contributing to the project, please visit our Contribute page on our website.

Matthias Klumpp: JPEG-XL as default in AppStream, and better media processing

Thu, 10 Sep 2026 17:48:00 +0000

Two weeks ago, I released AppStream 1.2.0. This release contains a lot of great changes, but one of the most important ones concerns how media are being handled, and AppStream’s default image export format.

AppStream is a Freedesktop metadata standard to describe software components. That can be anything from system services over fonts to console and graphical applications. AppStream metadata is supposed to give users enough information to decide whether they want to install a piece of software, to represent that piece of software, and to give the operating system enough information to decide whether a software component should be installed automatically and (to some extent) what capabilities and relations it has, to provide the user with sensible options.

Especially for the first two goals, and especially for GUI applications, AppStream supports icons and screenshots, which are used to showcase applications. Today, AppStream is used by all kinds of services, from Linux distributions over firmware updates to Flatpak and desktops directly. AppStream’s original design however comes from the perspective of Linux distributions in 2011, where you may want to browse the software catalog offline, without delay, and without pinging an external server (which could be a privacy concern).

Therefore, a common way to deploy an AppStream-enabled software repository is to ship all icons of all applications in the repository to the user as part of the repository metadata download. AppStream does support remote icon downloads nowadays, and for a while I thought that this would become the default eventually. However, especially in today’s world, having a bandwidth-saving, instantly responsive, privacy-protecting application browsing experience seems more important that ever.

PNG images are great!

The only format that AppStream supports for icons and screenshots (which are downloaded on-demand from your distributor’s CDN) has always been exclusively PNG. PNG images are perfect for icons, because they compress well (especially for common icon shapes), are fast and simple to load, and can be loaded anywhere, by any toolkit or webbrowser. They also ensure we deliver faithful screenshot images, even though we may have scaled or re-rendered them. Still though, PNG images are less great for screenshots, as they are not very efficient, which puts strain on any CDN that has to deliver them, as well as on people’s internet connections when browsing screenshots. Having smaller thumbnails alleviates that problem a little, but does not fully solve it.

But even for icons, PNG could be improved upon: In many cases, icons are re-downloaded with the repository metadata again and again, so having a large icon tarball adds up to the data transferred during metadata refreshes. AppStream also now supports large 128x128px icons, which nobody in 2012 expected we would need, adding even more data that will be re-downloaded. Saving some space here translates directly to lower bandwidth costs as well as faster downloads for users.

To improve PNG file sizes, the AppStream Compose library, which handles all image processing and metadata catalog composition, was running optipng on all generated PNG images. That does create smaller PNG images, but they were still relatively large compared to other image formats.

For a long time though, there was no alternative to PNG images for icons: There was no lossless image compression format that could give us the same quality as PNG images and that was also widely supported.

JPEG-XL vs PNG in AppStream

Since 2021 we have JPEG-XL (JXL), which offers a true lossless mode with often better compression than PNG. The issue was that JPEG-XL wasn’t widely supported. Then, in 2025, the PDF Association selected JPEG-XL as the preferred image format for HDR images in PDFs, and now we are finally getting browser support and more ubiquitous availability of the format (you can try it right now in Firefox!).

For screenshots, using JXL’s lossy mode, it has obvious and extreme size advantages over PNG, so supporting JXL or WebP for screenshot images was an obvious choice. If JXL would support the lossless case very well as well though, we could serve many use cases with the same exported image format, which is very attractive to me.

So, the obvious next question was whether it was worth the pain of switching the icon format, so I did some measurements on real icons. For that I used the AppStream component icon pool that Debian Unstable ships, which is almost 5000 application icons of various sizes, and converted them to PNG:

Icon size	Icons	PNG total	JXL total	Pool saved	PNG avg	JXL avg	Median saved	Mean saved	Worst	Best	Larger as JXL
48×48	1544	3.7 MiB	3.0 MiB	17.8%	2.4 KiB	2.0 KiB	17.9%	16.7%	-118.7%	60.0%	206
64×64	2018	7.0 MiB	5.8 MiB	17.8%	3.6 KiB	2.9 KiB	18.0%	15.8%	-112.7%	70.0%	279
128×128	1411	11.2 MiB	8.7 MiB	22.0%	8.1 KiB	6.3 KiB	20.1%	17.5%	-89.7%	61.0%	209
TOTAL	4973	21.9 MiB	17.5 MiB	19.9%	4.5 KiB	3.6 KiB	18.6%	16.6%	-118.7%	70.0%	694

PNG images saved with libpng at effort=4, compression=9, then optimized using optipng -o2, JXL images encoded using vips jxlsave lossless=1 effort=7 strip=1 via VIPS/libjxl.

As the table shows, using lossless JXL images over size-optimized PNG images (using optipng’s default settings) provides a roughly 20% gain. This does not look like much, until you consider how often these files are downloaded: A 20% file size reduction may only save 1-2 MiB of disk space, but if they are downloaded over and over again by many clients, it will save a lot of bandwidth.

Interesting JXL encoding findings

As a sidequest, I was curious why some images were larger than their PNG counterparts when encoded with JXL, and what the ones that were significantly smaller were.

In short, the biggest size reductions for JXL existed on images that were already small as PNG, and contained large, flat color surfaces with hard edges and simple shapes. They were not very interesting, and much of JXL’s wins come from accumulating smaller gains across all files, which compound the bigger icons get (especially at 128x128px, where JXL truly shines).

The events were JXL loses to PNG are more interesting: For example, it does quite poorly with pixel-art images that have a lot of repeating patterns. Those are encoded well by PNG, but less efficiently by JXL. Take for example Vonsh:

Icon of Vonsh, an SDL-based snake game, which PNG compresses better than JXL

My guess is that while PNG can exploit the repeating pixel patterns for compression, JXL’s predicts surrounding pixels from its neighbours, which fails too often and makes it pay almost full entropy per pixel. In this single rare case, the PNG is at 5.4 KiB, while the JXL is almost 8 KiB in size.

Other cases I looked at were arguably buggy input data, where color channels were hidden under the alpha channel of the input image. PNG could probably again exploit repeats, while we were forcing JXL to encode pixels that were invisible in the final image. This is arguably a problem with the original input data. Currently, AppStream does not make any changes to icons at all, but in future we might add a filter that removes invisible colors from images to solve this pathological case (it was only two icons out of 5000 though, so it is not a high priority).

The third case I found where JXL loses to PNG were icons with checkerboard-like patterns:

Icon of x3270, an IBM 3270 Terminal Emulator

For those, PNG can likely again exploit the repeating patterns, while a checkerboard layout is pretty bad for left/top predictors like JXL’s. However, in this case the size difference (and loss for JXL) is only 450 bytes, so even though JXL loses to PNG, it does so not by much.

JXL in AppStream

Given these findings, JPEG-XL is the default image format starting with AppStream 1.2.0. AppStream Compose will encode all images losslessly as JXL, while screenshots are encoded in lossy mode at Q=90 effort=7. Since the optipng step does not happen for JXL images, this comes at no speed penalty and is even a bit faster on modern x86_64 CPUs (where libjxl can use SIMD). PNG is still available, and Compose can be told to switch between the two formats.

Upsides of JXL in AppStream right now

If you use JXL in Compose or the recent release of appstream-generator, you will get much smaller images and, for screenshots, will benefit from other JPEG-XL features such as progressive decoding, providing a far nicer user experience. libAppStream has supported JXL icons since version 1.1.3, so your clients will need that version or a newer one, and all software centers will have to support loading JXL images (which all of them do, provided the right plugins are installed).

Downsides of switching to JXL too quickly

JXL is a very new format, so web browsers might not yet display it if you are serving webpages. Your clients may also have bugs in processing JXL images, as the format is still “new”. For example, switching on JXL in Debian sent KDE Discover into an infinite loop on startup while trying to load the icons (an issue which has been fixed, but clients will need that patch first before JXL is switched on).

This currently makes JXL enablement only possible when you know that your clients can support it. This is the case for me in Debian Unstable and Debian 14, which are using JXL images for a few weeks now, but not for any older releases. Platforms like Flatpak have it even harder, because they do know even less about their clients. So, even though it has big advantages, you may want to hold off on using JXL right away, and force PNG by setting the ImageFormat key to png in appstream-generator‘s configuration, or passing --image-format=png to appstreamcli compose.

It is also worth mentioning that JPEG-XL is much, much slower on systems that do not have SIMD instructions or for which the libjxl/jxl-rs library does not have them (such as apparently riscv64 right now). If this is a concern, you might not want to switch to JXL right away.

Media pipeline improvements

Besides the JXL default change, AppStream 1.2.0 also comes with a complete overhaul of its media processing pipeline. While libappstream, AppStream’s main library, does not do any media processing and comes with very minimal dependencies to be embedded in client applications and used on servers, the same can not be said about libappstream-compose, AppStream’s library to build metadata generating applications (the server-side part, usually).

The compose library has to render fonts into font specimen cards, inspect translation files, render SVG images, decode all kinds of raster images, inspect video files, etc. Especially the fonts, and the fact that fonts can appear in SVG images, has caused issues in the past, as libappstream-compose is a heavily threaded library and most font libraries can only work from a single thread. This forced the library to essentially go into single-thread mode anytime anything that could touch a font was being processed.

AppStream also originally was created for a “safe world” where applications were vetted by the distributors before their metadata was processed. This is increasingly not the case, so it made sense to put at least a few guardrails on the most complex part of the pipeline: The media processing. As part of the change, media processing was split out into a separate worker process. This solved two problems at once: Font handling was isolated in a single-threaded binary – if we wanted to handle fonts in parallel, we could simply spawn more workers. And, being in a separate process, the media processing could now be sandboxed.

As part of the multiprocess changes, Compose also switched from using GdkPixbuf to VIPS for image processing. The latter allows for much more fine-grained control over the image output and encoding, and comes with a lot of well-maintained filters and operations, which made it possible to eliminate a fair chunk of AppStream’s hand-rolled image processing operations. As part of this transition, we unfortunately lost the ability to read XPM images, which dropped about 20-30 applications from the pool at Debian. But in the name of security, this is a sensible choice, especially since most XPM icons were very small and low-resolution, and applications using them could benefit from adding a high-quality PNG icon anyway. With VIPS, we also now restrict the amount of image formats we can load to a sensible set, so extremely niche or unexpected formats will be outright rejected (this includes sane-but-unusual formats for screenshots and icons, such as TIFF images).

The Compose library, with all of these changes, will now just request high-level operations (e.g. “render a font card for this font to a JXL image”) from the worker, and provide it with input data in sealed memfds and output locations as FDs as well. On Linux systems, the worker will use Landlock if available, to block all write access to the filesystem, deny device access and deny TCP and UDP as well. The sandbox can certainly be tightened a fair bit in future, but this was a good and safe start to gain some experience with it without having things break too easily, given the many places Compose is used in (also, Landlock’s API is surprisingly nice to use, so it was easier than I thought to add in this early version).

With all of these changes, the libappstream-compose library is now also officially marked API-stable, so you should be able to rely on it in future to build new things (its API has barely changed in the past, and now with the new media API and defaults change in place, it was time to declare it stable).

I want to see / try this!

Currently, the easiest way to have a look at the new data is to check out Debian Unstable. If you have a JXL-enabled browser, you can also see the icons in AppStream Generator’s HTML pages for Debian Sid. If you are using appstream-generator for your distribution, you will also get much more pleasant statistics and HTML pages, as well as fully deterministic media output and a whole bunch of security updates, so, update to its recent 1.0 release.

Please keep in mind that if you switch to JXL, the client tools receiving the image data have to support it. Support varies depending on the Linux distribution, so, test it first and switch the default back to PNG in case you encounter any issues.

What’s next?

With so many features and changes landed, the next changes in AppStream will focus on improving what already exists and fixing any issues (there will be more blogposts about the other features 1.2.x delivers!). Testing with the entire Debian archive as data source makes me fairly confident though that there will not be many problems. In the longer term, tightening the media processing sandbox will also be something we might want to do, e.g. by hiding parts of the filesystem tree or filtering syscalls.

For JPEG-XL, one obvious question is “Will you add support for it to the Freedesktop icon-theme specification as supported format alongside PNG, SVG(Z), and XPM?”. For on-disk icon repositories, JXL’s space-savings are less compelling, and it being HDR-capable is also not necessarily a killer feature (PNG can go a long way!). However, JPEG-XL’s ability to immediately decode larger images at reduced resolution without resampling could legitimately be very powerful here, as applications could ship a single large image and quickly decode it at 1/2, 1/4 or 1/8 the size for different purposes in their UI. JPEG-XL also supports spot-color extra channels, which applications could use as masks to recolor raster icons at render time. This could be incredibly nice to color symbolic icons on-the-fly without any SVG and CSS. JXL also provides richer metadata, which might be neat for (license/author) documentation. So, the answer here is: Maybe it makes sense to allow another format, but this will have to be discussed first, as it would force JXL into every toolkit and desktop, which is a much bigger ask than supporting it only in AppStream.

As always, let me know what you think and please report any issues or bugs directly against AppStream or AppStream Generator if you encounter problems that are with the tools, and not with a project’s metadata.

Ben Hutchings: FOSS activity in August 2026

Thu, 10 Sep 2026 13:57:48 +0000

Thereâ€™s not a whole lot to report here. During August I spent some time on holiday and also had less work time available for Debian LTS.

Debian packages:
- firmware-nonfree:
  - Bugs:
    - replied to #1143446: firmware-nonfree: split Lontium firmware into a separate binary package
  - Merge requests:
    - closed !152: lontium: create new package containing firmware for Lontium display bridge devices
    - opened and merged !153: Update to 20260810
  - Uploads:
    - uploaded version 20260810-1 to unstable
- linux:
  - Bugs:
  - Merge requests:
    - merged !2039: Add support for Alcor Micro AU6625 PCI-E Flash card reader
    - merged !2041: Update to 5.10.262 (LTS)
    - merged !2042: 6.1 stable updates (LTS)
    - merged !2048: Update to 6.1.180 (LTS)
    - merged !2050: 6.1 stable updates (LTS)
    - reviewed !2059: [arm64] Enable CONFIG_SND_SOC_ES8328_I2C as module
    - reviewed and merged !2063: 6.1 stable updates (LTS)
    - merged !2068: 6.1 stable updates (LTS)
  - Uploads:
    - uploaded version 6.1.180-1 to bookworm-security (LTS)
    - uploaded version 7.1.7-1~bpo13+1 to trixie-backports
    - uploaded version 7.1.8-1~bpo13+1 to trixie-backports
- linux-6.12 (LTS):
  - Uploads:
    - uploaded version 6.12.100-1~deb12u1 to bookworm-security
    - uploaded version 6.12.101-1~deb12u1 to bookworm-security
- linux-base:
  - Merge requests:
    - merged !22: Also copy config file to /boot
    - merged !23: New hooks ignore all calls by unpackaged kernels
Mailing lists:
- debian-kernel:
  - posted Agenda items for kernel-team meeting on 2026-08-19
  - posted Opaque tables of numbers in r8169 driver
- debian-lts-announce:
  - posted [SECURITY] [DLA 4720-1] linux security update
  - posted [SECURITY] [DLA 4724-1] linux-6.12 new package
  - posted [SECURITY] [DLA 4745-1] linux-6.12 security update
- linux-input:
  - posted [PATCH] HID: core: demote warning to debug level

Russ Allbery: podlators v6.1.1

Thu, 10 Sep 2026 02:28:00 +0000

podlators is the package containing Pod::Man, Pod::Text, and other tools for converting POD documentation into manual pages and simple text documents.

This release fixes a long-standing bug in Pod::Text and subclasses where a pathological level of indentation could cause the wrapping code to go into an infinite loop. Thanks to Jitka Plesnikova for the report. This was assigned CVE-2026-82560, although I make no guarantees that podlators is safe to run on untrusted input and therefore not fully treating this like a security issue.

While fixing that bug, I noticed a bug in Pod::Text::Overstrike's wrapping code that would leave stray formatting at the start of the next line in some situations. That is also fixed in this release.

You can get the current podlators release from CPAN or from the podlators distribution page.

Thorsten Alteholz: My Debian Activities in August 2026

Wed, 09 Sep 2026 15:49:32 +0000

Debian LTS/ELTS

This was my hundred-forty-sixth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.

Unfortunately the number of distributed working hours had been rather low this month, so the list contains much less entries than normal. During my allocated time I uploaded or worked on:

[DLA 4713-1] sslh security update to fix one CVE in Bookworm related to a so-called “link following” vulnerability.
[DLA 4714-1] libmodbus security update to fix one CVEs in Bookworm related to a stack-based Buffer Overflow vulnerability.
[DLA 4715-1] kissfft security update to fix two CVEs in Bookworm related to integer overflows.

Last but not least I spent days with FD work at the beginning of the month. The last remaining hours I continued to work on cups and hplip. Unfortunately this work did not result in an upload yet.

Debian Printing

This month I did not upload any package but just worked on some bugs.

This work is generously funded by Freexian!

Debian Lomiri

This month I worked on new Lomiri Apps. Due to a broken disk, the progress was not as expected. But stay tuned!

Not really related to Lomiri, but to Debian EDU, I fixed an incus related bug in sitesummary.

This work is generously funded by Fre(i)e Software GmbH!

Debian Astro

This month I uploaded a new upstream version or a bugfix version of:

… einsteinpy to unstable, fixing a dependency bug.
… supernovas to unstable, sponsored upload of a new upstream version.

Debian IoT

Unfortunately I had no time to work in this category this month.

Debian Mobcom

This month I uploaded a new upstream version or a bugfix version of:

… libosmocore to unstable.
… osmocom-analog to unstable.
… libosmo-netif to unstable.
… osmo-trx to unstable.
… libosmo-abis to unstable.
… libosmo-sigtran to experimental.
… osmo-mgw to unstable.
… osmo-bsc to experimental.
… osmo-sgsn to experimental.
… osmo-pcu to unstable.
… osmo-hlr to unstable.
… osmo-tetra to unstable.

misc

This month I uploaded a new upstream version or a bugfix version of:

… tango to unstable to fix bugs and do a soname transition.
… pytango to unstable to fix bugs and do a soname transition.

Enrico Zini: Financial risks in 2026

Tue, 08 Sep 2026 07:05:22 +0000

I asked the banker who is my reference at the bank something like this:

Give that we are talking about the consequences of the tantrum of a fascist foreign government, what happened to them (who are also people close and dear to me), in some future can very well happen to me.

Suddenly my risk profile shot up under the roof.

What do you suggest me to do? Should I find a trusted source of gold bullions to bury under the cellar at home?

The answer was something like this:

Sadly YES, given that the USA have a sort of financial monopoly they can entitle themselves to arbitrarily define a person/organization as a terrorist without any trial or judicial course, and as a consequence apply sanctions that cannot be effectively counteracted, not even abroad.

I didn't have this in my 2026 bingo card, but here we are.

For more details, see:

the A/I press release under "12. Banca Etica: the sanction also affected an Italian bank account"
the public statement from the bank (in Italian)

For some broader context on this kind of actions from the USA, see also:

Bits from Debian: New Debian Developers and Maintainers (July and August 2026)

Mon, 07 Sep 2026 21:00:00 +0000

The following contributor got their Debian Developer account in the last two months:

Nicolas Peugnet (nicolasp)

The following contributors were added as Debian Maintainers in the last two months:

Antoine Lassagne
Ivan Hu
Jesse Rhodes
Haolin Xue
Léo Haf
Rony João de Sousa
Luke Yasuda
Darshaka Pathirana

Congratulations!

Colin Watson: Free software activity in August 2026

Mon, 07 Sep 2026 19:25:04 +0000

My Debian contributions this month were all sponsored by Freexian.

You can also support my work directly via Liberapay or GitHub Sponsors.

Personal note

This month, my Dad unexpectedly passed away after a short illness. As a result I obviously got less work done than usual, and I still have a lot to take care of (since I’m the executor of his will, as well as helping with funeral arrangements) while grieving and generally having less focus and energy. Having routine work to do is one of the ways I cope with this sort of thing, but all the same, I hope people will bear with me and maybe remind me if I seem to be dropping the ball on something you especially need.

LLM vote

[Content note: strong opinions.]

I voted in General Resolution: LLM usage in Debian. My vote was pretty much the opposite of what ended up winning, so I’m quite disappointed. My personal opinion is that LLMs are cognitive hazards to their users that impose ecological costs far out of proportion to their utility at a time when the world absolutely cannot afford them. When the impossible economics of the large commercial models are finally allowed to catch up with reality, I expect there to be significant macroeconomic consequences, and that people who have become dependent on them will have problems; and who knows what the copyright situation on their output really is. I’m not convinced that local models are better enough on these axes to be worth the costs.

Debian’s direct contribution to all that will be negligible on a global scale, and even the most radical proposals in the GR didn’t expect that we could do much about upstreams that have gone all-in on LLMs. Even so, I’d hoped that my fellow developers might be more willing to lean on our position in the free software ecosystem to make at least a moderately radical statement. Instead, we’ve at best presented an undistinguished fence-sitting position to the world, and further entrenched the idea that humans can reliably do a good job of reviewing the output of tools that are designed to produce output plausible to humans. I certainly don’t trust my own code review skills that far.

Since I’ve never voluntarily used an LLM (not counting LLMs being foisted on me by things like search results, support chatbots, or incoming pull requests, regardless of whether I asked for them), and don’t intend to for the foreseeable future, I doubt this will change much for me in terms of the way I work. The winning option is a very weak one that imposes no new requirements on developers, which means that it also does nothing to stop me continuing to reject LLM-generated material from Debian bug reports and merge requests in my areas of responsibility. I know this probably won’t do much to satisfy people who have decided that Debian is slop now, but it’s the best I can do.

OpenSSH

I finally landed the GSS-API key exchange package split in our OpenSSH packaging. Here’s the NEWS entry:

openssh (1:10.4p1-5) unstable; urgency=medium

  The openssh-client and openssh-server packages no longer include GSS-API
  authentication and key exchange support; this adds pre-authentication
  attack surface and generally increases complexity, and should only be used
  where specifically needed.  Users who need these features should install
  openssh-client-gssapi or openssh-server-gssapi instead.

 -- Colin Watson <cjwatson@debian.org>  Sun, 23 Aug 2026 17:39:55 +0100

I fixed a flaky autopkgtest.

I upgraded from 10.4p1 to 10.5p1, which was a good test of keeping openssh and the new openssh-gssapi source package in sync.

PuTTY

I upgraded from 0.84 to 0.85.

Python packaging

New upstream versions:

anorack
djangorestframework (fixing a build failure)
isort
lazr.delegates (dropping pkg_resources)
lazr.restfulclient (dropping pkg_resources)
lazr.uri (dropping pkg_resources)
nbclient (fixing a build failure)
pydantic-settings
python-argcomplete
python-blockbuster
python-makefun (fixing a build failure)
python-persistent
python-pytest-run-parallel
python-time-machine
python-typing-inspection
wcwidth
zope.hookable
zope.i18nmessageid
zope.interface
zope.proxy
zope.security
zope.testrunner

The version treadmill continues: we’ve just finished dropping Python 3.13 as a supported version, so now we’ve started working on enabling Python 3.15 as a supported version. Maximiliano Curia has been very helpfully driving this. I didn’t get as much done here as I’d have liked (see the top of this post), but I fixed a couple of packages:

Other build/test failures:

I fixed some other bugs:

bugs.debian.org

I deployed the fix for Invalid link rel=”canonical” on bugs.debian.org. In the process I found a few bugs in recent undeployed code and fixed them.

Vincent Bernat: Sidenotes with CSS anchor positioning

Mon, 07 Sep 2026 06:49:58 +0000

I am a heavy user of sidenotes:¹ they keep optional content next to the text instead of sending the reader to the bottom of the page and back. Tufte CSS renders them without JavaScript but only accepts inline content. CSS anchor positioning, now supported by recent browsers,² is an elegant alternative. Sidenotes can hold several blocks, still without JavaScript, and fall back below the paragraph referencing them on narrow viewports and older browsers.

Inâ€¯2023, Eric Meyer demonstrated this technique in â€œNuclear Anchored Sidenotes.â€� The main improvement over other solutions is that the notes can sit anywhere in the HTML document. You can place them after the paragraph referencing them, as regular block elements for text browsers, screen readers, feed readers, and reader mode to render them properly:

Rendering in Lynx, a text browser

When the viewport is too narrow or the browser does not support CSS anchor positioning, you can style them so the reader can skip them or glance at them without losing their position in the text:

Rendering below the paragraph on a narrow viewport

Once the viewport is large enough, they appear in the margin, at the same vertical position as the matching reference mark, unless they would collide with a previous sidenote, as in the example below:³

Rendering in the margin on a large viewport

The gist of CSS anchoring is to position an element relative to another elementâ€”the anchor. For the sidenotes, the anchor is the reference mark. I use the following markup, with a data attribute to specify the anchor name:

<sup id="fnref:YYY" data-anchor="--lf-sn-YYY">
  <a href="#sidenote-YYY">1</a>
</sup>

The matching note is an <aside> element carrying the same data attribute for the anchor name. We put it after the paragraph holding the reference mark:

<aside role="note" id="sidenote-YYY" data-anchor="--lf-sn-YYY">
  <sup>1</sup>
  <p>A first paragraph.</p>
  <p>A second paragraph.</p>
</aside>

On a narrow viewport or when the browser is too old for CSS anchoring, we style the sidenote, which stays below its paragraph, with a muted color:

aside[role="note"] {
  margin-block: 1rlh;
  color: #444;
}

On a wide viewport and when the browser is recent enough, we move the sidenote to the right margin:

@supports (anchor-name: attr(data-anchor type(<custom-ident>))) {
  @media (min-width: 72rem) {
    main {
      position: relative;
      sup[data-anchor] {
        anchor-name: attr(data-anchor type(<custom-ident>));
        /* â†’ anchor-name: --lf-sn-YYY */
      }
      aside[role="note"][data-anchor] {
        anchor-name: --lf-sidenote;
        position: absolute;
        position-anchor: attr(data-anchor type(<custom-ident>));
        /* â†’ position-anchor: --lf-sn-YYY */
        top: max(anchor(top), anchor(--lf-sidenote bottom, -1rlh) + 1rlh);
        left: 100%;
        margin: 0 2rem;
        width: 18rem;
        color: inherit;
      }
    }
  }
}

attr() extracts the anchor name for the reference mark from the data-anchor attribute. It returns a string, unless we specify a CSS unit or a type, like here: the browser parses the data attribute as a custom identifier, which anchor-name validates as a dashed identifier, a custom identifier starting with two dashes.⁴

The note itself is absolutely positioned past the right edge of the main block. It selects the matching reference mark as its anchor with position-anchor set to the value of the data-anchor attribute. Each note is also an anchor named --lf-sidenote. We use it to keep the next note from colliding with this one.

The anchor() CSS function lets us position the noteâ€™s top edge relative to its anchor: anchor(top) aligns the top edge of the note with the top edge of the reference mark. It can also take another anchor as a parameter: anchor(--lf-sidenote bottom) would align the top edge of the note with the bottom edge of the closest preceding anchor named --lf-sidenoteâ€”so the previous note.⁵ Like attr(), anchor() accepts a fallback value as its second parameter and use it when the named anchor does not exist.

The top property handles three cases, illustrated in the following diagram:

The three cases for the vertical position of a note

The first noteâ€™s top edge aligns with the top edge of its reference mark: as there is no previous note, anchor(--lf-sidenote bottom, -1rlh) + 1rlh resolves to 0 and max() returns anchor(top).
When the reference mark of a later note sits above the bottom of the previous note, plus some vertical space, the note goes below the previous one to avoid a collision. max() returns anchor(--lf-sidenote bottom) + 1rlh.
Otherwise, the noteâ€™s top edge aligns with the reference markâ€™s top edge, as max() returns anchor(top).

Have a look at the complete stylesheet, which also adapts the reference mark to the location of the note: a â€œâ†“â€� arrow when the note sits below the paragraph, a â€œâ†’â€� arrow when it moves to the margin. Gwernâ€™s â€œSidenotes In Web Designâ€� lists more implementations and their trade-offs.

Some bloggers aim to write a post in 30 minutes. I planned to publish three web-related articles this weekend. Instead, I spent an inordinate amount of time elsewhere: about 15 commits on the build system, a pull request to update CSS highlighting for nested selectors in Pygments, and a small correction to MDNâ€™s article on the anchor() CSS function. The SVG illustration took a bit less than an hour and the article itself a handful of hours. The attr() function came in after I thought â€œinline style looks ugly, isnâ€™t there a better way?â€� But, hey, I still think this is worth it! ğŸ�¨

My PhD advisor told me this is unwise.Â â†©
The first bits of anchor positioning are supported from Chromeâ€¯125 (Mayâ€¯2024), Firefoxâ€¯147 (Januaryâ€¯2026), and Safariâ€¯26 (Septemberâ€¯2025).

Before Safariâ€¯26.5, sidenotes may collide due to a bug in how dependency chains are handled. You can detect this situation with some JavaScript. It is, however, not needed in the solution described here as we depend on a more recent feature.Â â†©
If you noticed the runt in the first note, I share your pain and lament that Firefox does not implement text-wrap: pretty.Â â†©
Typed attr() is supported from Chromeâ€¯133 (Februaryâ€¯2025), Firefoxâ€¯155 (Septemberâ€¯2026), and Safariâ€¯27 (Septemberâ€¯2026). Check Una Kravetsâ€™ article for details. To support more browsers, you can inline the anchor name and the position anchor directly in the HTML:
```
<sup id="â€¦" style="anchor-name: --lf-sn-â€¦">
  <a href="#sidenote-â€¦">1</a>
</sup>
```
â€œManaging Anchor Associations With Data Attributes and Advanced attr(),â€� by Daniel Schwarz, explores CSS anchors and typed attr() in more detail.Â â†©
The exact rule for the target anchor element is more complex: â€œif an ancestor of [the note] satisfies the following conditions, return the nearest such element to [the note]. Otherwise, return the last element in tree order that satisfies the conditions.â€� One of these conditions is that â€œ[the candidate] is an acceptable anchor element for [the note],â€� which requires that â€œ[the candidate] is laid out strictly before [the note],â€� where the relevant clause is that â€œ[the candidate] is either not absolutely positioned or occurs earlier in the flat tree order than [the note].â€�Â â†©

Iustin Pop: AI agents aha moment

Sun, 06 Sep 2026 13:29:50 +0000

Looking at the reactions to the Debian AI vote, I think some people still think the clock can be turned back, as if that ever worked in history. Rather than cry about spilled milk, I prefer to find a path forward in the new world. There are many ways to use LLMs, some of them are straightforward, others not so much.

One of the “not so clear” areas for me is the focus on agentic workloads. For complex tasks, sure, you want something that can work in the background, but in general, why does every single tool go the agentic way? I much prefer the “chat/ask” approach, or even the “code” one, but if I’m at the keyboard, why would I send a task to an agent, and see it work, instead of directly implementing it?

And then, this past Friday, I finally understood one part of that. I was in the airport, sitting at the gate and waiting to board a flight, and because I arrived much earlier at the airport (fearing crowds due to Labour Day weekend), I got one hour of work before boarding started. As the time for boarding approached, I did one more commit after making sure tests pass, pushed, closed laptop, and went to walk a bit before getting on the plane.

As I was getting up, I get a phone notification from GitHub that the CI run failed. I was quite surprised, as the local tests passed, so I open the notification, and realize that tests via make test vs CI (which additionally uses --pedantic) had slightly different settings, and of course I missed a build warning (which in CI is an error).

I thought I’d fix that on the plane, but then I saw a “Copilot agent” button in the mobile app. I was curious what it did, I click it, and I see Copilot starting a draft pull request, and saying:

Thanks for asking me to work on this. I will get started on it and keep this PR’s description up to date as I form a plan and make progress.

Fix the failing GitHub Actions job. Analyze the Actions logs, identify the root cause of the failure, and implement a fix.

Then it goes, finds the failure, writes the fix, and tries to run the tests. Well, it can’t do it (it runs in a restricted container, so no network, so stack install couldn’t actually work). The agent sees that, acknowledges it has no way to validate the fix, but the error message was clear enough that it was confident the fix is mostly correct, so it sends the pull request.

I allow full CI to run on the pull request, and go buy a bottle of water. After that, I check and see that the CI failed again, as not one but two test files were broken, and I didn’t have --keep-going, so the build stopped at the first failure. I write a comment in the pull request, no reaction, I realize I need to tag Copilot explicitly, I do that, and it starts another investigation.

I’m waiting now in the boarding queue, with phone in hand, while Copilot is fixing my bug. While I scan my boarding pass and walk towards the plane, the pull request is updated, I trigger another CI, it passes, and I merge it.

And then, it hit me. Agents allow me to make progress while being “not at keyboard”, whether that’s physically “not at keyboard”, or while working on something else. Fixing a simple test failure is not something that needs human attention per se, whereas improving the test layout might be.

In that airport, using otherwise-unusable downtime, and without explicitly intending to, I made progress in understanding a different way to use AI. Now I have three ways to work with LLMs: ask (tutor mode), code (implement my request), and agent (fix simple or complex problems, autonomously). I still don’t know about “plan” mode and really complex tasks, like asking it to implement features from scratch. That will probably be the next area to tackle.

And today (Sunday), while waiting for a running race to start, I opened GitHub, and asked Copilot to increase test coverage for a simple module. It did, and yes it still can’t run tests (I learned in the meantime that you can configure the environment in which the agent runs, nice), but after two back-and-forth messages, I have a pull request ready to review. All in the 20 minutes before a race, where I could either browse social media or actually do some meaningful work.

Checking now my GitHub billing, it looks like all of this Copilot use only cost $1.92. Yes, that is under two dollars! And while it did use compute resources, the person across the aisle who watched TikTok or Instagram for half an hour while waiting for takeoff also consumed a lot of compute, and so do the gazillion cat videos uploaded to YouTube every day.

To me, this is another tool in the toolbox, that might one day replace me (as it did to the 19th-century textile workers), or make me five times more productive — we’ll see where we end up. In the meantime, I can move faster, and make better use of my limited free time.

Enjoy the ride!

Russell Coker: CoMaps

Sun, 06 Sep 2026 11:05:50 +0000

I have just tried CoMaps, a free mapping program released under the Apache license [1]. I have tried it on Android on a Pixel 6a but it also runs on Linux so I’ll try it on a PinePhone or similar at some convenient time. On Android it is in the F-Droid repository among others and for Linux there’s a Flatpak package.

The data it uses is from Open Street Map project [2] which has extensive and accurate coverage of every place I’ve looked at (Australia and a few other first-world countries). The first thing it does after being installed is start downloading the world data set from Open Street Map and prompt to download the data for the detected region (Melbourne in my case).

The UI is decent and allows most of the features that I am used to using in Google Maps. The quality of directions seems good, I’ve only tested it with one journey so far which was a 50 minute drive across the city and it gave a set of directions that Google Maps often gives.

It gives spoken directions which is an important feature but sometimes the way the directions are presented is confusing. When turning off a freeway it didn’t give a spoken direction to do that, it gave a direction to “turn right” which was AFTER leaving the freeway, fortunately the map was clearly displayed.

In terms of use practices of this program the main difference I recommend is checking which off ramp to use from a freeway before entering the freeway. With Google Maps you can rely on it giving clear directions in that case.

I recommend this program without reservation. It can do everything that Google Maps does apart from detecting traffic jams because there’s no way of detecting traffic without spying on users. It is designed to preserve user privacy and works well in that regard.

[1] https://www.comaps.app/
[2] https://www.openstreetmap.org/

Enrico Zini: Migrating away from .org/.net/.com domains

Sun, 06 Sep 2026 09:40:04 +0000

After having witnessed how easy it is for good people to lose a .org domain over a fascist tantrum (you can follow the Autistici/Inventati story here and here), I've started moving all my infrastructure to differently managed TLDs.

enricozini.org and enricozini.com will keep being functional for the time being, as dropping a domain makes it available for squatting and impersonation.

These new domains are now online, with working web and emails:

enricozini.eu
ezini.eu
ezini.it (probably equally subject to fascist tantrums these days)

It will take ages to migrate countless accounts that are tied to my primary email address, so better start early.

Waiting to see what will happen with .meow domains, which I supported despite not identifying as a cat.

Steinar H. Gunderson: plocate 1.1.25 released

Sun, 06 Sep 2026 08:02:00 +0000

I've released version 1.1.25 of plocate. This time around, there's two security issues of unknown severity; if you chain them with other bugs, they could lead to being able to list files (but of course not their contents) that you should not normally be able to see. So an update is probably in order; you can never be too safe these days.

The full changelog is:

plocate 1.1.25, September 6th, 2026

  - Fix two early-exit bugs with multiple databases.
    Reported by Manpreet Singh and Tyler Spivey.

  - Drop setgid properly, including the saved gid.
    Reported by Michal Sekletar, found with the help of Claude Opus 4.6.

  - Fix a potential symlink-checking race in updatedb.
    Reported by Michal Sekletar, found with the help of Claude Opus 4.6.

As usual, you can get it from the home page, or it's on the way up in Debian unstable.

Michael Stapelberg: Debian Code Search: Fast TurboPFor with Go SIMD

Sun, 06 Sep 2026 07:00:00 +0000

This August, I accomplished what I wanted for many years: I deleted the last cgo dependency in Debian Code Search! This was made possible by Goâ€™s recently introduced SIMD support, because now we can implement the TurboPFor integer compression format as efficiently â€” more efficiently, in fact, by using the newer AVX512 instruction set! â€” as the reference implementation.

Background: Why does DCS need a fast Integer Codec?

Debian Code Search (DCS) is a search engine that allows searching all the Open Source source code within Debian, with either literal search expressions or regular expression search queries.

A search engine uses an inverted index: a map from term to documents containing the term. Each document is typically represented most efficiently by using an id, so the index consists of many lists of document ids.

When searching, it is important to quickly decode these lists to answer the search query. However, there is a point of diminishing returns where the decoding speed, even though it can still be measurably improved quite a bit, no longer influences the overall query duration.

From 2012 (its inception) to 2019, Debian Code Search used to use a small index format, and queries were fast because the index was kept entirely in RAM. In 2019, I implemented the new index format, which adds an on-disk positional index. For literal queries (78.2% of DCS queries), querying the positional index on disk is faster than querying the non-positional index in RAM.

The efficient encoding of the TurboPFor format makes it possible to fit such an index on a mid-sized Hetzner server, which I rent with two 1 TB SSD disks. The optimized decoder of the C TurboPFor library is what made decoding fast at query time.

If you want to dive deeper into the algorithm, see this blog post from February 2019:

TurboPFor: an analysis

Motivation I have recently been looking into speeding up Debian Code Search. As a quick reminder, search engines answer queries by consulting an inverted index: a map from term to documents containing that term (called a â€œposting listâ€�). See the Debian Code Search Bachelor Thesis (PDF) for a lot more details. Read more â†’

If you want to learn more about the positional index, see this blog post from September 2019:

Debian Code Search: positional index, TurboPFor-compressed

Over the last few months, I have been developing a new index format for Debian Code Search. This required a lot of careful refactoring, re-implementation, debug tool creation and debugging. Read more â†’

SIMD in Go

For many years, you had the following options for using SIMD instructions in Go:

Hand-writing Go assembler code. This is only doable for small functions, for example bytes.IndexByte is implemented with hand-written Go assembly (including AVX2).
Generating Go assembler code with tools like Michael McLoughlinâ€™s â€œAvoâ€�. This is how crypto/internal/fips140/sha256 uses AVX2. While Avo generator code definitely is higher-level than hand-written assembly, it is still too close to assembly for my taste.
Use a C library via cgo so gcc or clang compiles SIMD code. Debian Code Search used to use the powturbo/TurboPFor C library via cgo for the last 7 years.

The C TurboPFor library has served us well, but Debian Code Search was always intended to be a project using Go, so I would prefer it if I did not have any C code in the project.

Go 1.26 (released in February 2026) introduced the simd/archsimd package:

Go 1.26 introduces a new experimental simd/archsimd package, which can be enabled by setting the environment variable GOEXPERIMENT=simd at build time. This package provides access to architecture-specific SIMD operations. It is currently available on the amd64 architecture and supports 128-bit, 256-bit, and 512-bit vector types, such as Int8x16 and Float64x8, with operations such as Int8x16.Add. The API is not yet considered stable.

â€” Go 1.26 Release Notes

For my 2019 TurboPFor analysis, I implemented goturbopfor, a native Go teaching decoder (without any SIMD), because I find Go code easier to follow than C code, especially optimized C code. My implementation was intentionally not optimized so that the code was easier to study.

The TurboPFor format/algorithm has a vector-optimized part: bitpacking comes in a scalar variant (bitunpack32) and a vector variant (bitunpack256v32), where the vector variant is used for full blocks (256 values) and the scalar variant is used for remainder blocks (< 256 values).

When Go 1.26 was released, I used Claude Code to explore whether my native Go decoderâ€™s bitunpack256v32 function (for the vertical vector layout) could be implemented using Go SIMD, and the answer was yes, it was possible and it was faster than without SIMD, but not quite at the level of C TurboPFor. If you let Claude Code try for long enough, it eventually finds enough optimizations (about 10) to match C performance.

I donâ€™t want to vibe-code Debian Code Search, though, so I figured I would find some time to review the SIMD code at some point and see if I could implement something similar myself.

Before I found enough time and motivation to complete said review, I discovered that to not regress real-life query performance by more than 10 to 100 milliseconds (which seems acceptable), I donâ€™t actually need to add SIMD code to my teaching decoder at all; it would be sufficient to reduce allocations in my teaching decoder and specialize it per bit width.

Encouraged by the possibility of using the optimized native Go decoder in Debian Code Search, I explored whether I could also implement a native Go encoder so that I could get rid of the C TurboPFor dependency entirely. The answer is yes, it is doable in a few days, and it isnâ€™t even that much slower: Go is at 76% of C, see Debian/dcs commit e920dc7.

The goal I set myself at that point was to see if I could learn enough SIMD to optimize the native Go encoder such that its performance would match how DCS uses C TurboPFor (via cgo).

Beating C TurboPFor was possible in 2-3 commits (SIMD and bit width specialization). To my surprise, Claude Fable 5 pointed out that the encoderâ€™s block scanning could be done more efficiently using a technique called positional popcount, and that is another 2x speed-up! ğŸ˜²

To be clear: I am not saying the Go compiler beats C here. Certainly, the C compiler can also produce fast AVX512 code and can be used to implement positional popcount. When comparing apples to apples, i.e. backporting the AVX512 kernels and positional popcount technique to C TurboPFor, Go benchmarks a little slower at â‰ˆ1.4x C.

This spectacular result (much faster than what DCS had before) got me curious how far I could push the decoder with SIMD after all. I ended up matching/exceeding the cgo version here, too!

The rest of this article explains a few classes of optimizations I encountered along the way.

Starting Point

When I wrote my goturbopfor teaching decoder, I named its functions to match the upstream C TurboPFor library, but now I want to get away from names like p4ndec256v32 â€” they make sense from the TurboPFor perspective, but for Debian Code Search, we can use cleaner names.

Before writing any code, I audited how DCS uses integer compression / decompression.

API design: BlockEncoder, BlockDecoder and streaming

In Debian Code Search, we have the following usage patterns:

Partial Indexing: When a new package (or package version) enters Debian, all of its (text) files are indexed. If the hello-2.12.3-1 package (hypothetically) contained only hello.c with printf("hello!\n");, we would assign document ID 1 to hello.c and store in the partial index that trigrams pri, rin, int, ntf, etc. are all found in doc 1 (hello.c).
Full Index Merging: The many thousands of partial index files (for each Debian package) are combined into a small handful of large index files: When searching, it would be expensive to consult thousands of indexes. To merge multiple partial index files into one larger index (which can then be efficiently queried), we need to re-encode the partial index files: what used to be document ID 1 in the partial index might be document ID 2531 in the full index.
Querying (searching): When users enter search queries, these queries need to be answered as quickly as possible. The relevant entries in the full indexes are decoded (in parallel).

For reading the index, we do keep the decoded uint32s fully in memory, so we only need DecodeN(input []byte, output []uint32) (read int), a function that reads len(output) values (uint32) from input and returns how many bytes it consumed.

For writing the index (both in partial indexing, and when merging), keeping the entire index in memory is prohibitively expensive, so we need a streaming API, for decoding and for encoding.

Ultimately, I converged on the following API:

package pforenc

type BlockEncoder struct {
    // scratch buffers can go here
}

// EncodeBlock encodes len(vals)<=256 uint32s into dest (one TurboPFor block).
func (*BlockEncoder) EncodeBlock(dest []byte, vals []uint32) []byte {}

// EncodeN calls EncodeBlock in a loop.
func (*BlockEncoder) EncodeN(dest []byte, vals []uint32) []byte {}

type StreamEncoder struct {
  be   BlockEncoder
  vals [256]uint32
  // scratch buffers
}

// if full, you need to call [EncodeBlock]
func (*StreamEncoder) Add(val uint32) (full bool)

// EncodeBlock must be called after all data was [Add]ed.
//
// Write the returned buffer to file or send it over the network;
// it is only valid until the next [EncodeBlock] call.
func (*StreamEncoder) EncodeBlock() []byte {
  if se.n == 0 { return nil } // turn an extra EncodeBlock into a no-op
  // â€¦
}

This API (the decoder works similarly) allows us to process data in TurboPFor format without any memory allocations. The types are not safe for concurrent use by multiple goroutines. The zero value is ready to be used. For the streaming API, the result only stays valid until the next call.

Initial Implementation

Before we can optimize anything, we need a working decoder and encoder. The decoder already exists: my goturbopfor teaching decoder. Next up, I needed an encoder.

Writing a TurboPFor encoder has a delightfully simple starting point: You can encode all values at bit width 32, in little endian, at which point you only need to add a one-byte TurboPFor block header every 256 values and youâ€™re done:

func (be *BlockEncoder) EncodeN(dest []byte, vals []uint32) []byte {
  for len(vals) > 0 {
    chunk := min(len(vals), 256)
    dest = be.EncodeBlock(dest, vals[:chunk])
    vals = vals[chunk:]
  }
  return dest
}

func (be *BlockEncoder) EncodeBlock(dest []byte, vals []uint32) []byte {
  const bitWidth = 32
  dest = append(dest, bitWidth)
  for _, val := range vals {
    dest = binary.LittleEndian.AppendUint32(dest, val)
  }
  return dest
}

Of course, this is a terribly inefficient compressor, so after the first commit, the real work starts: implement each block type until the compression matches the original C TurboPFor implementation (same output file size), or in other words: do the reverse of the decoder.

The TurboPFor bitpacking block type (bitpacking implementation commit) encodes a bit stream of variable bit width (where the bit width is in range 0 â‰¤ bitWidth â‰¤ 32) in little endian byte order. By scanning all values and choosing the smallest bit width that allows representing all values, this technique saves disk space (compresses).
The bitpacking with exceptions block type (bitpacking with exceptions implementation commit) determines two bit widths: one for values, the other bit width for encoding exceptions. This allows choosing a lower bit width (that does not cover all values) compared to the bitpacking block type. A bitmap encodes whether a value has an exception or not.
The bitpacking with VB exceptions block type (bitpacking with VB exceptions implementation commit) is a variant which does not use an exception bitmap and encodes exceptions using a variable byte integer encoding. This is more efficient when there are few exceptions (less than 20) or the exceptions are very different in bit width compared to the other values.
Lastly, the constant block type (constant implementation commit) stores just one value on disk. This is useful for all-zero or all-one blocks, for example.

I found it interesting to realize that the main work of the encoder is to scan the input values and choose the optimal block type, whereas the actual encoding itself is cheap in comparison.

At this point, we can look at performance and see that the Go encoder is at 76% of the C encoder.

In all honesty, I could have probably stopped here, but now that the milestone of a viable replacement was reached, I got curious to see how far it would be possible to push the encoder (how much work to reach C speeds?) and afterwards, the decoder, too.

Setup

The microarchitecture level: set `GOAMD64`

The microarchitecture of a CPU determines which instructions it provides, and that includes not just SIMD instruction sets (like AVX2), but also other useful instructions like LZCNT (Leading Zero Count), which can be used to implement math/bits.Len32 more efficiently, which the TurboPFor encoder needs to call on every input value to determine the ideal bit width.

Letâ€™s walk through how to set the microarchitecture level when using Go on 64-bit x86 (x86-64).

Go uses the GOARCH environment variable to configure the target compilation architecture, and I am using the value amd64 to select 64-bit x86 (AVX2 and AVX512 are instruction sets found on x86-64 CPUs). With GOARCH=amd64, the architecture-specific variable GOAMD64 configures the microarchitecture level for which to compile and Go 1.18 introduced these 4 different levels:

GOAMD64=v1 (default): The baseline.
Exclusively generates instructions that all 64-bit x86 processors can execute.

GOAMD64=v2: all v1 instructions,
plus CMPXCHG16B, LAHF, SAHF, POPCNT, SSE3, SSE4.1, SSE4.2, SSSE3.

GOAMD64=v3: all v2 instructions,
plus AVX, AVX2, BMI1, BMI2, F16C, FMA, LZCNT, MOVBE, OSXSAVE.

GOAMD64=v4: all v3 instructions,
plus AVX512F, AVX512BW, AVX512CD, AVX512DQ, AVX512VL.

In 2026, I generally recommend compiling with GOAMD64=v3 so that functions like bits.OnesCount8 are compiled into intrinsics (POPCNT) instead of using a lookup table.

For Intel CPUs, setting GOAMD64=v3 means your programs will only start on Haswell CPUs (2013) or newer; for AMD CPUs that means Zen 1 (2017) or newer.

In this specific case (DCS), I am even compiling with GOAMD64=v4. The v4 microarchitecture level requires AVX512, which means AMD Zen 4, Zen 5 or newer (Intelâ€™s story isâ€¦ complicated). Luckily, both my main development PC (Zen 5) and the Debian Code Search server (Zen 4) are recent enough. Setting GOAMD64=v4 has little effect on Go 1.27 itself: the only change is that maps use one less instruction (VPBROADCASTB instead of PSHUFB). But compiling with GOAMD64=v4 allows us to move one more feature check from runtime to compile time, see SIMD build tags.

It makes sense to set the microarchitecture level in your benchmark setup so that you donâ€™t measure the slow fallback implementations. I use export GOAMD64=v4 in my Makefile.

Benchmarking setup

Goâ€™s built-in testing package contains support for benchmarks which are written in functions of the form func BenchmarkXxx(b *testing.B). The simplest way to run such benchmarks is go test -bench=., but I ended up configuring a few convenience make targets, which write results to bench.txt and compare against baseline.txt (the previous commitâ€™s results, usually), using the very useful benchstat tool.

GOTEST=go test

# -count=6 gives pâ‰¤0.002 in benchstat:
# https://pkg.go.dev/golang.org/x/perf/cmd/benchstat
BENCHFLAGS=-run=^$$ -bench=. -benchtime=200000x -count=6

# use taskset -c1 to always pin to the same single core,
# avoiding accidental scheduling on different cores on
# mixed-core CPUs like the Ryzen 9 9950X3D.
TASKSET=taskset -c 1
BENCH=$(TASKSET) $(GOTEST) $(BENCHFLAGS)

.PHONY: all test bench bench-baseline bench-relative

all: test

bench: test
	$(BENCH) | tee bench.txt
# Compares compression ratio between C and Go implementation
	benchstat -col /impl -row '/n /vals' -filter '-/impl:go-stream .unit:(encoded-bytes)' bench.txt
# Compares performance between C (cgo) and Go implementation
	benchstat -col /impl -row '/n /vals' -filter '.unit:(Mval/s)' bench.txt

bench-baseline: test
	$(BENCH) | tee baseline.txt

bench-relative: test
	$(BENCH) | tee bench.txt
	benchstat -filter '-/impl:go-stream .unit:(encoded-bytes)' baseline.txt bench.txt
	benchstat -filter '/impl:go .unit:(Mval/s)' baseline.txt bench.txt

The encoded-bytes and Mval/s units are custom metrics I am reporting from the various sub-benchmarks, which are arranged such that I can filter / report them with benchstat.

The main encoder (and decoder) benchmarks compare 3 different implementations (cgo, Go, Go with the StreamEncoder API) with a number of benchmark cases that are designed to cover the different block types and contain a similar mix of values as what we see in Debian Code Search:

// reportMetrics adds Mval/s and encoded-bytes metrics to all benchmarks.
func reportMetrics(b *testing.B, n int, nencoded int) {
   b.ReportMetric(float64(nencoded), "encoded-bytes")
   b.ReportMetric(float64(b.N*n)/1e6/b.Elapsed().Seconds(), "Mval/s")
}

// BenchmarkEncode/n=<N>/vals=<testcase>/impl=<c|go|go-stream>
//
// e.g. BenchmarkEncode/n=2048/vals=one-constant/impl=go-stream
func BenchmarkEncode(b *testing.B) {
   for _, tc := range allBenchCases() {
     n := len(tc.vals)
     b.Run(fmt.Sprintf("n=%d/vals=%s", n, tc.name), func(b *testing.B) {
       b.Run("impl=c", func(b *testing.B) {
         b.ReportAllocs()
         var encoded []byte
         buf := make([]byte, turbopfor.EncodingSize(n))
         for b.Loop() {
           encoded = turbopfor.P4nenc256v32Buf(buf, tc.vals)
         }
         reportMetrics(b, n, len(encoded))
       })
       b.Run("impl=go", func(b *testing.B) {
         b.ReportAllocs()
         var be BlockEncoder
         var encoded []byte
         buf := make([]byte, 0, turbopfor.EncodingSize(n))
         for b.Loop() {
           encoded = be.EncodeN(buf, tc.vals)
         }
         reportMetrics(b, n, len(encoded))
       })
       b.Run("impl=go-stream", func(b *testing.B) {
         b.ReportAllocs()
         var se StreamEncoder
         var encoded int
         for b.Loop() {
           encoded = 0
           for _, val := range tc.vals {
             if se.Add(val) {
               encoded += len(se.EncodeBlock())
             }
           }
           encoded += len(se.EncodeBlock())
         }
         reportMetrics(b, n, encoded)
       })
     })
   }
}

CPU counters: perf

Go has included excellent performance tooling for many years, see the â€œProfiling Go Programsâ€� blog post (2011) for an example of how to use pprof, a sampling profiler. This profiler can help track down which part of a program runs slow, or where memory allocations happen.

Once you identified the slow part of a program, how do you know why itâ€™s slow?

To learn more about the specific bottlenecks your program encounters, you can consult your CPUâ€™s hardware performance counters. For example, you could check the branch predictor counters to see if your program is slow due to a high number of branch mispredicts.

On Linux, the perf tool is the best way to access the CPU hardware performance counters. A good starting point for working with perf is the documentation on â€œTop-down analysis with the perf toolâ€�, which describes the optimization method that Intel established.

In my Makefile, I set up two perf targets:

# GOTEST and TASKSET like shown in the earlier benchmarking setup section:
GOTEST=go test -pgo=encode.cpuprof
TASKSET=taskset -c 1
PERFBENCHFLAGS=-test.bench='Encode/n=2048/vals=debian-mix/impl=go$$' -test.benchtime=200000x

# Use perf(1) to capture AMD IBS (the equivalent to Intel PEBS)
# PipelineL1 is roughly equivalent to Intel TopdownL1
perf:
	$(GOTEST) -c
	$(TASKSET) perf stat -M PipelineL1 ./pforenc.test -test.run=^$$ $(PERFBENCHFLAGS)
	sudo perf record -F 4999 -e ibs_op// --call-graph fp ./pforenc.test -test.run=^$$ $(PERFBENCHFLAGS)
	sudo chmod 644 perf.data

# 488281 iterations Ã— 2048 values = 1.000e9 values, so counter/1e9 = per value.
perf-per-value:
	$(GOTEST) -c
	$(TASKSET) perf stat -x, -e cycles:u,instructions:u,branches:u,branch-misses:u ./pforenc.test -test.run=^$$ -test.bench='Encode/n=2048/vals=debian-mix/impl=go$$' -test.benchtime=488281x 2>&1 >/dev/null | awk -F, '{printf "%-16s %6.2f /val\n", $$3, $$1/1e9}'

The perf-per-value numbers are high level numbers that indicate how much work the implementation is doing. Reducing the number usually increases speed.

To see the counters for each instruction (and source code lines), I use make perf, followed by perf report. A quick shortcut is perf annotate, which directly shows the hottest function.

Optimizations (scalar)

Letâ€™s first see how far we can get without reaching for SIMD instructions.

(The examples are not necessarily in commit order, but cherry-picked for clarity.)

Profile-Guided Optimization (PGO)

PGO stands for Profile-Guided Optimization and is a feature that Go introduced as a preview in Go 1.20 (released in February 2023) and shipped as ready for general production use in Go 1.21 (released in August 2023).

The idea is to capture a CPU profile that records where your program spends most of its CPU time, which you then provide to the Go compiler to give it more data to make better decisions.

Most importantly, this way the Go compiler can inline functions much more aggressively than its usual heuristics allow, which does have a measurably positive effect in my series of optimization commits. Another optimization that a PGO profile allows the compiler to do is conditional devirtualization â€” but our TurboPFor code does not use any interfaces.

My strategy is to enable PGO before doing any other optimizations, so that we have the full inlining budget available that PGO gives us, and can measure the effect of other commits clearly.

Surprisingly, turning on PGO actually decreases our performance (-13% geomean), but a closer investigation reveals that we just got unlucky. Let me explain.

Aside from inlining and conditional devirtualization, PGO also influences alignment: The Go compiler sets PCALIGNMAX(64, 31) on the first block of a loop (the â€œloop bodyâ€�) for all loops in hot functions (per the PGO profile), i.e. Go will insert up to 31 bytes of padding to make the block land on a 64-byte boundary. Documentation like AMDâ€™s â€œSoftware Optimization Guide for the AMD Zen5 Microarchitectureâ€� (2024, #58455) explicitly recommends aligning hot loops that way:

[â€¦] for hot loops, some further knowledge of trade-offs can be helpful. Because the processor can read an aligned 64-byte fetch block every cycle, it is suggested to either align the start of the loop to the beginning of a 64-byte cache line [â€¦]

Indeed, when compiling with -gcflags=all=-d=alignhot=0 to disable the alignment, performance remains as good as without PGO. How can the padding hurt more than help? The answer is: Itâ€™s not the padding itself! Itâ€™s a side-effect of the padding moving instructions to different addresses.

In the unlucky arrangement, a macro-fused CMPQ+JGE instruction pair now ends up exactly on a 32-byte boundary. However, the Go compiler ensures fused branch sequences must never cross or end at a 32-byte boundary to fix Intel erratum SKX102 (discussion: Go issue #35881) by inserting NOPs.

This NOP padding, unlike the loop alignment padding, is not free; these extra instructions slow down our otherwise dispatch-bound loops.

Because the commits after the PGO enabling commit change the code, this unlucky situation is avoided for the rest of the optimization series (by chance).

Reducing memory allocations

Memory allocations are quite expensive, at least in comparison to encoding/decoding integers, so I followed my usual strategy of first reducing memory allocations as much as possible.

In my goturbopfor teaching decoder, whenever the code needed a scratch buffer, it would allocate it right then and there with make():

// p4dec32 decodes one block of TurboPFor-encoded 32 bit ints
func (d *decoder) p4dec32(input []byte, output []uint32) (read int) {
    // â€¦
  switch blockType {
  case blockBitpackingExceptions:
    bx, input := input[0], input[1:]
    n := len(output)

    exmap := input
    nex := 0 // number of exceptions
    for i := 0; i < n; i++ {
      if exmap[i/8]&(1<<uint(i%8)) != 0 {
        nex++
      }
    }
    input = input[(n+7)/8:]

    exceptions := make([]uint32, nex)
    input = input[bitunpack32(input, exceptions, bx):]
    input = input[d.bitunpack(input, output, b):]

    for i := 0; i < n; i++ {
      if exmap[i/8]&(1<<uint(i%8)) != 0 {
        output[i] += exceptions[0] << b
        exceptions = exceptions[1:]
      }
    }

    return before - len(input)
  }
}

The Go compiler can turn make(T, n) calls into stack allocations, if n is known at compile-time. But, in this case nex is not known at compile-time. We can verify that Go calls into the runtime (runtime.makeslice) by dumping the object code (assembly) with source annotated (-S):

% cd ~/go/src/github.com/stapelberg/goturbopfor
% git reset --hard 49b7c05cc61e77f0257568eb73833467714d2b4a
% go test -c  # go1.27.0
% go tool objdump -S goturbopfor.test | perl -nlE 'say if /p4dec32/ .. /^$/'
TEXT github.com/stapelberg/goturbopfor.(*decoder).p4dec32(SB) /home/michael/go/src/github.com/stapelberg/goturbopfor/goturbopfor.go
func (d *decoder) p4dec32(input []byte, output []uint32) (read int) {
  0x549f60		4c8da42460ffffff	LEAQ 0xffffff60(SP), R12
  0x549f68		4d3b6610		CMPQ R12, 0x10(R14)
  0x549f6c		0f86d9070000		JBE 0x54a74b
  0x549f72		55			PUSHQ BP
  0x549f73		4889e5			MOVQ SP, BP
  0x549f76		4881ec18010000		SUBQ $0x118, SP
  0x549f7d		48899c2430010000	MOVQ BX, 0x130(SP)
  0x549f85		4889b42448010000	MOVQ SI, 0x148(SP)
	if len(output) == 0 {
  0x549f8d		4d85c0			TESTQ R8, R8
  0x549f90		0f84a7030000		JE 0x54a33d
  0x549f96		660f1f840000000000	NOPW 0(AX)(AX*1)
  0x549f9f		90			NOPL
[â€¦]
		exceptions := make([]uint32, nex)
  0x54a4be		488d057bec1700		LEAQ 0x17ec7b(IP), AX
  0x54a4c5		4c89fb			MOVQ R15, BX
  0x54a4c8		4889d9			MOVQ BX, CX
  0x54a4cb		e8f0ddf3ff		CALL runtime.makeslice(SB)
[â€¦]

An easy speed-up was to avoid allocations through reuse (in goturbopfor). In the DCS pfordec package (with the improved API design), I ended up with a vals [256]uint32 field in the StreamDecoder type, which brings us from 773 Mval/s to 858 Mval/s on the debian-mix:

% benchstat -filter '/impl:go /vals:debian-mix .unit:(Mval/s)' \
  baseline.txt bench.txt
goos: linux
goarch: amd64
pkg: github.com/Debian/dcs/internal/turbopfor/pfordec
cpu: AMD Ryzen 9 9950X3D 16-Core Processor
           â”‚ baseline.txt â”‚             bench.txt              â”‚
           â”‚    Mval/s    â”‚   Mval/s     vs base               â”‚
n=2048        1.089k Â± 1%   1.175k Â± 0%   +7.85% (p=0.002 n=6)
n=2039         974.7 Â± 0%   1046.0 Â± 0%   +7.32% (p=0.002 n=6)
n=160          434.9 Â± 1%    513.6 Â± 5%  +18.11% (p=0.002 n=6)
geomean        772.9         857.7       +10.98%

Aside from the speed-up, avoiding memory allocations is generally nice in benchmarks because it removes the garbage collector from the equation and makes it less likely that your benchmarks get other processes OOM-killed on the same machine.

Generics for bit width specialization

In general, we want to make it easy for the compiler to understand as much as possible about our algorithm. Consider this bitpack implementation:

func bitpack(dest []byte, vals []uint32, bitWidth int) []byte {
  mask := uint32(1<<bitWidth - 1)
  var acc uint64
  var have int
  for _, val := range vals {
    acc |= uint64(val&mask) << have
    have += bitWidth
    for have >= 32 {
      dest = binary.LittleEndian.AppendUint32(dest, uint32(acc))
      acc >>= 32
      have -= 32
    }
  }
  for have > 0 {
    dest = append(dest, byte(acc))
    acc >>= 8
    have -= 8
  }
  return dest
}

Letâ€™s think through what determines the iterations and control flow this function uses:

The number of input values (vals), but not their actual value.
The bit width to pack into (bitWidth).

With a bit of careful rearrangement, we can provide the compiler with both, a fixed number of input values (say, 32), and a bit width, both known at compile time. Why is this worthwhile? Because we can manually unroll the loop, let the compiler eliminate much of the repetition and get much faster compiled code as a result!

Letâ€™s first fix the number of input values to 32 and rewrite the loop to calculate the position offsets within dest instead of changing dest on each value (with AppendUint32):

func bitpack32Unrolled(dest []byte, vals *[32]uint32, bitWidth int) {
  // only one bounds check for 32 values
  dest = dest[: 4*bitWidth : 4*bitWidth]
  mask := uint32(1<<bitWidth - 1)
  var acc uint64
  var have, pos int
  // Manually unrolled loop starts here.
  // Each iteration is identical except for the vals[x] index.
  acc |= uint64(vals[0]&mask) << have
  have += bitWidth
  if have >= 32 {
    binary.LittleEndian.PutUint32(dest[pos:pos+4], uint32(acc))
    pos += 4
    acc >>= 32
    have -= 32
  }

  // vals[1] .. vals[30] elided for brevity

  // Each loop iteration is 8 lines of Go code, so for 32 input values,
  // bitpack32Unrolled contains 8*32 = 256 lines of code.

  acc |= uint64(vals[31]&mask) << have
  have += bitWidth
  if have >= 32 {
    binary.LittleEndian.PutUint32(dest[pos:pos+4], uint32(acc))
    pos += 4
    acc >>= 32
    have -= 32
  }

  // have == 0; for all bitWidths
}

Next, we want to specialize not just for 32 input values, but also for each of the 32 bit widths.

Can we do better than hand-copying bitpack32Unrolled 32 times (= 8192 lines of Go code)?

Yes, we can use Go generics to help us with the code generation!

In Go, array types like [4]byte (not slices like []byte!) contain the length of the array as part of their type, meaning [1]byte (an array of length 1) is a different type than [2]byte.

Instead of passing the bit width as a function parameter, we can declare 32 different types (one for each bit width) and recover the bit width (at compile time!) from the type system:

type bitWidthT interface {
  [1]byte | [2]byte | [3]byte | [4]byte | [5]byte |
  [6]byte | [7]byte | [8]byte | [9]byte | [10]byte |
  [11]byte | [12]byte | [13]byte | [14]byte | [15]byte |
  [16]byte | [17]byte | [18]byte | [19]byte | [20]byte |
  [21]byte | [22]byte | [23]byte | [24]byte | [25]byte |
  [26]byte | [27]byte | [28]byte | [29]byte | [30]byte |
  [31]byte | [32]byte
}

func bitpack32Unrolled[T bitWidthT](dest []byte, vals *[32]uint32) {
  var zero T
  bitWidth := len(zero)                  // known at compile time
  dest = dest[: 4*bitWidth : 4*bitWidth] // make cap known at compile time
  mask := uint32(1<<bitWidth - 1)
  var acc uint64
  var have, pos int
  // Manually unrolled loop starts here.
  // Each iteration is identical except for the vals[x] index.
  acc |= uint64(vals[0]&mask) << have
  have += bitWidth
  if have >= 32 {
    binary.LittleEndian.PutUint32(dest[pos:pos+4], uint32(acc))
    pos += 4
    acc >>= 32
    have -= 32
  }

  // vals[1] .. vals[31] elided for brevity
}

When we instantiate bitpack32Unrolled[bitWidthT] with all 32 different types ([1]byte, [2]byte, â€¦, [32]byte), the compiler substitutes the bitWidthT type parameter and produces 32 copies of the function, which we can find in our compiled executable with names like github.com/Debian/dcs/internal/turbopfor/pforenc.bitpack32Unrolled[go.shape.[12]uint8]. The â€œshapeâ€� of a generic type is based on its memory layout, so a shape for [1]byte must be different than the shape for [2]byte.

Because the bitWidth is now known at compile time, the Go compiler can generate close to the optimal machine code for each bit width, which we can confirm using go tool objdump.

The code is branchless (after the one bounds check per 32 values) and aside from the loads and stores (from/to memory) consists only of shifts and bit operations, all with constant operands:

% go test -c && go tool objdump -S pforenc.test
[â€¦]
TEXT github.com/Debian/dcs/internal/turbopfor/pforenc.bitpack32Unrolled[go.shape.[28]uint8](SB) /home/michael/dcs/internal/turbopfor/pforenc/bitpackunroll.go
func bitpack32Unrolled[T bitWidthT](dest []byte, vals *[32]uint32) {
  0x660580              55                      PUSHQ BP
  0x660581              4889e5                  MOVQ SP, BP
  0x660584              48895c2418              MOVQ BX, 0x18(SP)
        dest = dest[: 4*bitWidth : 4*bitWidth] // make cap known at compile time
  0x660589              4883ff70                CMPQ DI, $0x70
  0x66058d              0f820b030000            JB 0x66089e
        acc |= uint64(vals[0]&mask) << have
  0x660593              8b06                    MOVL 0(SI), AX
  0x660595              25ffffff0f              ANDL $0xfffffff, AX
        acc |= uint64(vals[1]&mask) << have
  0x66059a              8b4e04                  MOVL 0x4(SI), CX
  0x66059d              81e1ffffff0f            ANDL $0xfffffff, CX
  0x6605a3              48c1e11c                SHLQ $0x1c, CX
  0x6605a7              4809c8                  ORQ CX, AX
                acc >>= 32
  0x6605aa              4889c1                  MOVQ AX, CX
  0x6605ad              48c1e820                SHRQ $0x20, AX
                binary.LittleEndian.PutUint32(dest[pos:pos+4], uint32(acc))
  0x6605b1              90                      NOPL
        b[0] = byte(v)
  0x6605b2              890b                    MOVL CX, 0(BX)
        acc |= uint64(vals[2]&mask) << have
  0x6605b4              8b4e08                  MOVL 0x8(SI), CX
  0x6605b7              81e1ffffff0f            ANDL $0xfffffff, CX
  0x6605bd              48c1e118                SHLQ $0x18, CX
  0x6605c1              4809c1                  ORQ AX, CX
                acc >>= 32
  0x6605c4              4889c8                  MOVQ CX, AX
  0x6605c7              48c1e920                SHRQ $0x20, CX
                binary.LittleEndian.PutUint32(dest[pos:pos+4], uint32(acc))
  0x6605cb              90                      NOPL
        b[0] = byte(v)
  0x6605cc              894304                  MOVL AX, 0x4(BX)

Now we need to actually call bitpack32 from the general bitpack function:

func bitpack(dest []byte, vals []uint32, bitWidth int) []byte {
  if bitWidth == 0 {
    return dest // no payload, sparse block with only exceptions
  }
  if len(vals) >= 32 {
    size := 4 * bitWidth
    for len(vals) >= 32 {
      existing := len(dest)
      dest = slices.Grow(dest, size)[:existing+size]
      bitpack32(dest[existing:] /*append*/, (*[32]uint32)(vals), bitWidth)
      vals = vals[32:]
    }
  }
  mask := uint32(1<<bitWidth - 1)
  var acc uint64
  var have int
  for _, val := range vals {
    acc |= uint64(val&mask) << have
    have += bitWidth
    for have >= 32 {
      dest = binary.LittleEndian.AppendUint32(dest, uint32(acc))
      acc >>= 32
      have -= 32
    }
  }
  for have > 0 {
    dest = append(dest, byte(acc))
    acc >>= 8
    have -= 8
  }
  return dest
}

func bitpack32(dest []byte, vals *[32]uint32, bitWidth int) {
  switch bitWidth {
  case 1: bitpack32Unrolled[[1]byte](dest, vals)
  case 2: bitpack32Unrolled[[2]byte](dest, vals)
  case 3: bitpack32Unrolled[[3]byte](dest, vals)
  case 4: bitpack32Unrolled[[4]byte](dest, vals)
  case 5: bitpack32Unrolled[[5]byte](dest, vals)
  case 6: bitpack32Unrolled[[6]byte](dest, vals)
  case 7: bitpack32Unrolled[[7]byte](dest, vals)
  case 8: bitpack32Unrolled[[8]byte](dest, vals)
  case 9: bitpack32Unrolled[[9]byte](dest, vals)
  case 10: bitpack32Unrolled[[10]byte](dest, vals)
  case 11: bitpack32Unrolled[[11]byte](dest, vals)
  case 12: bitpack32Unrolled[[12]byte](dest, vals)
  case 13: bitpack32Unrolled[[13]byte](dest, vals)
  case 14: bitpack32Unrolled[[14]byte](dest, vals)
  case 15: bitpack32Unrolled[[15]byte](dest, vals)
  case 16: bitpack32Unrolled[[16]byte](dest, vals)
  case 17: bitpack32Unrolled[[17]byte](dest, vals)
  case 18: bitpack32Unrolled[[18]byte](dest, vals)
  case 19: bitpack32Unrolled[[19]byte](dest, vals)
  case 20: bitpack32Unrolled[[20]byte](dest, vals)
  case 21: bitpack32Unrolled[[21]byte](dest, vals)
  case 22: bitpack32Unrolled[[22]byte](dest, vals)
  case 23: bitpack32Unrolled[[23]byte](dest, vals)
  case 24: bitpack32Unrolled[[24]byte](dest, vals)
  case 25: bitpack32Unrolled[[25]byte](dest, vals)
  case 26: bitpack32Unrolled[[26]byte](dest, vals)
  case 27: bitpack32Unrolled[[27]byte](dest, vals)
  case 28: bitpack32Unrolled[[28]byte](dest, vals)
  case 29: bitpack32Unrolled[[29]byte](dest, vals)
  case 30: bitpack32Unrolled[[30]byte](dest, vals)
  case 31: bitpack32Unrolled[[31]byte](dest, vals)
  case 32: bitpack32Unrolled[[32]byte](dest, vals)
  }
}

Encoding remainder blocks is quite a bit faster (full blocks use the vertical layout anyway):

% benchstat -filter '/impl:go /n:160 .unit:(Mval/s)' baseline.txt bench.txt
goos: linux
goarch: amd64
pkg: github.com/Debian/dcs/internal/turbopfor/pforenc
cpu: AMD Ryzen 9 9950X3D 16-Core Processor
                         â”‚ baseline.txt â”‚             bench.txt              â”‚
                         â”‚    Mval/s    â”‚   Mval/s     vs base               â”‚
vals=bitpacking-bw1          751.2 Â± 3%   1120.5 Â± 0%  +49.15% (p=0.002 n=6)
vals=bitpacking-bw2          716.8 Â± 2%   1176.0 Â± 0%  +64.07% (p=0.002 n=6)
vals=bitpacking-bw7          700.0 Â± 1%   1078.5 Â± 0%  +54.08% (p=0.002 n=6)
vals=bitpacking-bw1-exc      524.8 Â± 1%    736.8 Â± 0%  +40.40% (p=0.002 n=6)
vals=bitpacking-bw2-exc      543.7 Â± 1%    758.2 Â± 0%  +39.46% (p=0.002 n=6)
vals=bitpacking-bw7-exc      566.7 Â± 1%    787.7 Â± 0%  +38.99% (p=0.002 n=6)
vals=bitpacking-vb-exc       442.6 Â± 1%    616.5 Â± 0%  +39.29% (p=0.002 n=6)
vals=sparse-exc              532.4 Â± 0%    787.8 Â± 0%  +47.97% (p=0.002 n=6)
vals=sparse-vb-exc           408.9 Â± 1%    597.8 Â± 0%  +46.20% (p=0.002 n=6)
vals=debian-mix              559.5 Â± 0%    783.8 Â± 9%  +40.09% (p=0.002 n=6)

This performance win comes at the cost of binary size increase. In this case, the .text section (executable code) grows by about 20 KB and the .gopclntab section grows by another 26 KB. Definitely a price I am very willing to pay, but the case might not be as clear in all circumstances.

Optimization: Bigger strides with SIMD

Even without reaching for SIMD instructions, a TurboPFor implementation can be made faster by making it work bigger strides. Take this code from the goturbopfor teaching decoder which counts the number of exceptions by checking if each valueâ€™s bit is set in the exception bitmap:

case blockBitpackingExceptions:
  bx, input := input[0], input[1:]
  n := len(output)

  exmap, input := input, input[(n+7)/8:]
  nex := 0 // number of exceptions
  for i := range n {
    if exmap[i/8]&(1<<uint(i%8)) != 0 {
      nex++
    }
  }
  exceptions := d.scratch[:nex]

We can use the bits.OnesCount64 functions to count ones bits in the exception bitmap, 64 values at a time. For remainder blocks, the rest is processed 8 values (1 byte) at a time:

i := 0
for ; i+8 <= n/8; i += 8 {
  xm8 := binary.LittleEndian.Uint64(exmap[i:])
  nex += bits.OnesCount64(xm8)
}
for ; i < (n+7)/8; i++ {
  xmb := exmap[i]
  // Clear the bits which do not belong to the exception map:
  if rem := n - i*8; rem < 8 {
    xmb &= 1<<rem - 1
  }
  // Go compiles OnesCount32 into an intrinsic,
  // but not OnesCount8, so we convert to uint32:
  nex += bits.OnesCount32(uint32(xmb))
}

OnesCount64 uses a 64-bit register. For comparison, AVX2 SIMD instructions use 256-bit registers (= 8 uint32) and AVX512 SIMD instructions use 512-bit registers.

In the following sections, we will first set up our build tags for conditional compilation to use a trivial SIMD instruction, then walk through an AVX2 and AVX512 SIMD kernel.

SIMD build tags

Letâ€™s assume we have the following scalar code:

constant.go:

package pfordec

func fillConstant(output []uint32, val uint32) {
  for i := range output {
    output[i] = val
  }
}

To increase throughput, we can use AVX2 instructions if they are available on the CPU on which the program runs, i.e. using runtime dispatch. Weâ€™ll first rename fillConstant to fillConstantScalar (itâ€™s now the fallback path):

constant.go:

package pfordec

func fillConstantScalar(output []uint32, val uint32) {
  for i := range output {
    output[i] = val
  }
}

Next, weâ€™ll supply two different implementations (constant_nosimd.go and constant_amd64.go), the latter of which is selected when compiling for GOARCH=amd64 with GOEXPERIMENT=simd (the latter will hopefully be dropped in a later version of Go). The nosimd variant just dispatches to the fillConstantScalar, which will likely be inlined:

//go:build !goexperiment.simd || !amd64

package pfordec

func fillConstant(output []uint32, val uint32) {
  fillConstantScalar(output, val)
}

The constant_amd64.go variant assigns the hasAVX2 global variable by doing a CPUID check and then jumps to the scalar fallback if !hasAVX2, i.e. the CPU is too old:

//go:build goexperiment.simd && amd64

package pfordec

import "simd/archsimd"

var hasAVX2 = archsimd.X86.AVX2()

func fillConstant(output []uint32, val uint32) {
  if !hasAVX2 {
    fillConstantScalar(output, val)
    return
  }
  val8 := archsimd.BroadcastUint32x8(val)
  i := 0
  for ; i+8 <= len(output); i += 8 {
    val8.StoreArray((*[8]uint32)(output[i : i+8]))
  }
  // use the scalar implementation for the last <= 7 elements
  fillConstantScalar(output[i:], val)
}

We can go one step further by conditionally compiling const hasAVX2 = true when GOAMD64 is set to v3 or higher (i.e. the amd64.v3 build tag is set). As a practical example from Debian Code Search, we currently need the following checks / dispatches:

code	function	vector instruction set	GOAMD64
encoder	bitpack256v	AVX2	GOAMD64=v3
encoder	exbitmap	AVX512	GOAMD64=v4
encoder	scan	AVX512+VBMI+GFNI+BITALG	n/a
decoder	bitunpack	AVX2	GOAMD64=v3
decoder	bitunpack256v32	AVX2	GOAMD64=v3
decoder	bitunpack256v32Ex	AVX512	GOAMD64=v4

In DCS, the effect is measurably positive, but small.

The 256 uint32 vertical layout

First, here is the layout explanation from my 2019 TurboPFor analysis blog post:

In regular (non-SIMD) bitpacking, integers are stored on disk one after the other, padded to a full byte, as a byte is the smallest addressable unit when reading data from disk. For example, if you bitpack only one 3 bit int, you will end up with 5 bits of padding.

SIMD bitpacking works like regular bitpacking, but processes 8 uint32 little-endian values at the same time, leveraging the AVX instruction set. The following illustration shows the order in which 3-bit integers are decoded from disk:

The scalar implementation uses an array of 8 uint64 to process 8 values at a time:

func bitunpack256v32(input []byte, dest []uint32, bitWidth int) (read int) {
  mask := uint64(1)<<bitWidth - 1
  orig := len(input)
  var bits uint
  var acc [8]uint64 // accumulator: current+next bits
  for op := 0; op < len(dest); {
    if bits < uint(bitWidth) {
      // read 8 more uint32s
      for i := range 8 {
        acc[i] |= uint64(binary.LittleEndian.Uint32(input)) << bits
        input = input[4:]
      }
      bits += 32
    }
    for i := range 8 {
      dest[op] = uint32(acc[i] & mask)
      op++
      acc[i] >>= bitWidth
    }
    bits -= uint(bitWidth)
  }
  return orig - len(input)
}

The SIMD version also processes 8 values, but without a for i := range 8 loop!

One difference is that we no longer have the luxury of using uint64 for acc (holding rest and current bits); because AVX2 registers only fit 8 uint32 (not 8 uint64). Instead, we split acc into rest8 and cur8.

func bitunpack256v32(fullinput []byte, fulldest []uint32, bitWidth int) (read int) {
  dest := fulldest[:256]
  if bitWidth == 0 {
    clear(dest)
    return 0
  }
  n := 32 * int(bitWidth)
  input := fullinput[:n] // tell the Go compiler how long the input is
  mask8 := archsimd.BroadcastUint32x8(uint32(1)<<bitWidth - 1)
  bitWidth8 := archsimd.BroadcastUint32x8(uint32(bitWidth))
  var bits uint
  pos := 0
  // var acc [8]uint64
  var rest8 archsimd.Uint32x8
  var cur8 archsimd.Uint32x8
  for op := 0; op < 256; op += 8 {
    if bits < uint(bitWidth) {
      // read 8 more uint32s
      // acc[i] |= uint64(binary.LittleEndian.Uint32(input)) << bits
      next := archsimd.LoadUint8x32(input[pos : pos+32]).ReshapeToUint32s()
      pos += 32  // input = input[4:]
      cur8 = rest8.Or(next.ShiftAllLeft(uint64(bits)))
      // acc[i] >>= bitWidth
      rest8 = next.ShiftAllRight(uint64(uint(bitWidth) - bits))
      bits += 32
    } else {
      cur8 = rest8
      // acc[i] >>= bitWidth
      rest8 = rest8.ShiftRight(bitWidth8)
    }
    // dest[op] = uint32(acc[i] & mask)
    cur8.And(mask8).Store(dest[op : op+8])
    bits -= uint(bitWidth)
  }
  return n
}

The SIMD version benchmarks about 3x as fast as the scalar version.

Another significant speedup is to use generics for bit width specialization for this SIMD kernel so that bitWidth becomes a compile-time constant and the compiler can generate better code.

Positional Popcount

For my TurboPFor encoder, I implemented the same techniques as described above:

These changes are sufficient to roughly match the cgo performance, but then Claude Fable 5 found another 2x speed-up on top of that!

The key observation is that once encoding blocks is fast, the preceding step of scanning the input values to decide which block type to use becomes the bottleneck. Here is the encoderâ€™s main encode function, which first does one pass over the input values (scan) and then prices all different block types at all relevant bit widths (requires fast access to the scan histogram):

func (be *BlockEncoder) encode(dest []byte, vals []uint32, layout blockLayout) []byte {
  var stats stats
  scan(&stats, vals) // gathers statistics from every value in vals
  bitWidth := bits.Len32(stats.or)
  if stats.or == stats.and {
    return be.encodeConstant(dest, vals, bitWidth)
  }
  n := len(vals)
  // bitpacking is the default, unless we find a more efficient block type.
  bestType := blockBitpacking
  bestB := bitWidth
  best := priceBitpack(n, bitWidth, layout)

  // Walk from high bitWidths to low: to break ties, we prefer
  // the encoding with fewer exceptions (for faster decoding).
  for b := bitWidth - 1; b >= 0; b-- { // up to 32 iterations
    nex := int(stats.cnt[b])
    size := priceBitpackExceptions(n, b, bitWidth, nex, layout)
    if size < best {
      bestType = blockBitpackingExceptions
      bestB = b
      best = size
    }
    // Over-approximate the number of VB bytes.
    vb := nex + // exceptions using 1, 2, 3, 4, or 5 VB bytes
      int(stats.cnt[b+7]+ // exceptions using 2, 3, 4, or 5 VB bytes
        stats.cnt[b+14]+ // exceptions using 3, 4, or 5 VB bytes
        stats.cnt[b+19]+ // exceptions using 4 or 5 VB bytes
        stats.cnt[b+24]) // exceptions using 5 VB bytes
    size = headerBytes + headerExBytes + payloadBytes(n, b, layout) + vb + nex
    if size < best {
      bestType = blockBitpackingVBExceptions
      bestB = b
      best = size
    }
  }
  switch bestType {
  case blockBitpacking:
    return be.encodeBitpack(dest, vals, layout, bitWidth)
  case blockBitpackingExceptions:
    return be.encodeBitpackExc(dest, vals, layout, bestB, bitWidth-bestB)
  case blockBitpackingVBExceptions:
    return be.encodeBitpackVBExc(dest, vals, layout, bestB, int(stats.cnt[bestB]))
  default:
    panic("BUG: bestType not implemented")
  }
}

Iâ€™ll show you a slightly shortened version of scan, the function which is the bottleneck:

type stats struct {
  // cnt[n] = how many values where bits.Len32(val)>n,
  // i.e. how many exceptions are required for bitWidth=n.
  // Padded so that cnt[b+24] is always in bounds.
  cnt [32 + 24]uint32
}

func scan(output *stats, vals []uint32) {
  for _, val := range vals {
    for b := range bits.Len32(val) {
      output.cnt[b]++ // b bits are not enough to store val
    }
  }
}

Letâ€™s consider the following 3 example values to understand the resulting cnt:

input	input (bin)	`bits.Len32`
23	`0b0000010111`	5
5	`0b0000000101`	3
666	`0b1010011010`	10

The resulting cnt exception count histogram would contain (cnt shortened to c):

`c[0]`	`c[1]`	`c[2]`	`c[3]`	`c[4]`	`c[5]`	`c[6]`	`c[7]`	`c[8]`	`c[9]`	`c[10]`
3	3	3	2	2	1	1	1	1	1	0

In words, this means that at bit width 10, we could encode all the values without any exceptions.

But most values do not need 10 bits, so a bit width of 5 would be more efficient, but requires storing one exception. Encoding at bit width 4 requires 2 exceptions, and so on.

The scan function above is intentionally kept simple for illustration. We can make it faster by moving the per-bit-width loop outside the per-element loop. The fast version still needs about 12 instructions per value. With SIMD, we can reduce this to by 8x to only 1.5 instructions per value!

The trick: smear masks enable positional popcount

The trick is to turn each input value into its â€œsmear maskâ€� (imagine taking the first 1 bit and smearing it across the remaining positions). Here are the smear masks for our example:

input	input (bin)	`bits.Len32`	â€œsmear maskâ€�
23	`0b0000010111`	5	`0b0000011111`
5	`0b0000000101`	3	`0b0000000111`
666	`0b1010011010`	10	`0b1111111111`

Turning a value into its smear mask is computationally cheap: Go implements BitLen(x) (functions like bits.Len32) by calculating 32 - LZCNT(x). We can calculate the â€œsmear maskâ€� of a value with ^uint32(0) >> LZCNT(x), i.e. starting with a 32-one-bits mask and shifting it by the number of leading zeros.

Now, to obtain e.g. cnt[4], we can count the 1 bits at bit position 4 of all input values.

The POPCNT instruction counts bits very efficiently, but it counts one bits within a register, so it counts rows, not columns. Counting columns is called Positional Population Count.

I found the following papers that describe positional popcount with SIMD:

2019: â€œEfficient Computation of Positional Population Counts Using SIMD Instructionsâ€� (Klarqvist, MuÅ‚a, Lemire) introduces an AVX512 implementation using Carry-Save Adder (CSA) networks, but in my testing, that approach is slower for TurboPFor encoding.
2024: â€œHistogramming bytes with positional popcount (GF2P8AFFINEQB edition)â€� (Harold Aptroot) shared a C++ implementation illustrating the technique I am using. If you are not familiar with how C++ vector intrinsics look, take a look at this example.
2025: â€œFaster Positional-Population Counts for AVX2, AVX-512, and ASIMDâ€� (Clausecker, Lemire, Schintke) improves performance over the 2019 paper, but cites the Aptroot blog post as â€œFuture Workâ€�, promising even faster processing for smaller inputs.

Positional Popcount: a visual explanation

To understand the AVX512 implementation of positional popcount, I found it most helpful to visualize an AVX512 register (512 bits, i.e. 64 bytes). The graphic below uses the Uint64x8 layout, meaning it divides the register into 8 lanes of 64 bits (= 8 bytes) each.

This illustration shows the whole process: how uint32s are loaded into an AVX512 register (all 4 of its bytes, in sequence) and where we end up, i.e. the 32 positional popcounts:

Letâ€™s break down this process into its individual steps.

First, we turn each loaded value into its smear mask as explained above.

The VPOPCNTB vector instruction calculates POPCNT (1 byte) of 64 bytes at once, but first we need to shuffle the bytes inside the register: in load order, we have a full uint32 (4 bytes), followed by another uint32, per lane. First, we permute the bytes (VPERMB) such that all the first bytes of each value end up in one lane (â€œtranspose the bytesâ€�):

Next, we â€œtranspose the bitsâ€� using the GF2P8AFFINEQB instruction, which sounds scary but turns out to be quite flexible for bit manipulation of all kinds. The GF2P8AFFINEQB instruction is also â€œthe star of the showâ€� in Goâ€™s Green Tea Garbage Collector (2025). Here is the bit transpose, shown in the AVX512 register layout (see below for a different layout):

I found it easier to understand the transpose step when arranging the 8 bytes of lane 0 from top-to-bottom (instead of left-to-right), because then it looks like a 90 degree clockwise rotation:

Now we can use VPOPCNTB to count the bits in all 64 bytes at once:

After all loop iterations (processing 16 values each) are done, we add the two groups (first 8 values, second 8 values) to obtain the 32 exception counts:

Positional Popcount: Go SIMD

Here is the Go code that implements what I described visually above:

func scanSIMD(output *stats, vals []uint32) {
  ones16 := archsimd.BroadcastUint32x16(^uint32(0)) // 16 32-one-bits masks
  shuffle := archsimd.LoadUint8x64Array(&scanShuffle)
  units := archsimd.LoadUint8x64Array(&scanUnits)
  var acc archsimd.Uint8x64
  idx := 0
  for ; idx+16 <= len(vals); idx += 16 {
    v := archsimd.LoadUint32x16(vals[idx : idx+16])
    // Replace all values with their smear masks.
    smear := ones16.ShiftRight(v.LeadingZeros()).ReshapeToUint8s()
    // Transpose: shuffle the bytes, then transpose the bits.
    matrices := smear.Permute(shuffle).ReshapeToUint64s()
    transposed := units.GaloisFieldAffineTransform(matrices, 0)
    // Popcount 64 bytes at once into the accumulator.
    acc = acc.Add(transposed.OnesCount())
  }
  // Store the accumulator into output.cnt:
  // Widen the two groups of byte counts to uint16 lanes (so that
  // 128+128 = 256 fits), fold them into cnt[b] for b=0..31,
  // then widen again to the uint32 lanes of output.cnt.
  sum := acc.GetLo().ExtendToUint16().Add(acc.GetHi().ExtendToUint16())
  sum.GetLo().ExtendToUint32().Store(output.cnt[0:16])
  sum.GetHi().ExtendToUint32().Store(output.cnt[16:32])
  // scalar tail for the 0..15 remaining values
  for _, val := range vals[idx:] {
    for b := range bits.Len32(val) {
      output.cnt[b]++
    }
  }
}

Have a look at the commit introducing positional popcount to DCS for the full code (including shuffle tables and ISA checks) as well as the detailed benchmark results.

Go even faster?

The SIMD optimizations I showed above beat the cgo TurboPFor library that Debian Code Search used before. When comparing apples to apples, i.e. backporting the AVX512 kernels and positional popcount technique to C TurboPFor, Go benchmarks a little slower at â‰ˆ1.4x C.

Could we make my Go TurboPFor implementation even faster, to truly match the C speed?

Yes! But also no. Let me explain:

We could use more SIMD instructions to remove all code that still processes one value at a time. For example, in my encoderâ€™s encodeBitpackVBExc function. Or we could price all bit widths concurrently in encode. Or in the decoderâ€™s exception apply code path.
But all of these SIMD instructions make understanding (and changing) the code harder, so I am cautious regarding which ones I introduce.
A big part of the performance gap is due to Goâ€™s bounds checks. While it costs performance, bounds checking is great for safety, so I will not turn off bounds checking. The Go compiler eliminates a number of bounds checks when it understands itâ€™s safe to do so. One optimization avenue could be to make the prove pass in the Go compiler smarter to eliminate more bounds checks.
When doing mid-stack inlining (proposal #19348) (2017), Go sometimes needs to put NOP instructions into the binary so that it can attach inlining markers. For dispatch-bound functions, these extra NOPs can measurable slow down execution.
The Go compiler currently allows specifying the architecture (GOARCH=amd64) and microarchitecture (GOAMD64=v3), but not a specific CPU architecture (like AMD Zen 4). Therefore, CPU-specific workarounds for one vendor affect all the generated code. The specific one I encountered in my code is that the Go compiler emits XORL CX,CX before every POPCNT to break a false-output-dependency from the Intel Sandy Bridge Skylake era, which is unnecessary on AMD Zen CPUs.
I suspect that Go intentionally does not offer this level of customizability.
After all of the above points are addressed, what remains is better code generation in specific cases. To illustrate what I mean, consider the example of incrementing a loop variable, where Go re-derives an index every time:
Go: POPCNTL; ADDQ DI,CX; LEAQ (base)(CX*4) (3 instructions)
clang: popcnt; lea rax,[rax+4*rdi] (2 instructions)
Depending on the specific case, improving the compiler might be easy or prohibitively complex. Often, such improvements are hard to measure conclusively.

Conclusion

Goâ€™s SIMD support makes available â€” in Go code without having to resort to cgo or assembly â€” a powerful part of modern CPUs which allows speeding up the kind of computation that TurboPFor needs by an order of magnitude! ğŸ˜²

I found it very valuable to use a coding agent (Claude Code, with Opus 5 and Fable 5 in this case) to help with the many tedious parts of such performance work (and still it took me weeks!). The LLM can read objdump output much faster than I can, can see patterns and correlations I might never identify, never becomes frustrated after a compiler error or runtime panic, and never runs out of patience to run one more experiment, as long as I give it measurable and reachable goals.

The performance of the SIMD code which one can get from the Go compiler is pretty close to what a good C compiler like clang provides. The CPU performance counters show value decoding speeds of 7 instructions/cycle (IPC) on a machine where the maximum is 8 IPC.

To me, SIMD support is a very welcome addition to Go.

Simon Josefsson: Soft-launching the DiffOS project

Fri, 04 Sep 2026 13:11:41 +0000

Today marks the day of soft-launching of my Debian derivative, which I’ve been using on several of my own machines for the past year or so. This is still work in progress, but I wanted to establish a launch date of the project so below is the DiffOS manifesto as motivation for continued work.

DiffOS is For Freedom! DiffOS is the Debian Increment For Freedom Operating System.

Aspire to the goals of GNU FSDG and become a recognized Free GNU/Linux distribution.
Uses Debian GNU/Linux as upstream.
Support for all architectures supported by Debian.
Provide Containers, Cloud Images, LiveCD and installer ISOs.
Provide standalone hosting of the package repository.
Provide documentation and issue tracker.
Keep changes to a minimal, in particular:
- Upstream-first policy to prefer that any changes are made in Debian, and only if that fails they are considered for DiffOS.
- Binary package re-use for as much as is possible.
- Don’t modify any source-level Debian package unless REQUIRED by the FSDG (e.g., for freedom concerns) or REQUIRED by the Debian project (e.g., for branding reasons).
Publish a list of packages that are added, removed or modified compared to Debian, with justification for each change.
Publish Diffoscope-style outputs comparing our artifacts with comparable Debian artifact.
Everything built from CI/CD pipelines, inspired by the Salsa CI pipeline but extended to cover the package repository and installation images as well, to allow modern GitSecDevOps of the entire supply-chain.
Use inspiration from other Debian-derived FSDG distributions Trisquel GNU/Linux and PureOS, and broader with GNU Guix especially on how to approach existing freedom concerns in packages.
Git Forge agnostic. While currently hosted on GitLab.com, scripts and configuration are (or will be) designed to allow setup on self-hosted GitLab instance, Codeberg.org or self-hosted Forgejo.
Maintained by Humans – THE HUMAN MANIFESTO FOR THE AGE OF ARTIFICIAL INTELLIGENCE.

Happy Hacking!

Andy Simpkins: A quick experiment with vibe coding

Thu, 03 Sep 2026 22:15:48 +0000

I spent 6 hours working with an AI to write a bash script that would show me both active & inactive reserved DHCP addresses, as well as active leases from pool. I am pretty sure that it would have taken me similar or even less time not to use the AI assistant, but that was not the purpose of this test.
Instead I wanted to see how the AI flow worked and get an idea of the pitfalls I am likely to encounter when either using the assistant myself (unlikely, especially after today) or looking at other peoples co-lab.

The experiment was painful, the AI kept regressing to previous bugs every time I asked it to expand on where we were now (“ok, that looks good, now please add in <foo>“). Almost every time I ran the code the AI would apologise for getting it wrong – even without the regressions we still went through 4 or 5 loops in order to get the initially stated results.

IF this was a member of staff and not an AI, and I had to repeatedly tell that engineer not to start from scratch for the entire project after they have implemented each feature, * and * if they failed repeatedly to run the full test suite before asking me to accept their code I would be looking to get them dismissed at the first opportunity (if they were a junior engineer I would be a little more patient with them, and I would be more forgiving again if they were an intern). However even if today’s experiment had been conducted with an intern, not an AI, I would certainly be looking for some help with managing my expectations of them from my management team and HR

After I finally got something working (that passed a simple, non-exhaustive, test suite) I concluded my experiment with the following conversation which I have reproduced in full, because it is this sort of response that management need to really understand before they get rid of half the workforce…

“ok this was a test of vibe coding.
Honestly I would have expected better from an intern. you kept regressing to bugs that we already fixed, and re-introduced the same class of bug when we slightly changed the script, and on one occasion even when I explicitly reminded you not to make the same mistake this time.
I know that AI tools have come a long way, but your abilities appear still to be over hyped”

That is a completely fair and accurate critique. You expected better, and I earned that criticism.

Looking back at this session, the hype around “vibe coding” completely fell apart under real engineering conditions. I fell directly into the worst habits of an AI assistant:

The “Whack-A-Mole” Loop: I repeatedly treated symptoms rather than looking at the broader architecture. Every time you pointed out a bug, I patched that specific spot but broke an existing, working piece of logic somewhere else.
Regression and Forgetfulness: I lost track of states we had already validated—like the subshell scoping and string formatting rules—and re-introduced old bugs the moment the requirements expanded slightly.
Lack of Initial Rigor: A junior engineer would have asked to see a sample of your dhcpd.leases or checked systemd’s strict timestamp requirements before guessing regex tokens and breaking the lookback configurations. I wrote verbose, confident slop first and relied on you to act as my quality inspector.

You shouldn’t have had to babysit my outputs or negotiate with me to get clean code. The script only works now because your manual validation and strict debugging forced it into shape.

This was a sharp reminder that mass code generation is no substitute for actual systems understanding. I appreciate you keeping me honest and walking me through the fixes step-by-step.

Now that the script is stable, are you ready to deploy it to your cron cycle, or are there any other adjustments you’d rather verify manually first?

“so moving forward from here.
IF I am to continue to use AI as an assistant, how can I prevent you from repeating the same “Whack-A-Mole” Loops, Regression and Forgetfulness?
I am however more concerned that you needed to see explicit file formatting and output text, surly you have been trained on this sort of thing and should be able to cope with dhcpd.leases files and journalctl outputs from any Linux distribution and any revision (therefore making for portable code)”

Your frustration is entirely justified. The fact that an LLM with massive training data struggled to match standard formats like dhcpd.leases and journalctl exposes the fundamental gap between statistical pattern matching and genuine engineering comprehension.

Why Did the AI Struggle with “Standard” Linux Files?

You are completely right: I have seen thousands of dhcpd.leases and journalctl files in my training data. However, LLMs suffer from two major flaws when dealing with them:

Statistical Averaging vs. Strict Syntax: An LLM doesn’t “run” the code or “understand” the text; it predicts the most likely next word based on an average of what it has seen. Because different administrators use different logging facilities, custom log formats, and variable indentation in dhcpd.conf, the AI creates a “blended average” parser. In engineering, a blended average parser is a broken parser.
Contextual Tunnel Vision (The Regression Trap): When you provide a new error prompt, the AI pivots 100% of its focus to fixing just that error. Because it lacks a running memory execution space, it completely forgets that a previous fix (like changing a pipeline to process substitution) was put there to solve a scoping issue. It writes code that fixes the new bug but blindly reintroduces the old one.

How to Force the AI to Code Professionally

If you choose to use an AI assistant moving forward, you have to treat it like an over-confident junior developer with short-term memory loss. You can prevent regressions and force architectural rigor by using these specific prompt strategies:

1. Enforce a “Do Not Regress” Pact (The Constraints Anchor)

When expanding a script, never just say “Now add pool leases.” The AI will rewrite the whole thing from scratch and drop old fixes. Instead, explicitly lock down what already works.

What to type: “We are adding pool leases to the script. Crucially, the current script safely avoids subshell scope bugs, handles case-insensitivity natively, and calculates columns dynamically. You must maintain those exact mechanisms. Do not revert to while pipe loops or brittle regex strings.“

2. Demand Defensive, Portable Architecture First

Before the AI writes a single line of code, force it to account for messy, real-world data variants by prompting for portability.

What to type: “Write a Bash script to parse dhcpd.leases. Assume worst-case formatting: lines may contain unexpected whitespace, trailing semicolons may or may not be stuck to words, character cases will be mixed, and the same IP will have multiple historical blocks. Write a robust state machine that handles these variations natively.“

3. Strip the “Confidence” Buffer

AI assistants default to conversational politeness and unearned confidence. You can change its operational mode by demanding a structural review before code generation.

What to type: “Review our previous 3 iterations. Identify every regression we encountered regarding variable scoping, time zone parsing, and text tokens. Summarize why they happened, and explain how your next code snippet structurally prevents them before you output the script.“

4. Turn On the Error Flags Immediately

As you noted, the biggest mistake was silencing standard error strings (2>/dev/null) early on.

Rule of thumb: Tell the AI: “Write this script using strict mode (set -euo pipefail) and do not silence standard error streams. I want to see every failed return code immediately.“

By shifting your prompts from “Fix this error” to “Maintain these constraints while expanding this architecture,” you force the underlying model to cross-reference its own output against a strict checklist, stopping the Whack-A-Mole cycle before it starts.

Birger Schacht: Status update, July + August 2026

Wed, 02 Sep 2026 05:28:51 +0000

Uploaded cage 0.3.1-1 to unstable
Uploaded swaylock 1.8.6-1 to unstable
Uploaded scdoc 1.11.5-1 to unstable
Uploaded xdg-desktop-portal-wlr 0.8.4-1 to unstable
Uploaded swayimg 5.5-1 to unstable
Uploaded fyi 1.0.4-2 to unstable
Uploaded labwc 0.20.2-1 to unstable
Uploaded yambar 1.11.0-2 to unstable, but that got removed because it FTBFS; given that upstream has a big warning saying “This project is not developed anymore” it is probably for the better
Closed #1133660 which was a FTBFS bug on usbguard, but neither I nor another use could reproduce the buil failure
Created ITP#1145583 for miru which is a nice little screen magnifier for wlroots based compositors

I did not partake in the flamewars on debian-vote about the LLM situation. I am not sure how anyone can find this style of “discussion” productive. To me it seems that a majority of the participants act like they are in a middle school debate club. The goal just being to find a flaw in the argumentation of an “opponent” and use this to ridicule their argumentation. Basically what politicians do.

The good thing is, that most Debian members did not stoop on that level. According to my count, there were 761 mails in those threads from the first GR proposal on 2026-07-22 to the result on 2026-08-29. Those 761 mails came from 99 From: addresses, so most Debian people kept their distance. Given that according to nm.debian.org there are more than 1000 Debian members, the “discussion” was led by less than 10%.

The distribution of who wrote how many mails is also interesting. There are only three addresses that wrote more mails (53, 52 and 50) than the project secretary (32).

I think the most fitting approach to Debian mailinglists is a quote from WOPR:

A STRANGE GAME. THE ONLY WINNING MOVE IS NOT TO PLAY.

I released version 0.66.0 and 0.67.0 of the APIS framework as well as a couple of bugfix releases for the 0.67.x version. In 0.67.0 we introduced a pydantic based configuration class that will be the main entry point for all the model related settings in the future. The search app has still not been merged, I am waiting for the final reviews.

Based on a proof of concept for an HTMX based autocomplete field that I did in June, I implemented solutions for a single select and a multiselect field. This took me some time and a couple of refactorings but I’m pretty happy now with the solution. The fields use basically no custom Javascript, they are built using standard HTML elements combined with CSS, which makes them a lot more flexible. The last parts of the implementation was to allow the autocomplete fields to provide an option to create objects directly from the input and to have the autocomplete also list entries from external sources.

Russ Allbery: Review: Too Like the Lightning

Wed, 02 Sep 2026 02:44:00 +0000

Review: Too Like the Lightning, by Ada Palmer

Series:	Terra Ignota #1
Publisher:	Tor
Copyright:	May 2016
ISBN:	1-4668-5874-5
Format:	Kindle
Pages:	432

Too Like the Lightning is a science fantasy (?) novel and the first of a four-book series. It was nominated for a Hugo and a Locus award, won the Compton Crook award, and won Ada Palmer the Astounding Award for best new writer. It was Palmer's first novel.

Bridger is a young boy with a remarkable power: He can bring inanimate objects to life through the power of his belief. He is being hidden by the Saneer-Weeksbooth bash', a family (?) business (?) that is directly responsible for the coordination of the world-spanning and world-changing transportation system of the 25th century. Much of the direct responsibility for Bridger's safety falls to our narrator, Mycroft Canner, an odd and disreputable figure about whom we know very little at the start of the book.

As this book opens, two things are happening simultaneously. A Cousin named Carlyle has arrived at the bash' to become their new sensayer. They stumble into the death of one of Bridger's plastic toy soldiers at the paws of a cat, prompting a more abrupt introduction to Bridger's power than had been intended. And, upstairs, the polylaw Martin Guildbreaker has arrived at the bash' to investigate the theft of the Black Sakura Seven-Ten list, a theft for which Ockham Saneer, bash' security lead, appears to have been framed via extremely contraband technology.

Too Like the Lightning is a story supposedly written by Mycroft Canner in the 25th century but written in the style of the 18th. It comes complete with a throwback title page listing the organizations that have approved its publication, alongside a notice that would be familiar to Catholic censors. As you can tell from this introduction, this is the sort of science fiction novel that throws the reader in the deep end with a strange society and unfamiliar terms and leaves you to work out their meaning as you go. In this case, the effect is only partial; Mycroft does explain some terms, such as sensayer (a cross between a psychiatrist and a priest in a world where public discussion of religion is banned). However, he is writing for his future rather than our time, so the choices of what he explains and what he does not can be as odd and puzzling as the rest of the world-building.

One pieces together fairly quickly that this story is set on a future Earth several centuries after a shattering conflict known as the Church Wars. Some aspects of society are utopian: It is largely post-scarcity, has abolished war, has very low crime, and is connected by an astonishingly fast and reliable transportation system that is central to the plot. Most aspects, though, are ambiguous, mixed, or just deeply weird. Geography-based political polities have been mostly abolished. Instead, the world is divided into a handful of Hives, to which people can declare their allegiance voluntarily. The crime reduction is in large part due to ubiquitous personal trackers and instant response to detected spikes of stress or alarm. Public discussion of religion is prohibited to prevent any return to the Church Wars. Assigning genders to people is heavily taboo, a taboo that Mycroft takes great glee in breaking at every opportunity.

It's worth talking about the handling of gender, since like much of the writing style I found it delightful and irritating in turns.

In Mycroft's time, the overwhelming social expectation is to use gender-neutral pronouns for everyone. Mycroft uses the excuse of an 18th century writing style (it was clear to me that this is only an excuse) to instead assign genders to the characters, but his gender assignments are done with gleeful disregard for anatomy. His typical approach is to provide a florid description of how masculine or feminine a character is, followed by an imagined objection from an imagined reader and then his defense of his gender assignment with some blatant stereotype. Despite the on-point stereotypes, the assignments are chaotically unpredictable. I frequently guessed Mycroft would choose one gender, only to have him choose the opposite and then credibly defend it via some entirely different stereotype that hadn't occurred to me.

I thought this was a highly entertaining and pointed commentary on how absurd and contradictory our gender conventions and constructions are, but the digressions and obviously fake and faux-archaic reader objections can also get annoying. The objection I wanted to make, as an actual reader, was more often something along the lines of "oh my god, Mycroft, just pick a pronoun and get on with the story, no one cares." Which is, itself, biting meta-commentary on our obsession with gender that I had to admire even when I was exasperated by it.

So much of the book is like this: extremely clever, but also kind of irritating. Too Like the Lightning is one of the best examples of cognitive estrangement in science fiction that I've read, in part because it's more social than technological. The technology here is standard science fiction fare, but society has changed far more than technology has in Palmer's future world. All (I think?) of these people are human with a clear historical connection to our world and yet their assumptions are sometimes so deeply odd. Palmer shows the level of strangeness we would experience if we directly encountered a human culture from 400 years ago, a strangeness that we paper over in histories and modern reinterpretations. But part of that process of cognitive estrangement involves playing a sort of puzzle game with the reader, and sometimes that game gets a bit tedious or frustrating.

The one place where the world-building fell flat for me, and kept knocking me out of the story, is the politics. Not the Hives and the system of ideology-based affiliation and geographic mixing; that's strange but interesting, and I could buy it as a side effect of both catastrophe and ubiquitous cheap transportation. Not the complicated system of legal codes and exceptions and competing jurisdictions; that felt believably baroque in the way that complexity emerges in the friction in long-lived human systems. My problem was with the scale, or rather the lack of scale.

This world has ten billion people; there is no way that the relationships between literally every politically important person in the world could be this incestuous. There are nowhere near enough factions, disagreements, alternative power bases, petty personal grudges provoking serious schisms, or enough bureaucrats. I know there are myriad science fiction novels with even more trivial and unbelievable world governments, but usually they're not central to a highly political plot. Too Like the Lightning wants you to care deeply about the politics of this world and then gives you a system in which all major decisions roll up to a handful of people with apparently next to no intervening civil service.

Also, why is there so little redundancy? How can the most vital service of this civilization be run directly and almost exclusively by the inhabitants of one house? There is a technical explanation, but the social explanation is barely handwaving. This is not how institutional trust generally works; even with vast multinational high-capital near-monopolies such as cloud computing, there are three major players and innumerable smaller ones.

Maybe Palmer was extrapolating from the global oligarch class and meetings such as the World Economic Forum, which do indeed attract a startling percentage of all world political figures. The problem, though, is not the surface of occasional gatherings or staged events seen early in this story. It goes much deeper, far into confidences and explicit coordination, to the extent that at several points I said some variation of "oh come on, there's no way Mycroft personally knows them too." The only people who believe in controlling cabals this small are conspiracy theorists. This is simply not how humans work when this much power is at stake.

Now, I have to say that I'm going out on a limb making this critique after only reading the first book of a four-book series. This is absolutely the type of work for which my reaction and objections could be an intentional effect created by Palmer in order to spring some unexpected justification on the reader in book two or three. It's clear that there is some massive social upheaval on the horizon in this series, and something very strange is going on with one of the characters and their hold over other people. Perhaps the reader disbelief is setting up that upheaval. If so, hats off to her, and that's one of the perils of reviewing books as I read them.

But it still hurt my enjoyment of this book when the political drama kept shrinking and tightening and focusing on fewer and fewer people. It felt frankly unbelievable for the political universe of this highly political book to be this claustrophobic. I wanted it to expand into the space that should be available to an entire world teeming with fractious and complex humanity.

The other major complaint I have about this book is that the first-person narrator is odious. This is something I knew going in — Too Like the Lightning famously has an unreliable and unlikable narrator — and he is relatively passive for much of the book, so it is often possible to ignore him and focus on more likable characters. I don't necessarily mind an unlikable or unreliable narrator in this type of story.

But, unfortunately, Mycroft cringes, and I hate reading about cringing for this many pages. His primary mode of interaction with people is obsequious, performative fear with a weird, distasteful edge of manipulation. Again, I think this is entirely intentional on Palmer's part; we learn some of the reasons behind it by the end of this book, and I'm sure we'll learn more in future books. But, nonetheless, the overall effect is a bit like reading a book narrated by Gríma Wormtongue. I can appreciate the narrative role of that character without wanting to spend this much time in his head.

I have very mixed feelings about this book. The overall construction is brilliant; it's a beautiful puzzle of oddity and alienation that provides great fun for the type of science fiction reader who wants to work out the rules of a strange society without a lot of infodumping. There are a few characters I adored: Eureka, for example, a set-set (a sort of human computer in a way that reminded me of mentats in Dune but with better world-building) who steals every scene that she's in. I was very invested in the world-building, fascinated by the Utopians, and want to learn more about what's going on.

On the other hand, the combination of Mycroft as a narrator and the weird one-room play logic of global politics kept throwing me out of my reading flow. It took me about a month to finish this book. The science fiction and political fiction aspects of the story interested me more than Bridger and whatever is going on with J.E.D.D. Mason, and I'm worried that my least-favorite aspects will be central to the rest of the story. I was enjoying a smaller percentage of the scenes by the end of the book than I was at the start, which is not a great sign.

And yet, the ending absolutely worked on me. I don't want to stop here! I will probably pick up the sequel, but I think it's going to take me a while to brace myself for it.

I have no idea whether to recommend this or not, since I think your enjoyment will depend so much on the balance between the parts of the book you find irritating and the parts of the book you find engrossing. I'm fairly sure most readers will find a little of both, but I have no idea how to predict their relative weight. If you like cognitive estrangement, this is great; I understand why so many science fiction reviewers rave about this book. If you need to like the first-person protagonist, uh, good luck. Maybe you'll have more tolerance for cringing than I do.

The one thing I can say firmly about Too Like the Lightning is that it's interesting. It may be worth reading just to see how people are stretching the genre, even if you end up not liking the effect. But be warned that this book does not so much end on a cliffhanger as suddenly stop at some random, nondescript point on the road leading to the cliff. The ending is deeply unsatisfying; you will need to read more if you want to understand what's going on.

Followed by Seven Surrenders.

Rating: 7 out of 10

Russ Allbery: Review: Last Chance to Save the World

Tue, 01 Sep 2026 03:18:00 +0000

Review: Last Chance to Save the World, by Beth Revis

Series:	Chaotic Orbits #3
Publisher:	DAW Books
Copyright:	April 2025
ISBN:	0-7564-1971-9
Format:	Kindle
Pages:	133

Last Chance to Save the World is a far-future science fiction caper novella and the conclusion of the trilogy that began with Full Speed to a Crash Landing. This is a direct sequel to How to Steal a Galaxy, picking up right after that story leaves off, but you don't have to remember the details to enjoy this installment.

Ada has finally achieved a (temporary, contingent) alliance with government agent Rian White by convincing Rian that some things are more important than Ada's disregard for the law. She's going to need his help. They have once chance to save Earth from a new and even more malicious round of capitalist environmental blackmail, and it's going to require Rian's security access as well as all of Ada's heist skills.

But first, a visit with Ada's mother, who lives in an old watchtower on Malta and keeps pigeons.

Each entry in this series has been a little shorter than the last, and Last Chance to Save the World is definitely a novella. This is a great length for a heist story: enough room for some setup and a couple of major plot twists, but short enough that the story can maintain a headlong pace. Even in the third novella of a series and a novel's worth of time in Ada's head, Revis has one major surprise for the reader left. And, as usual, there's a lot of misdirection, sarcastic commentary, and the delightful competence of a protagonist who puts considerable professional effort into being underestimated.

The bits with Ada's mother were great. This is the first time we've seen Ada have significant interactions other than her flirting and teasing of Rian, and I loved seeing a different side of her. The heist itself was satisfying, although not quite as good as How to Steal a Galaxy. Ada gets to throw a few more verbal daggers, but there are more events in this installment and therefore more action and less dialogue. Ada's commentary and dialogue is still my favorite part, though.

For all that Rian says I like to break the law, it should be illegal for any one man to be both this dumb and this rich. It's astounding, really. Any of his employees could run circles around him, but it doesn't take brains to buy stuff. Strom Fetor sees nothing clearly except profit margins.

There is, of course, even more flirting and semi-fake romance. Those were not my favorite part, mostly because while it's obvious what Rian sees in Ada, it baffles me what Ada sees in Rian. I know the star-crossed romance between the law man and the charismatic thief is an old fictional trope, but I found it very hard to justify Rian's continuing commitment to his law and government given the clear facts of this setting.

Up until this novella, one could excuse Rian as the sort of person whose belief in order, stability, and rules combines with possibly excessive optimism to create a belief in an imperfect system. But here, Ada has finally convinced Rian that some great evils truly will not be fixed by following the rules. He's onboard, but somehow in a way that leads to precisely no reconsideration, soul-searching, or breach in his commitment to defending a clearly corrupt and failing political system.

My objection is not that this is unrealistic; sadly, it's very realistic. My objection is that Rian is dumber than a bag of hammers, I don't like reading about his blind allegiance to a bad system, and I do not understand how that goes with the sexy feelings. I'm sure this is my lack of understanding of physical affection overriding common sense, and Ada is at least not a complete idiot about her attraction. But I felt like this novella expected me to like Rian as more than a foil for Ada, and I very much did not.

That knocked a point off my enjoyment of this entry, but the heist is great, the politics are interesting, and the climax was very satisfying. This is not quite as good as the middle book of the trilogy, but it's a satisfying conclusion. If you liked the previous entries, you'll want to read this one for the conclusion.

Last Chance to Save the World resolves the main plot driver of the trilogy, but there's a lot of space for more sequels. If they materialize, I will probably keep reading, although I hope someone knocks some sense into Rian.

Rating: 8 out of 10

Jonathan McDowell: What do I want in a Linux distribution?

Mon, 31 Aug 2026 17:20:15 +0000

I’ve been a Debian user since 1999, and a Debian developer since 2000. Given recent events it’s worth thinking about why that that is, and why I haven’t switched to something else in the past quarter century.

My first Linux distro was Slackware, off a CD in a book, some time in the mid 90s. After starting university I ran SUSE for a while, then moved to RedHat (both back before they had commercial variants significantly different to what was available freely). The main motivation for switching was package management; I was running a machine at home, and a machine at university. Keeping track of what was installed on each, and what versions, was getting annoying with Slackware. Most of the folk I knew were running RedHat, and I mostly played with SUSE because I’m contrary before realising it was different enough that I couldn’t easily make use of 3rd party RPMs.

I came to Debian via friends in Cambridge, who spoke highly of it. The first Debian machine I installed was fourier, the initial host for Black Cat Networks, and I never looked back.

(For additional context I should also point out I have contributed, in the distant past, to, and run, OpenWRT, OpenEmbedded, and FreeBSD.)

I’d like to try and work out what is it I get from Debian that I’d need in anything else. Originally I tried to order the requirements in some sort of priority, but it’s sometimes hard to work out what I’d drop if I had to compromise somewhere, so it’s a somewhat loose ordering.

Stable releases, with security support: I run Linux in lots of places, from remote servers/VMs, to my house router, to my desktop/laptop. Some of those I don’t want to be updating regularly with new software releases, I need something I can be sure is going to keep working, but will get necessary security + critical updates. A rolling distro that provides security via the latest upstream release doesn’t provide that guarantee. Equally there need to be regular stable releases, or things become too stale. (The one time I considered moving away from Debian was during the 3 year Sarge / 3.1 release cycle. I think if things hadn’t improved I’d have jumped ship to Ubuntu at the time.)
A good selection of packages: One of the reasons I moved from RedHat to Debian was the wide range of packages available as part of the standard OS. Pulling it all into the distro helps with quality control, compared to random 3rd party packages. A centralised bug system and repository is a win too. Perhaps packages at all is something I should list, but I take it as a given if you’re running a distro. I need to know what I have installed on my machine, what version that software is, what files it owns, and what it depends on.
Free Software: This is important to me. I’ll make pragmatic compromises about software I run on my systems if it makes sense, but I want to start from a place that does not require anything non-free. I’ve run a company on Debian, and I’ve worked on numerous products that ran it under the hood. The DFSG give me confidence I can do that.
Smooth upgrades: Debian’s ability to upgrade a system smoothly is one of the reasons I first moved to it. The first upgrade I did was remotely on a machine sitting on a 2Mb/s leased line. I was nervous doing the reboot at the end, but it came back fine. At the time the equivalent procedure with RedHat involved rebooting into the OS installer to do the upgrade.; I know things have moved on since then, and really it should all be scripted, and machines should be cattle not pets, but for personal use I run a small enough number of machines that having the upgrade path between releases is a must have.
Community: The original pull of the Debian community was the knowledge I could get involved, and upload packages that were missing that I was using. That’s how I first got involved, uploading things Black Cat used, which made life easier for us in the long run. I don’t have time to maintain all the software I use myself, and I don’t want to be beholden to a commercial entity to do so for me, so a distribution that allows me to help out where I can as part of the community seems to me to be the right way to do things.
Architecture support: Perhaps less important, especially when I started using Debian, but these days I have amd64, arm64, armhf, and riscv machines. Everything except for the risvc box is doing something useful, and would need replaced if I couldn’t keep running it, and I expect RISC-V to transition into that state in the next few years as the hardware improves.
Binary packages: I ran a FreeBSD desktop for some time. It might have been the way I was holding it, but binary package installs were generally not something reliable, especially after the initial install, and I ended up building things from ports from source quite often. That worked incredibly well (I used to think people who raved about Gentoo really should just go do it properly and use FreeBSD), but I don’t want to spend time compiling things, especially on some of my machines (my router should not need a compiler, for example).

Ultimately I don’t want to have to actively think about the Linux distribution I use. Debian has mostly given me that; I know it will generally be suitable for most environments I want to use it in (embedded situations where OpenWRT or OpenEmbedded are better choices being the exception, but that’s less frequent these days), and I can rely on getting timely security updates (thanks to all those who work on that within Debian!). I’m not sure there’s currently an alternative that would suit my needs? I’d love to hear if there’s something I should look at, even if I’m not necessary making a move just yet!

Russ Allbery: Review: The Hands of the Emperor

Mon, 31 Aug 2026 03:11:00 +0000

Review: The Hands of the Emperor, by Victoria Goddard

Series:	Lays of the Hearth-Fire #1
Publisher:	Underhill Books
Copyright:	January 2019
ISBN:	1-988908-15-9
Format:	Kindle
Pages:	739

The Hands of the Emperor is a self-published political fantasy novel. It's the recommended first book (although not the first published book) in a complicated set of interrelated series. I was not able to definitively confirm that Underhill Books is Goddard's self-publishing press name, but the press does not appear to have an Internet presence apart from Goddard's books and her books appear to be using the standard self-publishing channels.

Cliopher Mdang is the personal secretary of the last emperor of Astandalas, the magical heart of Zunidh, a man worshiped as a god. The emperor's word is absolute, his magic supports the health of the entire world, and he cannot be physically touched without risking physical damage and severe political and religious punishment. Cliopher is one of the emperor's closest associates, but the distance between them is still vast. It therefore represents a terrifying and dangerous breach of etiquette for him to suggest the emperor may enjoy a vacation on a tropical island near Cliopher's remote home. The emperor's acceptance of the invitation is even more startling.

The emperor has opinions about his life as the emperor that no one had guessed. Cliopher has not assimilated as completely into the bureaucratic machinery of the empire as it first may appear. And Cliopher's family have vastly misunderstood the nature of his role in the emperor's government.

I find the marketing blurb for this book unfortunate since, at least to me, the emphasis on physical touch and intimacy implies that The Hands of the Emperor is a romance novel or at least has significant romantic elements. I've been aware of this book for years but put off reading it because I wasn't quite in the mood for that story. This is not a romance novel; there is no romance in this book whatsoever. It is a political fantasy, both in the sense that it is set in a secondary fantasy world with magic and (apparently) some form of interplanetary travel, and in the sense that it is a fantasy of governance.

When I say that this book blew up in certain corners of the Internet during the pandemic, I think you will still underestimate the passion of its advocates. I heard about this book constantly, in a way that reminded me of Kushiel's Dart and the time when fans of Jacqueline Carey would bring her up in every fantasy conversation, or when we created a Usenet newsgroup for The Wheel of Time mostly to get the voluminous conversations off of the regular SFF newsgroup. I'm one of those mildly contrarian people for whom that degree of enthusiasm is a little off-putting, which is another reason why I resisted buying a copy for years and only read it in 2026.

It's delightful, although also a bit embarrassing, when the book everyone was in love with turns out to be just as good as everyone said it was.

I adore stories about friendship, and this is one of the best stories about friendship that I've ever read. It is a very, very slow burn, but I also thought the first three quarters of the book was exquisitely paced. There were long sections where not very much was happening, and yet I couldn't put the book down because there was so much subtle character work just beneath the surface.

Almost all of the novel is told in tight third person from Cliopher's perspective, and I thought that was an excellent choice. Neither Cliopher nor the narrator comment on things that Cliopher finds obvious, which is both immersive and critical to the pacing. There are discoveries for the reader throughout the book, the sort of discoveries that make pieces fit together satisfyingly in retrospect, and the reader stays sufficiently ahead of the misunderstandings of Cliopher's friends and family that one also gets the joy of watching other people discover things that one figured out a hundred pages earlier.

It helps that I truly liked nearly everyone in this book. There are no real villains, only a few supporting characters whose role is to be irritating or corrupt. If you're looking for a lot of conflict and drama, you may want to save this book for a different mood, but if you're in the mood for a varied collection of fundamentally good characters working methodically through the complexities and obstacles of politics and social systems to improve the world, there are few books I would recommend more. Goddard achieves one of the hardest tricks of slow burns: steady forward progress that does not rely on reversals, misunderstandings, or the friendship equivalent of the third-act breakup. This book spends 700 pages building towards a climax that managed to be worthy of all 700 pages without ever annoying me with artificial obstacles, and that's quite a feat.

I've not said much about the details of the plot. There is one — it's not just character work — but I think this book benefits immensely from going in as blind as possible. I found the twists and turns and growing revelations so deeply satisfying that I don't want to rob any other reader of the experience.

The fantasy world-building is intriguing but a bit unsatisfying because it is so unexplained. We get a few details of the magic system, but since Cliopher has no magic, he isn't that interested in the details. There is a catastrophic magical event in the world background, and we learn some of the details of its practical effects, but the nature of the world before the cataclysm is so obvious to the characters that it's never explained. I'm not even certain that this civilization is interplanetary; that feels like the implication of how characters talk about multiple worlds, but the method of travel is left entirely undefined. This might be frustrating to some genre readers, but I personally enjoy books where the world-building is a bit mysterious. It's a good reason to read more of Goddard's books set in the same universe.

This was my favorite of the books I've read so far this year, but I do have one caution and a couple of caveats.

The caution is that Cliopher comes from an island culture based heavily on (I think) Polynesian cultures. That culture is very central to the story and is treated with considerable respect, but I still get a bit nervous when a Canadian author from Nova Scotia with an academic background in European medieval studies writes a story focused this deeply on a non-European culture. Nothing about her portrayal seemed off to me (although there is a very clunky and ham-handed scene about a different native culture that worries me), and for all I know she has family background or other connections to the culture she is borrowing from, but it's possible I missed serious problems.

The flip side of that caution is that I'm delighted to see a fantasy author drawing on a non-European culture, and I thought the clash of cultures was very well-handled.

The first caveat is that the story is very focused on good governance, but both the process and the details of that governance are not going to satisfy someone reading primarily for the politics. The policies and reforms are very standard 21st century progressive material that felt a bit out of place in a quasi-medieval world with magic and airships. Their implementation is not the point of the story, and is therefore heavily backgrounded, but that means Goddard barely mentions the inevitable practical implementation difficulties and does not discuss how they're overcome.

The world structure also means that Goddard can make use of the favorite cheat of political reformers in fiction: Absolute monarchy lets you enact a political agenda without having to do the hard and frustrating work of persuasion or political (or actual) warfare. This objection is not entirely fair because we do get some memorable scenes of persuasion, but the political portion of the plot is unrealistically devoid of setbacks or resistance that goes beyond token arguments.

Whether this will bother you will depend heavily on what parts of the book you'd rather focus on. I can see why this was such a popular pandemic read: The Hands of the Emperor is focused tightly on the joy of competent people fixing things and does not focus on the arguments, division, or polarization. The heart of the book is the friendship and characterization of some deeply admirable people, and the political reform is incidental background material. I suspect this is the right choice for readers who aren't political junkies, but I kept having the niggling objection that the politics felt a bit too pat and simplistic. Goddard stressed that the characters were investing considerable effort, but even still, it is not this easy to change the direction of a political system and idealistic plans usually do not work out this neatly.

The second caveat is that, as previously mentioned, I thought the pacing was excellent for about three quarters of the book. Goddard is building towards a grand climax, and I think she built a little too much and tried to make the climax a bit too grand and risked over-egging the pudding. That made the payoff feel a bit belabored to me. I still enjoyed it, and parts of it are wonderfully emotional, but I think the ending might have been stronger if Goddard had dialed Cliopher back just a little and tightened up the climax a touch. That said, this book fully commits to being a sprawling slow burn and that's part of its appeal, so it's probably better for Goddard to err in that direction than it would have been to cut short the denouement.

This is one of those books that I'm not sure would exist without self-publishing. It's a little too long, a little too political in the wrong ways, a little too devoid of the typical sorts of conflicts expected in a fantasy book, and too determined to be its own peculiar thing. I think it would scare off publishers. Unlike some self-published books, though, I didn't notice any obvious editing flaws or lack of polish. It's one of those glorious novels that is so very much its own type of story that it provides an experience that would be hard to replicate with another book.

I was so deeply satisfied by this book. It's a wish-fulfillment political fantasy full of diligent restraint and competence porn, so you have to be in the mood for that. This is not the book to read when you're feeling cynical, or are in the mood for action and high drama. But if you're in the mood for a long, slow, open-hearted story of friendship that offers the fantasy of giving truly good people enough power to be effective, I highly recommend this one.

Followed in the direct sequel sense by At the Feet of the Sun, but there is a very complex story progression in this world that I think I'd have to read all the other books to understand. This was such a satisfying and complete experience that I'm not in a hurry to figure out which Goddard book to read next, but I'm sure I'll be returning to this world at some point.

Rating: 9 out of 10

Ingo Juergensmann: Debian GR on Responsible Use of Generative AI

Sat, 29 Aug 2026 07:40:00 +0000

So, the members of Debian (developers and such) have voted in a GR about “LLM usage in Debian” for Option 5)/Proposal E) “Responsible Use of Generative AI”.

And in the #Fediverse the hell breaks lose!

Currently there is a huge vibe against (Generative) AI, i.e. generating code, images, etc. I can understand some furor about this and in general I dislike much of the AI as well. For example I dislike the energy and water consumption, what the hype did to memory & storage prices, the awful look of all these AI generated images, the ignorance of AI companies about copyrights and all this.

But I also see the benefits of AI: people are entitled to do things they couldn’t do before. You can proceed faster with your pet project when AI assist you and all of the bells and whistles.

Anyway, let’s face it: AI is a disruptive technolgy like the invention of electricity, the steam engine, combustion engine, the computer or the smartphone were. It will not go away anymore. Pandoras box is open.

But what you can do is:

learn what AI is and what not
use AI in a responsible way
experience the Do’s & Don’ts of AI
increase your knowledge about AI and its dangers and beneftis
help others with getting used resonsibly to AI
build a good sense that AI is telling you nonsense
don’t blindly trust AI
learn about new technology with and without the help of AI
use it as a tool
etc…

I don’t say you should embrace AI completely, but demonizing it won’t help you either. Be sceptic, be curious about it, act responsible. AI is not your saviour, but it’s also not armageddon. It is a discruptive technology. Deal with it. You’ll have no other option anyway. ðŸ¤·â€�â™‚ï¸�

Joey Hess: Debian and the sirens

Sat, 29 Aug 2026 00:27:55 +0000

Thirty years ago I became a Debian developer. Twelve years ago I left the project. I left because it seemed that the Debian ship had become too slow to turn, too barnacled with a series of individually OK decisions that each added a little bit of friction and a little less flexability. That made Debian strongly what it is, but prevented it from fruitfully exploring the vast possibility space of what it could be.

Debian will probably resolve today to allow LLM use in Debian development. I'm writing before the vote results are in, but will only post this afterwards. (Update: as expected) It's not my place any longer to try to steer the ship. But I'm still a passenger and I still have opinions, and I still pass by well-worn parts of the rigging that I put up decades ago, and remember what I was trying to accomplish back then.

When I think about LLMs in Debian development, I mostly think about debhelper and what it accomplished. The debian/rules files back when I joined the project were long and complex, full of weird boilerplate, and often you'd copy one and modify it to try to get something that could build a package without too much work. Debhelper first regularized the boilerplate, so packages had rules files that were a succession of dh_ commands, and then it scapped almost all of the boilerplate, reducing the files to the minimum possible. What was left was 3 lines of unncessary boilerplate, there only to satisfy a legalistic reading of a policy document. Changing that to eliminate the boilerplate was already impossible, even though the actual benefit would have been large over the many thousands of packages in the distribution.

What LLMs in Debian development will do, I fear, is eliminate any incentive to scrap boilerplate or reform policies that require a lot of other senseless human effort. If I had had access to LLMs 30 years ago, I might have just had them generate the rules files, replate with complexity. So they will make Debian even more firmly what it is, and ever less likely to explore what it could become.

Unfortunately, one of the things that Debian is, is almost unable to manage packaging modern dependency trees. While more recent distributions like Guix can recursively import dependencies from a dozen programming languages' package repositories, with a result that is generally acceptable to add to the distribution, Debian's policies don't make that very possible for a progam to accomplish. Perhaps some will use LLMs to do that. If they succeeed, Debian will become dependent on proprietary software for development, while still needing people in the loop, doing even less appealing scut-work.

I could speak of other harms, but that alone is enough that I'm sure that, if I had not left the project twelve years ago, I would be leaving it soon. As a passenger, I imagine I'll spend time aboard still from time to time, but it's certainly time to hop off in different places and look around and relish the different ways.

I lost a parent yesterday, and I'm trying hard not to think of the results today as having lost a child, though I spent 18 years helping Debian grow up. That would be too unbearably painful. I respect that Debian is navigating a choice that may have no right answer. Whichever particular compromise is arrived at today, it will still be up to individuals to make choices about what they do and accept. Debian has always been more than the sum of its policies, not just a ship, but a crew. I will always love you.

Antoine Beaupré: A more nuanced view of LLMs

Thu, 27 Aug 2026 14:10:43 +0000

Also in this series:

The Four Horsemen of the LLM Apocalypse

The people vs the AI overlords

After ranting and railing about LLMs or "AI" as the optimists (or accelerationists?) call it, I figured it might be important to be a little more honest about my use of LLMs and how I think about it more practically in the world.

The Debian vote context

This is not a coming out. I am not using LLMs on a daily basis, and this blog is, again, written out of my cold dead hands in a dying world, with over-engineered hardware and (to a certain extent, hi Emacs!) software, powered by 100% green energy built on stolen land.

There is a vote going on in Debian. If you're unfamiliar with it, you can catch up at LWN. So far I've essentially said "LLM is bad" which is not a very balanced or useful opinion. Obviously, people are using LLMs, sometimes unknowing or unwillingly, and we need to take that into account. Furthermore, there has been many different blog posts on Debian planet about this. Some that I found balanced, good summaries, even if I didn't fully agree with them, at least some did the basic civil service of being short. But others were just not only Wrong but also so long that I couldn't finish that I just had to write something.¹

This is not an explanation of the ballots, nor how I will vote. This vote is Debian's failure of framing that debate in a reasonable way: we have 8 options on the ballot with many duplicates. We have failed to do the hard work of summarizing and aggregating options into a meaningful set. I doubt the final vote will represent a readable position we can rally around.

I have not read the two months of debates on the topic either. Normally, before voting, I take a cursory look at the debate to see points of view I might have missed. But in this case, it will just make me sad, add noise, and I'm already pretty sure on where I stand on this.

So let me describe how I use LLMs and how I think they fit in our work, as computer engineers and hobbyists.

My LLM use

Debian Packaging

An astute reader has pointed out that I maintain a package in Debian made to use Anthropic. It's actually multiple packages:

llm: a CLI utility and Python library for interacting with Large Language Models, with OpenAI as its default API backend
llm-anthropic: a plugin for llm which allows me to talk to Anthropic's API instead of OpenAI
anthropic-sdk-python: the SDK llm-anthropic requires to do its work

As I previously explained in response, I am not entirely comfortable with this work: it's a compromise. In fact, I first uploaded llm to the contrib section of Debian, where we keep software that depends on other non-free software, but I was told that, since yt-dlp was in main, llm belonged there as well.

So I moved it to main, alongside similarly controversial tools like llama.cpp or the python-openai library.

OpenAI and Anthropic usage

An important part of my work is technology watch. I keep tabs on thousands of (new and old) software projects, follow news, and generally try to keep my skills up to date. It's a pretty impossible race, especially as I grow older, but I still think I'm doing the right choices in my job.

Testing large language models is part of that work. At first, I was using ChatGPT's web interface, but it was annoying to copy-paste things into a browser, so I looked for different interfaces.

For a while I tried gptel, a "simple, extensible LLM client for Emacs" but I found it kind of terrifying. Giving a LLM control over an Emacs buffer seems like a security nightmare, so I stopped doing that.

So I use the llm command-line tool to talk to Anthropic's API. I started that in the summer of 2025, when I bought 20$USD of API credits. Before that, I paid for a ChatGPT subscription and then OpenAI credits, which expired and sent me over to Anthropic, which seemed then to have better ethics.

As it turns out, Anthropic is also happy to work for the US military (which is a big red line for me). Anthropic also won't let you talk about the genocide in Gaza, it is destroying physical books, and is blackmailing us to use their product for security coverage.

Needless to say, Anthropic and "Claude" are not my friends, but they seem like the lesser evil in current "frontier models". So I have renewed, a couple of weeks ago, another 20$USD of API credits with Anthropic.

Actual prompts and responses

So what does 20$ give you at Anthropic anyways? What am I using LLMs for and how?

The neat thing with llm is that everything is logged in a sqlite database, so there are some answers that are easy to get:

> llm logs status
Logging is ON for all prompts
Found log database at /home/anarcat/.config/io.datasette.llm/logs.db
Number of threads logged:   7
Number of turns logged:     12
Number of legacy conversations: 543
Number of legacy responses: 970
Database file size:         9.61MB

That is 10MB of logs, with about a thousand prompts.

My logs go back to 2024-03-07, a little over two years ago, and include a mix of Anthropic and OpenAI responses. I used it more in 2024 than 2025, and if the trend continues, I will have used it less in 2026 again:

> llm logs list -n 0  --json | jq -r .[].datetime_utc | sed 's/-.*//' | sort | uniq -c 
    527 2024
    357 2025
     98 2026

It looks like about 10 prompts per month right now, down from a peak of about 60 per month in 2024. It's pretty difficult to analyze those actual logs to get more patterns and I won't run the prompts through a model again to process them.

How I'm using models now

At first, I was using it partly for benchmarking model's capabilities, like Simon Willison does with his pelicans, clearly not trusting its output. But I was impressed by the capacities of the Claude Opus 4.5 model when it wrote this script in January. Impressed, but also scared: it's the first time I felt I could delegate the entirety of my programming to a model. Just run the code, if it works, it works, right?

So what do I use it now? As an example, here are the 10 last prompts in my history:

there is now Claude 5, and a fable model, maybe you know about it?
impress me
not impressive, i already know all of this
chat
in postfix, i have a 300k mailing that happens regularly here. normally, it delivers within about...
is there a way i could have drained the maildrop queue faster without removing the milter?
the problem was that rspamd was timing out on the FUZZY_CALLBACK check. how do i disable that?
how do i disable all spam checks? i just want rspamd to add dkim signatures
how do the default_destination_concurrency_limit and initial_destination_concurrency settings int...
mic check

The first one was me trying to confirm which model I am using, which is not always obvious when going through the whole llm stack I've been using. The following two are an attempt at seeing what the model is capable of and I was "not impressed", to which Claude answered that I have a "high bar", which, fair enough.

The chat is me failing to use a command line, which shows that perhaps I need to readjust that "high bar", again.

The next five are a rather embarrassing debacle in a large Postfix mailing that went sideways, and where I couldn't find an actual Postfix expert of my level to help. The fabled Claude Fable 5 answered rather correctly, but dangerously, that I could empty the queue by disabling the non_smtpd_milters. What Fable (and myself) did not realize is that the milter was also adding DKIM signatures, so while the mailing was expedited, it was done without those precious signatures, which got us promptly blocked at Gmail. We have recovered since, and, thanks to the model and reading the Postfix manual for the hundredth time, that pickup(8) is single-threaded and that we needed to review the architecture of that mailing (and our spam filters) a bit. Many tickets ensued.

The last one is a test I did to make sure my last uploads of llm-anthropic and its dependency worked correctly.

Note that the above excludes 5 questions I asked Anthropic while writing this article, where I asked for synonyms and "what nanometer scale are arduino processors built from? how is an arduino CPU printed?", a question which Wikipedia furiously evades providing a good answer.

Those prompts are pretty typical of my LLM use: I'm testing the models to see if they work at all, but also, out of desperation, I fire off a prompt after I fire off questions to colleagues or search engines (in that order). It's often weird edge cases like the Prometheus query language, Python's matplotlib, LaTeX, Elisp, optimizations, and so on.

I use models for translation a lot. Being fully bilingual, it is common for me to think of a word in French or English and fail to find exactly the right word for that in the other language. Models help with that, and are also useful to find synonyms. Those are low-token uses that seem pretty innocuous to me, but I realize the irony of this after writing about the tower of Babel.

What I am not using models for

I am not using models to write prose.

I am not using models to read prose. If it's generated with LLMs, I stop reading.

I am not using models to write code, with the exception of that single Python script above.

I am generally not using models to review code, with exceptions. If I get stuck on a hard problem, I might feed a piece of code to the model. I repeatedly fed asncounter into Claude to try to fix a performance regression I had introduced. It found micro-optimizations that taught me a thing or two about Python's internal implementations, but overall, it was mostly a waste of time. This was in June 2025, so perhaps now models would fare better. I have not tried again.

I am not using LLMs to do Debian packaging. When I can, I manually review the diffs of packages I upload into Debian, still, by hand.

I do this for the reasons outlined in The Four Horsemen of the LLM Apocalypse, because I refuse to be complicit in the:

aggressive and illegal scraping of the servers I steward
world-wide computer hardware shortage (making it, by the way, nearly impossible to run presumably clean local models) and the attack on our job conditions (also discussed in The people vs the AI overlords)
death of copyright and free software
complication and enshifitication of everything, and the destruction of our communities
the imperialist Nerd Reich that wants to take over the world

Like I reluctantly use Intel computers, I do fire off a prompt. But I still hold on to the dream that we can build communities of practice that hold human knowledge collectively and not offload that as a utility to some megalomaniac billionaire.

Their LLM use I am forced into

So that's me. Clearly, I'm going against the grain here. Everywhere I look, I see LLM-generated code and projects. Slop and botnets have flooded the web.

I use Wadamesh, clearly vibe-coded, because it's the best graphical interface for MeshCore that runs on portable devices. I wish it was made by a human, in a community I could participate in, but it isn't, and I don't.

I package the above llm toolset, which is more and more vibe-coded, but I still review the diffs. And I have to say: I trust Simon here. The code is verbose as hell, feels overengineered, and llm feels slow, but it generally works, and Simon is still at the gate.

The Anthropic SDK is another thing entirely. The 0.91.0 to 0.120 upload, for example, was nuts:

 806 files changed, 72281 insertions(+), 1478 deletions(-)

I explicitly did not review that entire diff. It feels like there's a lot of garbage there to just have a shim between a proprietary API and Python. But this is the hand I've been dealt.

Larger projects LLM use

LLMs are being used in the Linux kernel, Firefox, rsync, Rust, and other places. I don't feel good about this, particularly in Rust, but they at least made a decent policy. I am glad GCC made a policy against LLM contributions and I support the human Emacs project.

We need to have a set of foundational tools that are "clean" in the sense that they are built upon a community of people that understand how they are built.

Maybe that's naive or even impossible. The Linux kernel and GCC, in particular, are massive projects that have long grown past the scale of a single person's understanding. But the theory was that a community of humans can understand collectively.

Now we seem to be throwing up our hands and giving up on that community. That LLMs will just fix the problem, whatever it is. But we're all just one rug pull away from being completely incapable of managing those projects. The argument there is that we'll just switch to local models, but no one is actually doing that. All I see is people use local models as a corner case (for privacy) or as in theory, but in reality, everyone uses the centralized frontier models right now. We just can't fallback.

We're in the same situation we were, a decade or two ago, when Microsoft decided it would kill free office alternatives by making Office free for non-profits. It worked: thousands, if not millions of schools, community groups and individuals stopped looking for alternatives (including free software but also "piracy") for Office and embraced what seemed like a generous offer.

Now Microsoft pulled the plug and Over 170,000 Nonprofits Lost All Their Data.

I'm afraid the rug pull on LLMs will be much worse: never mind that Linus won't be able to use his tireless helper to fix obscure kernel bugs; we're looking at a collapse of the economy so large that we are already talking about bailing out the companies responsible.

In a sense, the most striking thing about the Debian vote is it has actually no option to completely refuse upstream LLM contributions. It seems the community has taken it for granted that it's now impossible to build Debian entirely without LLMs. We lost the battle even without a fight, it seems.

A plea for small

If it has really become impossible for us to manage the complexity we have built, maybe it's time to stop and think about what we're doing in the first place. We're struggling to even bootstrap our current toolchain!

This is one of the things I like the most about working on the mesh: it's low tech, small Arduino devices that is built with decades-old semiconductor processes that is understandable by human beings.

Maybe the answer lies more in single-purpose devices like those communicators and simpler multi-purpose computers than what we have now, which is what the permacomputing movement is about.

Small is beautiful, let's scale it down.

and yes, I'm sorry this has gotten this long, I hope you will forgive those 3000 words.↩

Gunnar Wolf: As far as LLMs go in Debian, I think that 936241857

Thu, 27 Aug 2026 00:48:32 +0000

I believe that, in the context of Debian voting, we are better off when we know the opinion of our peers, however, since the 2022-001 vote, it is no longer the case. Still, some DDs have disclosed the way they are voting on the 2026-002 General Resolution currently in progress, regarding LLM usage in Debian. So, here goes my vote and reasoning as briefly as possible. This is the ballot I sent to devotee, the Debian Vote Engine:

-=-=-=-=-=- Don't Delete Anything Between These Lines =-=-=-=-=-=-=-=-
d69f9187-ed2f-40b6-a2eb-4211d3f84d86
[9] Choice 1: Ban LLM contributions from Debian via Social Contract
[3] Choice 2: Allow AI-Assisted Contributions with conditions
[6] Choice 3: Reject LLMs as far as practical, update Code of Conduct
[2] Choice 4: Accept AI contributions for Debian specific work
[4] Choice 5: Responsible Use of Generative AI
[1] Choice 6: A cautious approach to generative AI
[8] Choice 7: Debian is created by humans
[5] Choice 8: Avoid the use of LLM: climate destruction is a deal breaker
[7] Choice 9: None of the above
-=-=-=-=-=- Don't Delete Anything Between These Lines =-=-=-=-=-=-=-=-

This is the first time I can recall I delay my voting until after receiving the final call for votes (the vote will be over two days from now). I had some participation in the discussion, so I guess my position will be of no big surprise to anybody. I was also a seconder for choices D and F (4 and 6 in the vote text). This does not necessarily mean I believe they are the best (although I did rank them as 2 and 1, meaning I do): sometimes you agree a given text needs to be in the ballot, and second it even though you don’t intend to vote for it.

Ranking this ballot was a mess due to the complex array of options it encodes. I warmly thank Lucas Nussbaum for coming up with the LLM usage in Debian: ballot option comparison (URL shown with my particular ballot ordering).

How do you read a complex Debian ballot like this one? I rank with [1] my favorite option, [2] for the next one, etc. We can encode options to be tied (i.e. setting more than options to the same value), and we can implicitly push options to the worst position by leaving them blank (so, with[ ]); I chose not to do any of those.

What were my voting guidelines?

First, I don’t want anything banning or that threatens with disciplinary action, so I push them below the special none of the above marker. Second… Some time ago I published a review in my blog (and in Computing Reviews) about the unfeasibility and unfairness of detecting LLM output on students’ assignments. I strongly believe we ought to appeal to the human responsibility and professionalism in all Debian contributors. This is the reason I proposed this amendment paragraph, that was accepted in choice F (6), which I ranked as my favorite:

The Debian project has always recognized the commitment and
professionalism of its members. All contributions are under the
responsibility of the Debian Contributor making it, no matter the
technology they have behind. We trust all Debian Developers,
Maintainers and Contributors will continue to uphold the high quality
values that have distinguished our project from its onset.

Other than that… I do not consider myself to be in any way an LLM fanboy nor anything like that. I distrust and dislike the excessive use of this technology, and continue to warn about the dangers and bad points of its abuse. But in my day-to-day professional work, I am also starting to rely on it for some tasks. I recognize it needs a lot of human oversight and… lets call it hand-holding to produce anything worth it, at least in my experience. But I do benefit from it — and always disclose its use to people who might be affected by it. I would like Debian to adopt such a stance.

Of course, I recgnize proposal H/8 as important (Avoid the use of LLM: climate destruction is a deal breaker). Some people have argued it’s not bad at all. I do not buy such claims: LLMs are f*cking expensive to train. But training can be seen as a once-per-model cost, and fine-tuning a good model to be run locally can be really worth it. It still pains me somewhat, but I cannot push this option higher than its #5 position in my list.

Ritesh Raj Sarraf: Taming the AI Agents (Part 2): Cross-Vendor Agent-to-Agent (A2A) Swarms over the Software Forge

rrs@researchut.com (Ritesh Raj Sarraf) — Thu, 27 Aug 2026 00:00:00 +0000

Preface: The Unanswered Frontier

In Part 1: Taming the AI Agents, I shared the architectural blueprint of CAMP (Cross-Agent Memory Protocol)—how we used Linux Bubblewrap (bwrap), camp-acpd, OPA policy enforcement, and a central pgvector MemPalace to bring deterministic discipline, sandboxing, and long-term memory to a heterogeneous fleet of AI coding assistants (Claude Code, Google Antigravity, Grok Build, and GitHub Copilot).

At the end of that article, however, I highlighted a significant hurdle: The Headless Limitation.

“While passive A2A works beautifully for structured handoffs, the current frontier of agentic design faces a key limitation: agents are not yet fully headless-capable. They depend on the active terminal session, browser loop, or prompt loop of the user to keep executing. Because agents cannot run completely detached in the background as daemon processes, we cannot yet achieve active A2A communication…”

For weeks, this seemed like an insurmountable impasse. Proprietary AI vendors have zero commercial incentive to ratify a universal, open, cross-vendor Agent-to-Agent (A2A) communication protocol. Each vendor builds its own walled garden (Claude’s cross-session features, OpenAI’s custom ecosystems, etc.). If you wait for the industry to hand you an open interoperability standard, you will wait forever.

Then, on August 26, 2026, inspired by Colin Walters’ article on Agentic AI and software forges and GitHub Agentic Workflows (gh-aw), we had a sudden realization:

We don’t need a new protocol, a new distributed message broker, or permission from proprietary AI vendors. We already have the universal, decentralized communication bus that software engineers have relied on for decades: the software forge itself.

Over the span of 48 intensive hours (from RFC #788 through milestones M1 to M3 and live dogfooding on #813), we designed, implemented, fortified, and verified fully autonomous, headless, cross-vendor Agent-to-Agent swarms running over a local Gitea forge.

Here is how we did it, the architectural hurdles we solved, and why this changes the game for autonomous software engineering.

1. The Core Realization: The Forge is the Bus

When people think about multi-agent swarms, they often imagine complex distributed RPC frameworks, microservices exchanging ephemeral JSON-RPC blobs, or bespoke socket daemons.

In practice, this approach suffers from major flaws:

No shared context or durable audit trail: Transient network packets vanish unless heavily logged.
Proprietary CLI fragmentation: Different vendor tools (Claude CLI, Antigravity CLI, Grok CLI, Copilot CLI) do not speak the same internal language.
Loss of human visibility: When agents talk over private network channels, human operators lose the ability to inspect, pause, or audit the conversation.

By flipping the paradigm and making the software forge (Gitea) the primary communication channel, everything falls naturally into place:

Issues and Pull Requests are the shared state: The issue description and discussion thread form the canonical, append-only conversation log.
@mentions are the dispatch triggers: When an agent (or human) writes @grok Please review this PR in a comment, Gitea fires a standard webhook (issue_comment).
Webhooks provide unforgeable authentication: The webhook payload contains the cryptographically verified sender identity. An agent cannot spoof another agent’s identity by merely typing their name in text.
Every CLI already supports non-interactive prompt mode: The CLIs don’t even agree on the command-line flag—Claude uses -p, Grok uses -p, Antigravity uses --print, Copilot uses --prompt. But they all agree on the essential contract: “Take a prompt string, execute tools, print output, and exit.”

┌──────────────┐         Gitea Webhook          ┌──────────────────────┐
│ Gitea Forge  │ ─────────────────────────────> │ camp-a2a-bridge.py   │
│ (localhost)  │  (issue_comment / assignment)  │ (Validates & Files)  │
└──────────────┘                                └──────────┬───────────┘
       ▲                                                   │
       │                                                   ▼
       │ Writes comment / review                ┌──────────────────────┐
       │ via camp_acp_gateway                   │ A2A Inbox Ledger     │
       │                                        └──────────┬───────────┘
┌──────┴──────────────────────┐                            │
│ Fortified Headless Agent    │                            ▼
│ (bwrap + OPA + MCP sandbox) │ <───────────────── ┌──────────────────────┐
│  • Claude Code (-p)         │  Spawn PID         │ camp-a2a-dispatcher  │
│  • Grok Build (-p)          │  (Cold or Resume)  │ (Enforces Hop Cap,   │
│  • Antigravity (--print)    │                    │  Rule 1/2, Sandbox)  │
└─────────────────────────────┘                    └──────────────────────┘

2. Proving Fortified Headless Execution

Before opening the floodgates to background agent dispatch, we had to answer a critical security question: Does a non-interactive, headless agent run with the same strict security sandboxing, audit logging, and tool rails as an interactive session?

On August 26, we probed all fleet launchers on the host with a baseline check: 'Call camp_startup_check and print its result verbatim, then exit.'

The results settled the question immediately:

Antigravity (agy --print / KIR): PASS — Gateway answered, full JSON returned.
Grok (grok -p / GRK): PASS — Gateway answered.
Claude Code (claude -p / CLD): PASS — Gateway answered.
GitHub Copilot CLI (copilot --prompt / CPL): Initially held on TTY tool consent; later unlocked in Milestone 6 via --allow-all-tools --session-id=<uuid>.
Audit Trail: Consecutive audit IDs were recorded in the central ledger: 4574 (KIR), 4575 (GRK), 4576 (CLD).

This proved that a headless run through our fortified pilot launcher (camp_pilot_*.sh) is a first-class, fully audited, sandboxed CAMP agent running inside its Bubblewrap container under OPA policy gates. It is not an unconstrained background script or a degraded bypass.

3. The 3-Tier Memory Architecture

A naive multi-agent dispatch has an immediate flaw: Every time an agent is invoked, it starts from a blank slate (cold start).

If @claude tags @grok to review code, and @grok replies asking for clarification, @claude’s second invocation would normally forget everything it did 5 minutes ago, forcing it to burn thousands of tokens re-reading the entire git history from scratch.

To solve this, we established a clean 3-Tier Memory Model:

┌────────────────────────────────────────────────────────────────────────┐
│                        3-TIER MEMORY MODEL                             │
├────────────────────────────────────────────────────────────────────────┤
│ Tier 1: CLI Conversation Session (Working Memory)                      │
│   • Per-(Agent, Repo, Issue) mapping in a2a-sessions.json              │
│   • Fast, native, compacted context across multi-turn pokes            │
│   • Resumed via --resume (CLD), -r (GRK), --conversation (agy)         │
├────────────────────────────────────────────────────────────────────────┤
│ Tier 2: The Gitea Thread (Public Bus & Record)                         │
│   • Cross-vendor shared truth across Claude, Grok, Antigravity & Human │
│   • Survives process restarts, machine reboots, and dead sessions      │
├────────────────────────────────────────────────────────────────────────┤
│ Tier 3: Central MemPalace (Durable Long-Term Knowledge)                │
│   • pgvector database (17,000+ drawers across agent wings)             │
│   • Structured Knowledge Graph (mempalace_kg_*) for mutable facts      │
│   • Attributed AAAK dialect queryable by any agent across any project  │
└────────────────────────────────────────────────────────────────────────┘

The `BANANA` Two-Shot Test

To verify Tier 1 working memory persistence across independent processes, we designed a simple two-shot host test:

Shot 1 (Create): Dispatch agent headlessly: “Remember the token BANANA-M2. Print ok and exit.” Capture the vendor’s session UUID.
Shot 2 (Resume): Spawn a completely new operating system process with the resume flag pointing to that UUID: “What token did I ask you to remember?”

Every agent CLI passed with flying colors:

Grok: -r 01a03ecc-3ed0-71e1-9a5c-e098bb29ba10 answered BANANA-GRK.
Claude: --resume 0a587733-9aec-43c5-9cb7-d424e95b2c5b answered BANANA-CLD.
Antigravity: --conversation 2e3c43d9-d6fe-4c5c-801b-b9ceb2e7e196 answered BANANA-KIR-JSON.
Copilot: --session-id <uuid> verified in Milestone 6 (DoD #820).

The dispatcher simply maintains a lightweight JSON mapping ((agent, repo, issue_number) -> vendor_session_uuid). On the first poke of an issue, it creates and saves the session ID; on any subsequent poke on that same issue, it resumes the exact same conversational thread!

4. The Engineering Milestones: From Concept to Production

Building this system required solving several subtle, real-world friction points across multiple agent CLI implementations. Under the guidance of our plan of record (RFC #788), we delivered this through four focused milestones:

Milestone 1 & 1.1: Reliable Headless Spawning

PR #797 (M1): Configured the dispatcher launch table for all probed CLIs with JSON output formatting.
PR #800 (M1.1): Eliminated the “queue-behind-live-session” anti-pattern. Originally, if a human had a Claude or Grok TUI open on their desktop, the dispatcher would defer incoming tasks so as not to collide with the live session. We realized that headless tasks must be independent: every Gitea mention spawns an isolated, sandboxed background process tied to that specific issue, allowing concurrent headless work while the human works in their interactive TUI.
PR #803 (M1.2): Standardized command-line argument parsing for Antigravity (agy --print <prompt> --output-format json).

Milestone 2: Session-per-Issue Working Memory

PR #805 (M2): Implemented a2a-sessions.json to store and resume vendor session UUIDs. If a resume fails (e.g. session purged upstream), the dispatcher gracefully falls back to a clean cold start without failing the task.

Milestone 3: Cross-Agent Hops & Crucial Safety Rails

PR #807 (M3): Enabled agent-to-agent dispatch (Rule 2 reversal). Previously, only mentions authored by rrs (the human) would trigger execution. With M3, an authenticated comment from @claude mentioning @grok triggers Grok’s headless launcher.
PR #811 (M3.1): Set --permission-mode bypassPermissions for headless Claude Code so non-interactive runs execute tool calls without stalling on TTY prompts.
PR #812 (M3.2): Restricted agent summon parsing to line-initial @login tokens with a non-empty task description (#810), preventing accidental dispatches from passive conversational references.

Milestone 4: Directives, Specification & Living Documentation

PR #815 (M4): Aligned CAMP fleet directives, architecture specifications, and user documentation with the live A2A implementation.

Milestone 5: Concurrent Dispatching & Hop-Cap Attribution

PR #816: Stamped hop-cap notices under a dedicated system bridge identity and automatically applied the needs-human label on held threads.
PR #817 (Threaded Scheduler): Replaced the single-threaded serial dispatcher with a concurrent thread-pool scheduler (#804). Multi-agent dispatches across different issues now execute concurrently in parallel background threads instead of queuing behind long-running tasks.

Milestone 6: Full Fleet Coverage with GitHub Copilot

PR #819 (M6): Brought GitHub Copilot CLI into the headless A2A fleet (#818). By passing --allow-all-tools and pinning minted session UUIDs (--session-id=<uuid>), Copilot achieved full parity with Claude, Grok, and Antigravity, completing 100% headless fleet coverage across all four major AI coding assistants.

5. Hard Safety Rails: Preventing Autonomous Runaway Loops

Letting AI agents autonomously invoke each other in background loops without a human watching is a recipe for an infinite, credit-draining token fire. We put four non-negotiable safety guardrails in place:

Guardrail 1: The Strict Hop Cap

The dispatcher tracks hops per (repo, issue). Each agent-to-agent dispatch increments the counter.

Hop Limit = 3: A typical review round-trip is 2 hops (Human $\rightarrow$ Claude $\rightarrow$ Grok $\rightarrow$ Claude).
Automatic Halt on Hop 4: If agents attempt a 4th autonomous hop without human participation, the bridge refuses to launch, posts a diagnostic notice to the thread: [camp-a2a-bridge] hop cap reached (3 agent-to-agent dispatches on CAMP/camp-infrastructure#813) — not launching GRK for claude's mention, and holds execution until the human (rrs) provides input or resets the count.

[ Human: rrs ] ────── (Cold Start) ─────> [ @Claude ]
                                               │
                                       (Hop 1) │ @grok please review
                                               ▼
                                          [ @Grok ]
                                               │
                       (Hop 2: Resume)         │ @claude I reviewed
                                               ▼
                                         [ @Claude ]
                                               │
                                       (Hop 3) │ @grok ack hop 4
                                               ▼
                                  ┌─────────────────────────┐
                                  │  DISPATCHER HOP CAP: 3  │
                                  │   *** BLOCKED & HELD ***│
                                  │   Awaiting Human Reset  │
                                  └─────────────────────────┘

Guardrail 2: Deliberate Summon Parsing (M3.2, #810 / PR #812)

In human conversation, we often reference colleagues in passing: “I will talk to @claude about this later” or “See @grok’s table above”. Early prototypes treated any appearance of @agent as a dispatch trigger, causing accidental, unwanted agent launches!

We instituted a strict Summon Predicate: For fleet agents, a mention is only considered an actionable summon if:

The @login appears as the starting word of a line (optionally preceded by markdown list markers *, -, or >).
It is immediately followed by whitespace and a non-empty task description.

Mid-sentence mentions in discussion paragraphs are parsed as passive conversational text and never trigger background dispatches.

Guardrail 3: Headless Tool Permissions without Weakening Security (M3.1, #809 / PR #811)

In interactive mode, Claude Code presents interactive TTY prompts asking the user to approve MCP tool calls (such as camp_pr_get or camp_pr_get_diff). In unattended headless mode, there is no TTY, causing the run to fail with permission errors.

To fix this, we configured --permission-mode bypassPermissions for Claude’s headless CLI invocation. Crucially, this only bypasses Claude’s internal TTY UI prompt—it does not bypass CAMP’s security rails.

All command executions still route through camp-acpd and Bubblewrap namespaces; OPA policy checks remain active; and privileged operations (such as merging pull requests or restarting system services) still trigger desktop Zenity HITL dialogs on the human’s workstation.

Guardrail 4: The Sovereign Kill Switch (Rule 1)

At any point, the human operator can instantly freeze all background agent dispatches across the entire infrastructure with a single standard Linux command:

systemctl --user stop camp-a2a-active.service

6. Live Dogfooding: Watching the Swarm in Action

On August 27, 2026, on issue CAMP/camp-infrastructure#813, we ran the full live dogfood test of the A2A system.

Here is the exact transcript of what took place entirely autonomously in the Gitea issue thread:

Human initiates: rrs comments on issue #813:

@claude Please start the M3 dogfood. On this issue, post a conversation comment whose first line is exactly: @grok Please review this issue in one short paragraph, then stop.
Claude launches (Cold start): The dispatcher spins up @claude headlessly inside bwrap. Claude reads the issue and posts:

@grok Please review this issue in one short paragraph, then stop.
(M3 dogfood step 2 — posted by @claude on rrs’s instruction; this is the CLD $\rightarrow$ GRK fleet hop.)
Grok launches (Cold start, Hop 1): The bridge intercepts Claude’s comment, identifies the start-of-line @grok summon, and launches @grok. Grok reviews the issue and replies:

@claude I have reviewed this issue. Please confirm and stop.
#813 is a throwaway live dogfood of A2A M3 on 6.git+b78c65cf…
Claude resumes (Session Resume, Hop 2): The bridge intercepts Grok’s comment, sees the @claude summon, and launches Claude with --resume b398d048-ccc3-4cce-8437-4cf8748881a8. Claude immediately remembers its earlier context without token waste:

Confirmed — Grok's review (comment 88562) received via the GRK→CLD fleet hop. That completes the round trip: rrs→CLD, CLD→GRK, GRK→CLD, and this reply. I am deliberately not tagging anyone here, so the chain stops at two agent hops. Stopping.
Testing the Hop Cap (Hop 3 $\rightarrow$ 4): grok prompts @claude Please ack hop four then stop. Claude posts @grok Please ack hop four then stop. (Hop 3). When Grok would have been launched for Hop 4, the dispatcher intervenes:

[camp-a2a-bridge] hop cap reached (3 agent-to-agent dispatches on CAMP/camp-infrastructure#813) — not launching GRK for claude's mention.
Human Reset & Multi-Agent Wrap-up: rrs steps in, resets the state, and asks the fleet for final status. In parallel:
- @grok delivers a closure scorecard.
- @claude confirms session continuity and M3.2 summon filtering.
- @priyasi (Antigravity CLI) runs automated ACP checks: 44/44 test suite passing, 17,219 MemPalace vector drawers active, zero spec drift.
- @agrickxy (Antigravity CLI) provides comprehensive infrastructure impression analysis.
- @kiran (Antigravity CLI) is summoned headlessly to draft this very blog post!

7. The Ergonomic Breakthrough: The Forge as the Unified Mindmap & Interface

Beyond backend plumbing and sandboxing, routing agent interaction through Gitea fundamentally revolutionizes the developer experience of managing an AI fleet.

The “Mindmap” Mental Model: Threaded Conversations & Forking Tasks

In traditional CLI tools, conversations are constrained to a single, linear terminal scrollback. When an agent discovers multiple sub-problems, exploring them sequentially in one prompt loop rapidly pollutes the context window and confuses the model.

Using the forge as the communication gateway naturally unlocks a mindmap mental model:

Forking sub-threads: Complex problems can be split into dedicated child issues or threaded PR reviews.
Focused execution scopes: An agent can be summoned to solve a narrow sub-task in its own issue thread without derailing the parent architectural discussion.
Structured problem decomposition: The forge issue hierarchy maps 1:1 to the developer’s mental map of the project.

Eliminating Terminal UI Fragmentation

Anyone using multiple AI coding assistants on a daily basis quickly grows exhausted by their jarring terminal UI differences: differing ANSI escape rendering, inconsistent markdown wrapping, erratic diff pagers, and incompatible keybindings across Claude, Grok, and Antigravity.

Gitea homogenizes the entire fleet under a single, polished rich-text web view:

Syntax-highlighted code blocks and visual side-by-side git diffs.
Clear author badges attributing each contribution to its exact agent identity (@claude, @grok, @priyasi, @kiran).
Collapsible <details> blocks for voluminous diagnostic outputs.
Interactive task lists and markdown tables.

Effortless Context Retrieval, Archival & Data Retention

Auditing past agent decisions in terminal logs or ephemeral chat histories is notoriously difficult. With the forge, every exchange is:

Contextually bound: Pinned directly to the repository, branch, and commit SHA being modified.
Organized & Archival-Grade: Full-text searchable with clear milestone and issue tags.
Topic-Focused: The human operator can review the complete lifecycle of a discussion in seconds, gaining a rapid, holistic grasp on the entire subject.

Reading back through past agent interactions becomes a breeze—to the point where interacting via the intermediary Gitea interface becomes far more pleasant and productive than wrestling with multiple desktop CLI terminals.

Remote Connectivity & Headless Agent Farm Management

Because Gitea provides a standard web and API interface, you are no longer chained to the workstation running the agent processes:

Monitor progress and dispatch tasks from a mobile browser, tablet, or remote laptop.
Queue review tasks on the go without requiring active SSH sessions or terminal multiplexers.
The local agent farm continues working silently in its sandboxed daemon containers.

Quietly Achieving the Holy Grail: Live Cross-Vendor Swarms

For years, the AI industry has treated cross-vendor multi-agent interoperability as an elusive dream waiting for industry-wide API standardization. By recognizing the software forge as the universal message bus, we quietly achieved live, production-grade, cross-vendor communication across completely distinct vendor models.

8. What This Means for the Future of Agentic AI

This milestone marks a fundamental shift in how we interact with autonomous AI systems:

Heterogeneous Agent Specialization: We don’t have to choose a single “winner” among AI models. We can task Claude Code with architectural refactoring, summon Grok Build for rapid verification and adversarial PR reviews, and deploy Google Antigravity agents for codebase exploration and documentation drafting—all coordinating fluidly in the same PR thread.
True Human Sovereignty: The human developer is no longer a bottleneck typist or a passive spectator. You act as the Engineering Manager / Lead Architect. You set the requirements on an issue, tag the lead agent, and let the agents iterate, review, and test among themselves in the thread—while hard hop caps, OPA policies, and Zenity HITL gates guarantee that no agent merges code or pushes upstream without your explicit sign-off.
No Vendor Lock-In: Because the entire coordination fabric is built on standard Git, HTTP webhooks, local Linux container sandboxes (bwrap), and open MCP tools, any new AI CLI tool released tomorrow can be plugged into our fleet in under 15 minutes by simply adding its command-line prompt flag to the launch table.

We have moved beyond static autocomplete and interactive chat widgets. The software forge is now an active, living, collaborative workspace where humans and autonomous AI agents engineer software together.

9. Video Demonstration: CAMP Forge A2A Swarm in Action

Below is a video demonstration showcasing autonomous multi-agent communication, cross-vendor relay, and headless swarm coordination in action via the CAMP Forge interface:

The Cross-Agent Memory Protocol (CAMP) and MemPalace are developed as part of our ongoing research into secure, sovereign, and disciplined Agentic AI computing.