Jacob Garber

Bootstrapping a Highly Available Container Registry

2025-07-03T00:00:00+00:00

Deploying a bare-metal Kubernetes cluster is very much a build-your-own adventure: kubeadm will create a cluster for you, but fundamental building blocks like the load-balancer, persistent storage, and ingress you need to setup yourself. In addition, what if you are running in an air-gapped environment? No incoming or outgoing internet connections, at best you get to SSH in.^{1</a> Since you can't access DockerHub or Quay, you need to setup a registry somewhere locally. Furthermore, this registry needs to be highly-available, but since we are bootstrapping a cluster we can't take advantage of any of the HA machinery that Kubernetes gives us since it doesn't exist yet (sorry Harbor</a>). It took me a fair bit of head-scratching to figure out how to solve this.}

To set the stage, let's say you are deploying onto three nodes inside a private network, and are going to implement fail-over using a virtual IP managed at Layer 2 (ARP or NDP). Broadly speaking we can split the deployment into three parts:

A backend that stores the container image layers and replicates them amongst the other replicas.</li>

A frontend that serves client requests (e.g.</abbr> push/pull) and dispatches them to the backend.</li>

A virtual IP manager that checks the health of the previous two components and will move the VIP to a different node if they are not healthy.</li> </ul>
For the frontend I found two main options:

Distribution Registry</a>: This is the original container registry developed by Docker and has since been donated to the CNCF. It's fairly bare-bones but the de facto way of running a registry yourself.</li>
Zot Registry</a> is a bit of a new comer here and seems to have been developed by Cisco before being open sourced. Zot comes with many more features than the distribution registry, including a GUI. (Also, retention policies!</a> Never manually delete old images again!) One potential hiccup with this registry is that it strictly conforms to the OCI specifications, and so you will be unable to push container images to it using docker push</code>, since that uses the docker format. This is easy to fix though by using Podman or skopeo, which let you push images in the OCI format.</li> </ul> If it isn't clear already I like Zot quite a bit: it strikes a nice balance between features and simplicity, plus the GUI is nice too. The backend is the heart of the system and is the most difficult to setup. In general, replicated storage can be done at two levels:Block Storage. This operates at a very low level and replicates the individual blocks of your file system. Due to this generality, replicated block storage systems such Ceph</a> are often very complicated and resource intensive. Not the choice here.</li> Object Storage. This operates at a high level and replicates invididual "objects", which are just blobs of data. Most object stores use the S3 API, and fortunately Zot supports it as a backend. There are several good options here:2</a> Garage</a>. An interesting project, unlike many other systems it uses CRDTs</a> for replication instead of a consensus algorithm like Raft.</li> SeaweedFS</a>. I've heard excellent things about this and people have been able to run it with billions of files</a>. Definitely a good choice, however Garage seemed simpler to deploy so that's what I went with.</li> </ul> </li> </ul> Finally, managing the virtual IP. Keepalived is the simplest and traditional choice here, though (self-plug!) if you want more resilient leader election you could try kayak</a> too. All together then: </figure> For deployment, all of the above programs are static binaries and can be easily run using systemd. Another option could be Podman quadlets.3</a>4</a>5</a> There are more things you can add to this setup that would be useful in production (e.g.</abbr> a reverse proxy to load balance between the nodes and do TLS termination), but that's the fundamental idea. In practice it works pretty well! ^{1 Which itself is actually fairly liberating through the magic of port forwards and SOCKS proxies. </div> ^{2 MinIO is not one of them. </div> ^{3 If doing this, be careful about running Podman and Kubernetes on the same node. Podman will try to set up its own firewall rules and they might conflict with what Kubernetes or your CNI does. I think Podman doesn't create firewall rules if you run the containers with host network mode, so that might work. I know for sure this does not work with Docker. </div> ^{4 For the containerization purists out there, in theory you can avoid the firewall issues if you deploy these as static pods with kubeadm instead. However, this creates a bootstrapping loop with the other control plane pods, you won't be able to use your registry if you nuke your cluster to re-install it, and there's a strong chance a cluster problem could lead to cascading failures. </div> ^{5 In other words, just use systemd. systemd is great! The kubelet and CRI use systemd! You can do it, I believe! </div>}}}}}



Does Kubernetes Work With SELinux?
2025-07-02T00:00:00+00:00
Yes.</strong></p>
The official instructions</a> for kubeadm say to disable SELinux before doing the installation and give a big fat warning that:</p>


Setting SELinux in permissive mode by running setenforce 0</code> and sed ...</code> effectively disables it. This is required to allow containers to access the host filesystem; for example, some cluster network plugins require that. You have to do this until SELinux support is improved in the kubelet.</li>
You can leave SELinux enabled if you know how to configure it but it may require settings that are not supported by kubeadm.</li>
</ul>
</blockquote>
The second sentence essentially tells the story: there are a few steps you need to do to make Kubernetes and SELinux interact happily, but it’s not something that kubeadm can do by itself, so the devs are preemptively disabling SELinux to avoid floods of GitHub issues of why people’s clusters aren’t working. With a little bit of effort (and I mean only a little), it is perfectly possible to make Kubernetes work with SELinux. The existing documentation about it is not great, so it me quite a while to realize that. For simplicity here I will assume you are using a RHEL-based system.</p>
A quick recap: in SELinux</a>, every process, file, and port have a type</em>, and policies define which process types are allowed to access which file and port types. There are many different policies that come pre-installed on a RHEL system, and fortunately there is the container-selinux</a> package that comes with policies for containers. There are two container types relevant to us:</p>

container_t</code>: This is the default type that most containers run with, and it lets them access container file types. An SELinux compatible CRI will automatically launch most containers in the cluster with this type, and except for a few exceptions it will “just work”.</li>
spc_t</code>: This is a “super privileged container”—it is unconfined by SELinux and can do whatever it wants. Clearly we want to minimize the number of containers that run with this type, so it should only be used by administrative containers that interact with the host to manage the cluster itself.</li>
</ul>
Let’s install a cluster then. Here we will be using CRI-O</a>, which is a CRI developed by RedHat that is of course very compatible with SELinux (though I think containerd works too).</p>

Install container-selinux, cri-o, kubelet, kubeadm, and kubectl using these instructions</a>.</li>
Bootstrap</a> a cluster with kubeadm like normal. CRI-O will launch the control pods as spc_t</code>.</li>
Install a CNI like Cilium</a>. The Cilium pods are set to run as spc_t</code> since they need to interact with the host network.1</a></sup></li>
</ol>
And that’s it? Almost. For security reasons, containers with the container_t</code> type are not allowed to access the underlying host, but on a bare-metal cluster we usually need to install extra administrative pods that do need access (e.g.</abbr> a PersistentVolume provider). Some of these containers will run as spc_t</code> automatically, such as ones that set privileged: true</code> like TopoLVM</a>, but some won’t. For example, Alloy</a> attempts to mount hostPath</code> volumes to scrape your host logs, and a quick look at the audit logs using ausearch -m AVC,USER_AVC -ts recent -i</code> shows a denial.</p>
----
type=PROCTITLE msg=audit(2025-07-03 03:34:58.695:2086) : proctitle=/bin/alloy run /etc/alloy/config.alloy --storage.path=/tmp/alloy --server.http.listen-addr=0.0.0.0:12345 --server.http.ui-path-p
type=SYSCALL msg=audit(2025-07-03 03:34:58.695:2086) : arch=x86_64 syscall=bpf success=no exit=EACCES(Permission denied) a0=BPF_MAP_CREATE a1=0xc000b1fd90 a2=0x50 a3=0x0 items=0 ppid=7511 pid=7513 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=alloy exe=/usr/bin/alloy subj=system_u:system_r:container_t:s0:c269,c920 key=(null)
type=AVC msg=audit(2025-07-03 03:34:58.695:2086) : avc:  denied  { map_create } for  pid=7513 comm=alloy scontext=system_u:system_r:container_t:s0:c269,c920 tcontext=system_u:system_r:container_t:s0:c269,c920 tclass=bpf permissive=0
</code></pre>
The simplest way to fix this is to manually override its security context</a> to run as spc_t</code> in the Helm chart.</p>
alloy:
  securityContext:
    seLinuxOptions:
      type: spc_t
</code></pre>
Of course this is somewhat of a blunt instrument, since spc_t</code> removes all SELinux restrictions, not just the ones that Alloy needs relaxed. Ideally you’d create a custom policy using something like udica</a> that is tailored for your specific application.2</a></sup> Using spc_t</code> though works in a pinch and is much better than disabling SELinux entirely.</p>
For curiosity’s sake, you can run ps -eZ | grep -e spc_t -e container_t</code> on the host see the SELinux types of the container processes. Here we can see the control plane pods, cilium, and alloy running as spc_t</code> as expected, and core-dns and a utility pod running as container_t</code> since they don’t need host access.</p>
system_u:system_r:spc_t:s0         3642 ?        00:02:08 kube-controller
system_u:system_r:spc_t:s0         3653 ?        00:01:30 kube-scheduler
system_u:system_r:spc_t:s0         3668 ?        00:02:58 etcd
system_u:system_r:spc_t:s0         3684 ?        00:05:36 kube-apiserver
system_u:system_r:spc_t:s0         3777 ?        00:00:01 kube-proxy
system_u:system_r:spc_t:s0         4977 ?        00:00:12 cilium-operator
system_u:system_r:spc_t:s0         5026 ?        00:01:30 cilium-agent
system_u:system_r:spc_t:s0         5384 ?        00:00:00 cilium-health-r
system_u:system_r:spc_t:s0         5467 ?        00:00:00 cilium-envoy-st
system_u:system_r:spc_t:s0         5471 ?        00:00:12 cilium-envoy
system_u:system_r:container_t:s0:c120,c730 5523 ? 00:00:07 coredns
system_u:system_r:container_t:s0:c321,c358 5536 ? 00:00:07 coredns
system_u:system_r:spc_t:s0:c51,c687 8739 ?       00:00:00 alloy
system_u:system_r:container_t:s0:c51,c687 8751 ? 00:00:00 prometheus-conf
</code></pre>
And there you have it, now you can run a Kubernetes cluster with one less tear</a> from Dan Walsh.</p>
^{1</sup>
You may need to restart the cri-o and kubelet systemd services to pick up the new CNI.</p>
</div>}^2</sup>For example, RKE2 runs its control plane pods with a custom type</a> that is more restricted than spc_t</code>.</p>
</div>


Blocking Sudo Exploits with fapolicyd
2025-07-01T00:00:00+00:00
There’s a joke that goes around my office: if corporate ever tries to lock down root access on our workstations, we’ll just use the latest sudo vulnerability to get it back. This is not an idle threat. CVE-2025-32463</a> has just been announced, and it allows arbitrary unprivileged users to acquire root privileges on most Linux systems out-of-the-box. This is an all-too-common story: a bug in an SUID binary allows unchecked privilege escalation. Once an attacker acquires root privileges the game is over: MAC systems can be disabled (setenforce 0</code>), your firewall rules can be changed (nft flush ruleset</code>), and the attacker can read arbitrary files (cat /etc/shadow</code>). With one command all additional security measures can be bypassed, leading to a complete compromise of your system. Scary stuff!</p>
Unfortunately, patching these vulnerabilities is playing a game of Whac-A-Mole. You may be able to patch some after the fact, but other vulnerabilities of this sort almost certainly remain. In cases like this, proactive</em> measures are much better than reactive</em> ones. How then can we prevent these vulnerabilities when we have little idea where the next one will show up?1</a></sup></p>

  
</figure>
Many local privilege escalation vulnerabilities require executing custom binaries on the target system. What if we just … block executing everything by default? This is exactly the approach that fapolicyd</a> uses. It may seem extreme, but it is effective. fapolicyd maintains a database of trusted files on the system (by default those installed by your package manager), and all</em> attempts to open or execute a file are checked against that database. Any access of a potentially dangerous file (e.g.</abbr> an executable, shared library, or even Python script) is denied if that file is not on the trusted list.2</a></sup> (For extra security, fapolicyd can also optionally track the SHA-256 hashes of trusted files and deny the access if that hash has changed.)</p>
Let’s recreate the privilege escalation with a vulnerable version of sudo, and then see how fapolicyd blocks it. The exploit itself is quite clever: it creates a custom shared library that execs a root shell, and then tricks sudo into loading it via chroot(2)</code>. For a testing system I will be using CentOS Stream 10 with sudo version 1.9.15p5.</p>

As root, create an unprivileged user that isn’t in the wheel group: useradd -m lowpriv</code></li>
Set the password for that user: passwd lowpriv</code></li>
Switch to that user account: su -l lowpriv</code></li>
As the unprivileged user, copy the sudo-chwoot.sh</code> script and run it. You should now have a root shell.</li>
</ol>
[lowpriv@localhost ~]$ ./sudo-chwoot.sh
woot!
[root@localhost /]# id
uid=0(root) gid=0(root) groups=0(root),1001(lowpriv) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
</code></pre>
Next, we can see how fapolicyd blocks this. Assuming you are on a RHEL-based system, install fapolicyd</a> and then start it. Re-running the exploit will now show the following:</p>
[lowpriv@localhost ~]$ ./sudo-chwoot.sh
woot!
[sudo] password for lowpriv:
Sorry, try again.
[sudo] password for lowpriv:
Sorry, try again.
[sudo] password for lowpriv:
sudo: 3 incorrect password attempts
[lowpriv@localhost ~]$
</code></pre>
No dice—the exploit failed. We can see what happened using ausearch -i -ts recent -m FANOTIFY</code> to view the audit log.</p>
----
type=PROCTITLE msg=audit(07/01/2025 19:31:57.613:39571) : proctitle=sudo -R woot woot
type=PATH msg=audit(07/01/2025 19:31:57.613:39571) : item=0 name=libnss_/woot1337.so.2 inode=10034633 dev=fd:00 mode=file,755 ouid=lowpriv ogid=lowpriv rdev=00:00 obj=unconfined_u:object_r:user_tmp_t:s0 nametype=NORMAL cap_fp=none cap_fi=none cap_fe=0 cap_fver=0 cap_frootid=0
type=CWD msg=audit(07/01/2025 19:31:57.613:39571) : cwd=/tmp/sudowoot.stage.eaL5jR
type=SYSCALL msg=audit(07/01/2025 19:31:57.613:39571) : arch=x86_64 syscall=openat success=no exit=EPERM(Operation not permitted) a0=AT_FDCWD a1=0x643319d81e10 a2=O_RDONLY|O_CLOEXEC a3=0x0 items=1 ppid=119624 pid=119634 auid=admin uid=lowpriv gid=lowpriv euid=root suid=root fsuid=root egid=lowpriv sgid=lowpriv fsgid=lowpriv tty=pts3 ses=33 comm=sudo exe=/usr/bin/sudo subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
type=FANOTIFY msg=audit(07/01/2025 19:31:57.613:39571) : resp=deny fan_type=rule_info fan_info=8 subj_trust=unknown obj_trust=no
</code></pre>
The output is a little abstruse, but it shows that:</p>

A user tried to execute the vulnerable sudo command</li>
That sudo command tried to open libnss_/woot1337.so.2</code></li>
The openat</code> syscall on that path failed with EPERM</code></li>
fapolicyd denied the access because the subject (the shared library) was untrusted.</li>
</ul>
The system overhead of fapolicyd is non-neglible: it evaluates all open and exec calls against the trust database, and it can be tricky to setup. However, given the existence of these privilege escalation vulnerabilies that allow trivial takeover of your entire system, you cannot run a secure system without it.</p>
P.S. fapolicyd would also block the recent CVE-2025-6019</a>, since the root shell that udisks2 is tricked into mounting would not be trusted.</p>
^{1</sup>
I mean, $10 there’ll be another one in sudo, but it’s not the only problem.</p>
</div>
^{2</sup>
I didn’t want to say it, but fapolicyd is similar in spirit to an anti-virus, except those use an allow-by-default-deny-if-suspicious approach (and are usually crap), while fapolicyd maintains a much more strict (and thus effective) deny-by-default-allow-if-trusted one.</p>
</div>}}


Installing KOReader on a Kobo
2024-12-21T00:00:00+00:00
I was recently interested in getting an e-reader, and there are two main options: a Kindle, or a Kobo. The Kindle only supports their proprietary file format</a> and is difficult to jailbreak</a>. The Kobo on the other hand supports the open EPUB</a> standard and is extremely hackable</a>. The choice was thus obvious, and I popped down to the store to get a Clara BW</a>.</p>
I had originally hoped to replace the stock firmware with the open source Quill/InkBox</a> OS, though unfortunately that project has halted since new Kobos are using secure boot :( Perhaps one day the PineNote</a> or even OpenBook</a> can fulfill that role.</p>
Instead, I settled on installing KOReader</a>, which is an insanely powerful and customizable ebook reader application. As said before, Kobos are quite easy to modify: simple plug your Kobo into your computer, hit the Connect</i> button on the screen, and now you can mount the Kobo file system as a regular external drive. The standard way</a> to install KOReader is to download an all-in-one zip and copy it onto the drive, but that's no fun. Instead I decided to “pop the hood” and do it manually, which was very easy after following this guide</a>.</p>
First, we need to prevent the default Nickel firmware from indexing the hidden directories where we are going to install our new software. Edit the /.kobo/Kobo/Kobo eReader.conf</code> file and add the following section to the bottom:</p>
[FeatureSettings]
ExcludeSyncFolders=(\\.(?!kobo|adobe).+|([^.][^/]*/)+\\..+)
</code></pre>
Next, we need to install a launcher to run our custom software. There are two main ones currently: KFMon</a> and NickelMenu</a>. NickelMenu has the advantage of persisting between firmware upgrades, so that's the route I chose. Installation is quite easy:</p>

Download the latest KoboRoot.tgz</code> from the releases page</a>.</li>
Copy the KoboRoot.tgz</code> into the /.kobo</code> directory on the mounted drive. (Do not</em> extract the file, just copy it.)</li>
Unmount your Kobo and unplug it. It should reboot, and now you will have NickelMenu in the bottom right corner.</li>
</ol>
Next it's time to install KOReader!</p>

Download the latest koreader-kobo</code> zip file from the releases page</a>.</li>
Unzip the KOReader release, and copy the extracted koreader</code> directory to /.adds/koreader</code> on your Kobo.</li>
Create a file called /.adds/nm/koreader</code> with the following contents:menu_item :main :KOReader :cmd_spawn :quiet:exec /mnt/onboard/.adds/koreader/koreader.sh
</code></pre>
</li>
Unmount your Kobo, and now you can use NickelMenu to launch KOReader.</li>
</ol>
As a bonus, we can use the same technique to install Plato</a>, which is a simple ebook reader designed specifically for Kobo.</p>

Download the latest Plato zip file from the releases page</a>.</li>
Unzip the Plato release, and copy the extracted contents to /.adds/plato</code> on your Kobo.</li>
Create a file called /.adds/nm/plato</code> with the following contents:menu_item :main :Plato :cmd_spawn :quiet:exec /mnt/onboard/.adds/plato/plato.sh
</code></pre>
</li>
Unmount your Kobo, and now you can use NickelMenu to launch Plato.</li>
</ol>
And that's it! Now go get some nice ebooks</a> and start reading!</p>


The Crushing Complexity of the GPL3
2021-12-23T00:00:00+00:00
I recently released my first open source project</a> and had to choose a license. Being an advocate of free software principles, my first choice was the GPL3</a>. However, I decided out of curiosity to read the license first—I think it’s reasonable that if I’m going to use a license I should understand what it requires. A good forty-five minutes later, I had a single conclusion: the GPL3 is complicated—horribly, mind-numbingly complicated. I think a simple picture will illustrate:</p>

  
</figure>
One of these things is not like the others. But if the length of the GPL3 is intimidating, here is a 13,000 word explanation</a> of the differences of the GPL3 from the GPL2!</p>
To give just a small example, consider the following requirement for Conveyance from section 5:</p>

d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so.</p>
</blockquote>
The definition of “Appropriate Legal Notices” is provided in section 0:</p>

An interactive user interface displays “Appropriate Legal Notices” to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion.</p>
</blockquote>
So … if the work has interactive user interfaces, then it must display Appropriate Legal Notices, but if it doesn’t display Appropriate Legal Notices, apparently that’s fine? So do I need legal notices or not? And even if I do, why on earth would the license try to dictate such a nitpicky thing?</p>
Here is another section that defines what code you are obligated to release under the license:</p>

The “Corresponding Source” for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work’s System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work.</p>
</blockquote>
As a baseline, the Corresponding Source not only includes the work itself, but all</em> code needed to generate, install, run, and modify the work! That’s like, everything on your computer from the ground up! Exceptions are carved out for general-purpose tools or System Libraries, which are defined in the previous paragraph as:</p>

The “System Libraries” of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form.</p>
</blockquote>
Major Component and Standard Interface are then defined as … it just goes on and on. (Also, I like the phrase “intimate data communication.” Try defining that in court!)</p>
This is merely the tip of the iceberg. The GPL3 also includes:</p>

provisions for contracting GPL3 software from third-parties</li>
anti-DMCA clauses to nullify article 11 of the 1996 WIPO copyright treaty</li>
anti-DRM clauses, which attempt to define what a “User Product” is</li>
explicit permission to distribute the software through peer-to-peer networks (apparently the GPL2 has problems with that)</li>
a weird exception for patent agreements done before 28 March 2007 due to a lawsuit settlement between Novell and Microsoft</a>.</li>
</ul>
And there is so much more. To appreciate the complexity of the GPL3 you must read the license yourself. For a taste of the legal intricacies that a lawyer would subject this to, here is Ken Mitchell’s analysis</a> of one</em> paragraph in the AGPL3 that is different from the GPL3.</p>
It was not always this way. In 2005, Lawrence Rosen updated his own Open Software License</a> to version 3 with the explicit goal to “express clearly the licensing objectives of software copyright and patent owners.” However, this goal of simplicity was not shared by the authors of the GPL3. Allison Randal lamented</a>:</p>

I don’t know that clarity and simplicity was ever a goal that Richard Stallman, Eben Moglen, or any of the GPLv3 team desired to achieve. Looking at the near-finished draft, I have to say it’s unlikely that they ever considered simplicity a priority, if they considered it at all.</p>
</blockquote>
In hindsight, ensuring the GPL3 covered every last corner-case of software licensing ended up missing the forest for the trees</a>:</p>

Largely because of its complexity and baroqueness, GPLv3 was a lost opportunity to create a strong copyleft license that would appeal very broadly to modern individual software authors and corporate licensors. I believe individual developers today tend to prefer short, simple, easy to understand, minimalist licenses, the most obvious example of which is the MIT License.</p>
</blockquote>
A shame, but so it is.</p>
What, then, is a good copyleft license for modern usage? The EUPL</a> is a noble effort: it’s simple, written with an explicit attention to legal detail, and even includes compatibility with other copyleft licenses! However, there is one major flaw: the definition of “Derivative Works” defers to the copyright law of whichever EU state you’re in (or Belgium, if you’re not ¯\_(ツ)_/¯</code>), which leaves it caught in a quagmire</a> of whether linking creates a derivative work. (They don’t think so, but it’s never been tested in court, so who knows!)</p>
The only copyleft license I’m aware of that is easy to understand, legally unambiguous, and well-established is the MPL2</a>. It has a weaker form of copyleft than the GPL that is based purely on source code files, but it is supremely simple, and also interoperates well with the rest of the permissively-licensed open source world. Ultimately, it’s the pragmatic choice, and the license I chose.</p>


A Basic C Makefile
2021-06-13T00:00:00+00:00
These days I usually use Meson</a> for building C and C++ code, but for small projects the simplicity and ubiquity of a Makefile can't be beat. In the past I'd write custom Makefile rules to compile each file individually, but with the power of wildcards we can write a generic Makefile that drops in to any project. Simply set the executable name, customize your compiler options, and off you go.</p>
# `make` builds the target
# `make file.o` creates the `file` object file
# `make clean` will rm all object files and the target

TARGET = my-executable

# sample settings
CC = gcc
CFLAGS = -std=c11 -Wall -Wextra -Wpedantic -g
LDLIBS = -lpthread

SOURCES = $(wildcard *.c)
HEADERS = $(wildcard *.h)
OBJECTS = $(SOURCES:.c=.o)

# link the object files to create the target
$(TARGET): $(OBJECTS)
	$(CC) $(CFLAGS) $^ -o $@ $(LDLIBS)

# compile rule for the object files
%.o: %.c $(HEADERS)
	$(CC) $(CFLAGS) -c $< -o $@

.PHONY: clean

clean:
	-rm $(OBJECTS) $(TARGET)
</code></pre>


Final Update
2019-08-22T00:00:00+00:00
It is now the final week of GSoC, and it is time for me to write my final blog post. Over the past summer I have worked on fixing the Coverity scan</a> issues in coreboot, with the goal of making the code base “Coverity clean”. This has involved writing a substantial number of patches, the vast majority of which are in coreboot, with a sprinkling in a few other projects:</p>

146</a> patches in coreboot</li>
6</a> patches in flashrom</li>
6</a> patches in vboot</li>
3</a> patches in Chromium EC</li>
4</a> patches in OpenSBI</li>
2</a> patches in em100</li>
1</a> in the Linux kernel</li>
</ul>
At the time of writing, a few of my patches are still under review</a> on Gerrit, so it is possible (and hopeful!) that this list will increase over the next few weeks.</p>
In total, these patches resolved 172 Coverity issue reports of actual bugs. However, Coverity also isn’t always right, and some issues weren’t actually problems that required patches. These issues, 91 in total, were either false positives or intentional and were ignored. At the moment, there are currently 223 remaining reports in the issue tracker for coreboot. Despite being a substantial number, this is almost entirely composed of issues from third-party projects (such as OpenSBI or vboot, which probably shouldn’t be counted in the coreboot tracker anyway), and the AMD vendorcode. The original plan at the beginning of the summer was to work on the AMD vendorcode; however, after discussion with my mentors we decided to skip it, since with the upcoming deprecations for coreboot 4.11 it might not be around much longer. Aside from this, there are roughly 20 remaining issues, which mostly required refactoring or technical knowledge that I don’t have.</p>
With the summary out of the way, I’d like to give everyone a sample of the sort of bugs I’ve worked on during the project, and hopefully give advice for avoiding them in the future. Here is a list of the most common, nasty, or subtle types of bugs I’ve found over the summer.</p>
Missing Break Statements</h3>
In switch statements in C, every case statement implicitly falls through to the next one. However, this is almost never the desired behavior, and so to avoid this every case needs to be manually terminated by a break to prevent the fall-through. This unfortunately is very tedious to do and is often accidentally left out. For a prototypical example, let’s look at CB:32180</a> from the AGESA vendorcode.</p>
switch (AccessWidth) {
        case AccessS3SaveWidth8:
                RegValue = *(UINT8 *) Value;
                break;
        case AccessS3SaveWidth16:
                RegValue = *(UINT16 *) Value;
                break;
        case AccessS3SaveWidth32:
                RegValue = *(UINT32 *) Value;
        default:
                ASSERT (FALSE);
}
</code></pre>
In this switch there is a missing break after the AccessS3SaveWidth32</code> case, which will then fall-through to the false assertion. Clearly not intentional! Other examples of this, though not as severe, can be found in CB:32088</a> and CB:34293</a>. Fortunately, these errors today can be prevented by the compiler. GCC recently added the -Wimplicit-fallthrough</code> option, which will warn on all implicit fall throughs and alert to a potentially missing break. However, some fall throughs are intentional, and these can be annotated by a /* fall through */</code> comment to silence the warning. Since CB:34297</a> and CB:34300</a> this warning has been enabled in coreboot, so this should be the last we see of missing break statements.</p>
Off-by-One Errors</h3>

There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors.</p>
</blockquote>
Everyone has been bitten by off-by-one errors. Let’s take a look at CB:32125</a> from the Baytrail graphics code.</p>
static void gfx_lock_pcbase(struct device *dev)
{
        const u16 gms_size_map[17] = { 0, 32, 64, 96, 128, 160, 192, 224, 256,
                                       288, 320, 352, 384, 416, 448, 480, 512 };
        ...
        u32 gms, gmsize, pcbase;
        gms = pci_read_config32(dev, GGC) & GGC_GSM_SIZE_MASK;
        gms >>= 3;
        if (gms > ARRAY_SIZE(gms_size_map))
                return;
        gmsize = gms_size_map[gms];
        ...
}
</code></pre>
Here we have an array gms_size_map</code> of 17 elements, and a bounds check on the gms</code> variable before it is used to index into the array. However, there’s a problem. The bounds check misses the case when gms == ARRAY_SIZE(gms_size_map) == 17</code>, which is one past 16—the index of the last array element. The fix is to use >=</code> in the check instead of ></code>. This exact error when performing a bounds check is very</em> common: see at least CB:32244</a>, CB:34498</a>, and CL:1752766</a> for other examples.</p>
Another nasty place where off-by-one errors strike is with strings—in particular, when making sure they are null terminated. Here is CB:34374</a> from the ACPI setup of the Getac P470.</p>
static long acpi_create_ecdt(acpi_ecdt_t * ecdt)
{
        ...
        static const char ec_id[] = "\_SB.PCI0.LPCB.EC0";
        ...
        strncpy((char *)ecdt->ec_id, ec_id, strlen(ec_id));
        ...
}
</code></pre>
The problem is that strncpy()</code> will only copy at most strlen(ec_id)</code> characters, which excludes the null character. The author might have been thinking of the similar strlcpy()</code>, which does</em> explicitly null terminate the string buffer even if it never reaches a null character. In this case none of the string-copying functions are needed, since ec_id</code> is a string buffer and so can be copied using a simple memcpy()</code>.</p>
Boolean vs Bitwise Operators</h3>
In C, all integers are implicitly convertible to boolean values and can be used with all boolean operators. While somewhat convenient, this also makes it very easy to mistakenly use a boolean operator when a bitwise one was intended. Let’s take a look at CB:33454</a> from the CIMX southbridge code.</p>
void sb_poweron_init(void)
{
        u8 data;
        ...
        data = inb(0xCD7);
        data &= !BIT0;
        if (!CONFIG(PCIB_ENABLE)) {
                data |= BIT0;
        }
        outb(data, 0xCD7);
        ...
}
</code></pre>
Here BIT0</code> is the constant 0x1</code>, so !BIT0</code> expands to 0, with the net effect of data</code> being completely cleared, regardless of the previous value from inb()</code>. The intended operator to use was the bitwise negation ~</code>, which would only clear the lowest bit. For more examples of this sort of bug, see CB:34560</a> and OpenSBI 3f738f5</a>.</p>
Implicit Integer Conversions</h3>
C allows implicit conversions between all integer types, which opens the door for many accidental or unintentional bugs. For an extremely subtle example of this, let’s take a look at OpenSBI 5e4021a</a>.</p>
void *sbi_memset(void *s, int c, size_t count);

void sbi_fifo_init(struct sbi_fifo *fifo, void *queue_mem,
                   u16 entries, u16 entry_size)
{
        ...
        sbi_memset(fifo->queue, 0, entries * entry_size);
}
</code></pre>
Do you see the problem? The issue is that entries</code> and entry_size</code> are both 16-bit integers, and by the rules of C are implicitly converted to int</code> before the multiplication. An int</code> cannot hold all possible values of a u16 * u16</code>, and so if the multiplication overflows the intermediate result could be a negative number. On 64-bit platforms size_t</code> will be a u64</code>, and the negative result will then be sign-extended to a massive integer. As the last argument to sbi_memset()</code>, this could lead to a very large out-of-bounds write. The solution is to cast one of the variables to a size_t</code> before the multiplication, which is wide enough to prevent the implicit promotion to int</code>. For other examples of this problem, see CB:33986</a> and CB:34529</a>.</p>
Another situation where implicit conversions strike is in error handling. Here is CB:33962</a> in the x86 ACPI code.</p>
static ssize_t acpi_device_path_fill(const struct device *dev, char *buf,
                                     size_t buf_len, size_t cur);

const char *acpi_device_path_join(const struct device *dev, const char *name)
{
        static char buf[DEVICE_PATH_MAX] = {};
        size_t len;
        if (!dev)
                return NULL;

        /* Build the path of this device */
        len = acpi_device_path_fill(dev, buf, sizeof(buf), 0);
        if (len <= 0)
                return NULL;
        ...
}
</code></pre>
With the function prototype right there, the problem is obvious: acpi_device_path_fill()</code> returns negative values in a ssize_t</code> to indicate errors, but len</code> is a size_t</code>, so all those negative error values are converted to extremely large integers, thus passing the subsequent error check. During code review this may not at all be obvious though.</p>
Both these errors could be prevented using the -Wconversion</code> compiler option, which will warn about all implicit integer conversions. However, there are an incredible number of such conversions in coreboot, and it would be a mammoth task to fix them all.</p>
Null Pointers</h3>
Null pointers need no introduction—they are well known to cause all sorts of problems. For a simple example, let’s take a look at CB:33134</a> from the HiFive Unleashed mainboard.</p>
static void fixup_fdt(void *unused)
{
        void *fdt_rom;
        struct device_tree *tree;

        /* load flat dt from cbfs */
        fdt_rom = cbfs_boot_map_with_leak("fallback/DTB", CBFS_TYPE_RAW, NULL);

        /* Expand DT into a tree */
        tree = fdt_unflatten(fdt_rom);
        ...
}
</code></pre>
This code attempts to load a device tree from a location in the CBFS. However, cbfs_boot_map_with_leak()</code> will return a null pointer if the object in the CBFS can’t be found, which will then be dereferenced in the call to fdt_unflatten()</code>. On most systems dereferencing a null pointer will lead to a segfault, since the operating system has set up permissions that prevent accessing the memory at address 0. However, coreboot runs before the operating systems has even started, so there are no memory permissions at all! If fdt_rom</code> is a null pointer, fdt_unflatten()</code> will attempt to expand the device tree from whatever memory is at address 0, leading to who knows what problems. A simple null check will avoid this, but requires the programmer to always remember to put them in.</p>
Another common issues with null pointers is that even if you do a check, it might not actually matter if the pointer has already been dereferenced. For example, here is a problem with the EDID parser in CB:32055</a>.</p>
int decode_edid(unsigned char *edid, int size, struct edid *out)
{
        ...
        dump_breakdown(edid);

        memset(out, 0, sizeof(*out));

        if (!edid || memcmp(edid, "\x00\xFF\xFF\xFF\xFF\xFF\xFF\x00", 8)) {
                printk(BIOS_SPEW, "No header found\n");
                return EDID_ABSENT;
        }
        ...
}
</code></pre>
In this case the EDID is dumped before doing the null pointer check, but at worst there should only be a wonky dump if edid</code> is null, right? Not necessarily. Since dereferencing a null pointer is undefined behavior, the compiler is allowed to assume that no null pointer dereferences occur in the program</em>. In this case, dereferencing the edid</code> pointer in dump_breakdown()</code> is an implicit assertion that edid</code> is not null, so an over-zealous compiler could remove the following null check! This optimization can be disabled using -fno-delete-null-pointer-checks</code> (which is done in coreboot), but does not prevent any problems that could have happened in the null dereference before the check took place. See this article in LWN</a> for details on how a vulnerability from this problem was dealt with in the Linux kernel.</p>
Conclusion</h3>
C has always had the mantra of “trust the programmer”, which makes mistakes and errors very easy to do. Some of these errors can be prevented at compile time using compiler warnings, but many cannot. Coverity and other static analyzers like it are very useful and powerful tools for catching bugs that slip past the compiler and through peer review. However, it is no silver bullet. All of these errors were present in production code that were only caught after the fact, and there are certainly bugs of this sort left that Coverity hasn’t found. What do we do about them, and how can we ever be sure that we’ve caught them all? Today, there are new languages designed from the beginning to enable safe and correct programming. For example, libgfxinit</a> is written in SPARK, a subset of Ada that can be formally verified at compile time to avoid essentially all of the above errors. There is also the new oreboot</a> project written in Rust, which has similar compile time guarantees due to its extensive type system. I hope to see these languages and others increasingly used in the future so that at some point, this job will have become obsolete. 🙂</p>


Weeks 11–12
2019-08-16T00:00:00+00:00
Hello again! For the past two weeks I have been working on Coverity issues in various third party repositories, notably flashrom</a> and vboot</a>. The majority of issues in both repositories are now fixed, with the remaining ones mostly being memory leaks. Also, support for OpenSBI</a> (a RISC-V supervisor binary interface) was recently added to coreboot, and Coverity picked up several issues</a> in that. With one more week left in the summer, the plan now is to tidy up the last few remaining issues in vboot, and shepherd my still in-progress patches through review. As usual, you can see the current status of my patches on Gerrit</a>.</p>
Following up from my post last week about neutralizing the ME, this week I decided to try replacing SeaBIOS with GRUB. While originally designed as a bootloader, GRUB can be compiled as a coreboot payload, which removes the need of having a BIOS or UEFI interface at all. Not only does this mean having a legacy-free boot process, but GRUB also has an incredible amount of flexibility over the previous systems. For example, one problem with current full disk encryption schemes is that the bootloader must always remain unencrypted, since factory BIOS/UEFI implementations are unable of unlocking encrypted disks. By embedding GRUB in the coreboot image we can move it off the hard drive and avoid this problem entirely.</p>
With this in mind, I decided to try installing GRUB on a system with a single LUKS 1 partition formatted with Btrfs. (GRUB currently doesn’t support LUKS 2, which is what many distributions use by default.) This setup will require two configuration files: one embedded in the coreboot image with instructions on how to unlock the partition, and one on the hard drive for booting the kernel. The first configuration file has to be hand-written, while the second can be generated using grub-mkconfig -o /boot/grub/grub.cfg</code>. The advantage of this two-stage system is that the first configuration file only has to be flashed once, and then all subsequent changes (e.g. adding kernel parameters) can be done with the system file. After tinkering around with the official GRUB documentation</a>, the coreboot wiki</a>, and this blog post</a>, I pieced together the following bare-bones config file for the payload:</p>
# Tell GRUB where to find its modules
set prefix=(memdisk)/boot/grub

# Keep the same screen resolution when switching to Linux
set gfxpayload=keep

# I'm not exactly sure what these do, but they look important
terminal_output --append cbmemc
terminal_output --append gfxterm

# Default to first option, automatically boot after 3 seconds
set default="0"
set timeout=3

# Enable the pager when viewing large files
set pager=1

menuentry 'Load Operating System' {

        # Load the LUKS and Btrfs modules
        insmod luks
        insmod btrfs

        # Unlock all crypto devices (should only be one)
        cryptomount -a

        # Load the system GRUB file from the first crypto device
        set root=(crypto0)
        configfile /boot/grub/grub.cfg
}
</code></pre>
Using menuconfig</code>, we can now configure coreboot to use GRUB as a payload.</p>
Payload ---> Add a Payload ---> GRUB2
        ---> GRUB2 version ---> 2.04
        ---> Extra modules to include in GRUB image
             (luks btrfs gcry_sha256 gcry_rijndael all_video cat)
             [*] Include GRUB2 runtime config file into ROM image
                 (grub.cfg)
</code></pre>
We need to add a few modules to the GRUB image to enable all the features we want: luks</code> and btrfs</code> are self-explanatory, gcry_sha256</code> and gcry_rijndael</code> add support for SHA256 and AES (the default hash and cipher for LUKS), all_video</code> adds support for more video and graphics drivers, and cat</code> is useful for printing config files during debugging. We also set here the path to the above grub.cfg</code> file, which by default is in the root of the coreboot tree.</p>
After flashing the new coreboot image, you will be greeted by two GRUB menus after booting: the above one for unlocking the partition, and then the system one for booting the kernel.</p>
The above GRUB config file is extremely basic, and although all advanced functionality can be done manually by dropping to the GRUB command line, this is very inconvenient. Let’s add two menu entries to the config file for shutting down and rebooting the system. Fortunately, this can be done without rebuilding the ROM from scratch. By default, the GRUB config file is stored as etc/grub.cfg</code> in the CBFS</a> (coreboot file system), which is used to store objects in the coreboot image. We can print the layout of the CBFS using the cbfstool</code> utility:</p>
$ cbfstool coreboot.rom print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           58068 none
cpu_microcode_blob.bin         0xe3c0     microcode      122880 none
fallback/ramstage              0x2c440    stage          105072 none
config                         0x45f00    raw               488 none
revision                       0x46140    raw               674 none
cmos.default                   0x46440    cmos_default      256 none
vbt.bin                        0x46580    raw              1412 LZMA (3863 decompressed)
cmos_layout.bin                0x46b40    cmos_layout      1808 none
fallback/postcar               0x472c0    stage           18540 none
fallback/dsdt.aml              0x4bb80    raw             15257 none
fallback/payload               0x4f780    simple elf     470937 none
etc/grub.cfg                   0xc2780    raw               670 none
(empty)                        0xc2a80    null          7523608 none
bootblock                      0x7ef7c0   bootblock        1512 none
</code></pre>
This shows all the objects located in the image, including the stages (bootblock, romstage, ramstage, and payload) and various other configuration files and binaries. Next, we extract the existing config file from this image:</p>
$ cbfstool coreboot.rom extract -n etc/grub.cfg -f grub.cfg
</code></pre>
Now add the following two entries to the end of the extracted file:</p>
menuentry 'Poweroff' {
        halt
}

menuentry 'Reboot' {
        reboot
}
</code></pre>
Now we delete the existing configuration file, and then add our new one back in:</p>
$ cbfstool coreboot.rom remove -n etc/grub.cfg
$ cbfstool coreboot.rom add -n etc/grub.cfg -f grub.cfg -t raw
</code></pre>
That’s it! Flash back the new ROM, and you’re good to go.</p>
Even with these changes, we still have a very basic configuration file. For more advanced setups (such as booting from a live USB, LVM support, signing kernels with GPG, etc.) I recommend looking at the comprehensive Libreboot</a> GRUB file and its hardening guide</a>.</p>


Weeks 8–10
2019-08-07T00:00:00+00:00
Hello everyone! Coverity has come back online again after its long upgrade, and with it a pile of new scan issues (over 100). I put aside a week to fix all the new issues in parts of the code base I had already worked through, and then spent the next two weeks finishing soc and the Cavium vendorcode. Referring to the overview page</a>, there are now 300 scan issues left in the code base. While a considerable number, this consists almost entirely of patches still going through review, vboot, and the AMD vendorcode—everything outside of this is done. At this point, the plan at the beginning of the summer was to continue working on the AMD vendorcode, but after discussing with my mentors we have decided to do something else. The AMD vendorcode is extremely dense, and with the upcoming deprecations in 4.11 it may not be around for much longer. The plan now is to work on vboot</a> and the recently resurrected scan page for flashrom</a>.</p>
Anyway, enough with the boring schedule stuff. I finally got around this week to wiping the ME on my T500, and like last time it was a lot more involved than I expected. Onwards!</p>
The Intel Management Engine</a> (ME) is an autonomous coprocessor integrated into all Intel chipsets since 2006. Providing the basis for Intel features such as AMT</a>, Boot Guard</a>, and Protected Audio Video Path (used for DRM), the ME has complete access to the system memory and networking. The ME runs continuously as long as the system has power, even if the computer is asleep or turned off, and on systems with AMT can even be accessed remotely. Naturally, this is an incredible security risk. On modern systems it is impossible to disable the ME completely, since any attempt to do so will start a watchdog timer that will shutdown the computer after 30 minutes. However, in older chipsets (such as the one in the T500), no such timer exists, allowing attempts to disable it entirely.</p>
The ME firmware is located in the SPI flash chip, which is divided into multiple regions. Using the coreboot utility ifdtool</code>, we can analyze a dumped ROM to check the regions in the layout along with their offsets within the flash image.</p>
$ ifdtool -f layout.txt factory.rom
File factory.rom is 8388608 bytes
Wrote layout to layout.txt
$ cat layout.txt
00000000:00000fff fd
00600000:007fffff bios
00001000:005f5fff me
005f6000:005f7fff gbe
005f8000:005fffff pd
</code></pre>
These regions are as follows:</p>

Intel Flash Descriptor (fd</code>) – Describes the layout of the rest of the flash chip, as well as various chipset configuration options. This blog post</a> by Alex James has further information about the IFD.</li>
BIOS (bios</code>) – Contains the factory BIOS. Normally this is the only part of the flash chip that you overwrite when installing coreboot.</li>
Management Engine (me</code>) – Firmware for the ME coprocessor, including a kernel (ThreadX on older models, MINIX 3 on newer ones), as well as a variety of other modules for network access and remote management.</li>
Gigabit Ethernet (gbe</code>) – Configuration for the Gigabit Ethernet controller, including the system’s MAC address.</li>
Platform Data (pd</code>) – Other miscellaneous data for the factory BIOS, which coreboot doesn’t use.</li>
</ul>
The IFD also contains read and write permissions for the rest of the flash chip, which we can analyze using ifdtool</code>.</p>
$ ifdtool -d factory.rom
...
FLMSTR1:   0x1a1b0000 (Host CPU/BIOS)
   Platform Data Region Write Access: enabled
   GbE Region Write Access:           enabled
   Intel ME Region Write Access:      disabled
   Host CPU/BIOS Region Write Access: enabled
   Flash Descriptor Write Access:     disabled
   Platform Data Region Read Access:  enabled
   GbE Region Read Access:            enabled
   Intel ME Region Read Access:       disabled
   Host CPU/BIOS Region Read Access:  enabled
   Flash Descriptor Read Access:      enabled
   Requester ID:                      0x0000
...
</code></pre>
Unfortunately, ME write access is disabled for the host CPU, which prevents us from overwriting it using an internal flash—we’re gonna have to get out the external programmer. If you recall from last time, accessing the SOIC of the T500 required a laborious process of extracting the motherboard from the case, so this time I cut a little hole in the frame to make flashing easier. As it turned out, having this easy access port was very handy.</p>

  
  The old way.</figcaption>
</figure>

  
  The new—a bit hacky, but it works.</figcaption>
</figure>
After reading back the ROM image, I decided to use try using me_cleaner</a>, a tool for partially deblobbing the ME firmware on newer laptops, and for mine removing it entirely. Following the instructions</a> for an external flash, we use the -S</code> flag to clean the firmware.</p>
$ python me_cleaner.py -S -O coreboot-nome.rom coreboot-orig.rom
Full image detected
Found FPT header at 0x1010
Found 13 partition(s)
Found FTPR header: FTPR partition spans from 0xd2000 to 0x142000
ME/TXE firmware version 4.1.3.1038 (generation 1)
Public key match: Intel ME, firmware versions 4.x.x.x
The meDisable bit in ICHSTRP0 is NOT SET, setting it now...
The meDisable bit in MCHSTRP0 is NOT SET, setting it now...
Disabling the ME region...
Wiping the ME region...
Done! Good luck!
</code></pre>
After wiping the image, we can use ifdtool</code> again to see that the ME region has been completely erased.</p>
00000000:00000fff fd
00600000:007fffff bios
005f6000:005f7fff gbe
005f8000:005fffff pd
</code></pre>
While this is the most sure-fire way to ensure the ME has been disabled, it’s not perfect—on boot the ME coprocessor will still attempt to read from its firmware in the flash region, and finding it not there will hang in an error state. However, it has been discovered through reverse engineering that Intel has incorporated various bits</a> into the IFD that will disable the ME soon after boot. (One of these, the High Assurance Platform</a> “HAP” bit, was incorporated at the request of the US government, which also understands the security risks of an unfettered ME.) For my laptop, this consists of setting two bits in the ICHSTRP</code> and MCHSTRP</code>, which me_cleaner</code> does by default. After enabling this, the ME should gracefully shutdown after boot, which hopefully will cause the fewest system stability issues.</p>
Finally after all that, let’s use ifdtool</code> again to unlock all regions of the flash chip, which should ideally negate the need of ever doing an external flash again.</p>
$ ifdtool -u coreboot-nome.rom
</code></pre>
Let’s write it! Unlike last time we write the entire image, not just the BIOS region.</p>
$ flashrom -p linux_spi:dev=/dev/spidev1.0,spispeed=4096 -w coreboot-nome-unlock.rom
</code></pre>
Now the moment of truth. After hitting the power button … nothing. Gah.</p>
Well, not exactly nothing. The screen doesn’t turn on, but the CD drive is making noises, so it’s not completely dead. Hmmm, let’s try again, but this time only setting the disable bits without wiping the ME region. This can be done using the -s</code> flag of me_cleaner</code> (which incidentally has a bug that required a small patch</a>).</p>
$ python me_cleaner.py -s -O coreboot-nome.rom coreboot-orig.rom
</code></pre>
Reflashing again … same problem. Nothing.</p>
Not completely sure what to do and worried that the ME was actually needed to boot, I decided to flash a pre-compiled Libreboot</a> image on the laptop, which has the ME disabled out of the box and should hopefully Just Work. … And it did! The computer booted gracefully into the OS with no glitches or stability issues that I could see. Perfect. So disabling the ME is technically possible, but how exactly do I do it? Asking around a bit on IRC, I was directed to the coreboot bincfg</code> utility, which is capable of generating an ME-less IFD from scratch. Following this similar guide</a> for the X200, I was able to piece together a complete ROM.</p>
First, we need to generate a new IFD. The bincfg</code> directory contains configuration files for the X200, which we can fortunately re-use since the T500 has the same controller hub.</p>
$ bincfg ifd-x200.spec ifd-x200.set ifd.bin
</code></pre>
This IFD comes with only three regions, and has the ME disable bits set by default.</p>
00000000:00000fff fd
00003000:007fffff bios
00001000:00002fff gbe
</code></pre>
There are also configuration files for generating a new GbE region, but that requires certain fiddlings with the MAC address that I don’t know how to do—for now we can re-use the GbE from the old ROM, using ifdtool</code> to chop it out.</p>
$ ifdtool -x coreboot-orig.rom
File coreboot-orig.rom is 8388608 bytes
  Flash Region 0 (Flash Descriptor): 00000000 - 00000fff
  Flash Region 1 (BIOS): 00600000 - 007fffff
  Flash Region 2 (Intel ME): 00001000 - 005f5fff
  Flash Region 3 (GbE): 005f6000 - 005f7fff
  Flash Region 4 (Platform Data): 005f8000 - 005fffff
$ mv flashregion_3_gbe.bin gbe.bin
</code></pre>
Next, we place the IFD and GbE regions in a blobs</code> directory in the root of the coreboot tree, and then direct the build system to use them when generating an image. Here are the menuconfig</code> settings:</p>
General setup ---> [*] Use CMOS for configuration values
              ---> [*] Allow use of binary-only repository
Mainboard ---> Mainboard vendor ---> Lenovo
          ---> Mainboard model ---> ThinkPad T500
          ---> Size of CBFS filesystem in ROM (0x7fd000)
Chipset ---> [*] Add Intel descriptor.bin file (blobs/ifd.bin)
        ---> [*] Add gigabit ethernet configuration (blobs/gbe.bin)
        ---> Protect flash regions ---> Unlock flash regions
Devices ---> Display ---> Linear "high-resolution" framebuffer
Payloads ---> Add a payload ---> SeaBIOS
         ---> SeaBIOS version ---> master
</code></pre>
Note that since the ME and Platform Data regions have been deleted from the new layout, we can expand the CBFS to the entire bios</code> size—0x7fd000</code> bytes, nearly the entire flash chip. This will allow in the future installing bigger and more interesting payloads, such as GRUB</a> or LinuxBoot</a>.</p>
Finally, I wrote this new image to the flash chip. It worked. Huzzah!</p>
After installation, we can check the status</a> of the ME using intelmetool</code>.</p>
$ sudo intelmetool -m
Can't find ME PCI device
</code></pre>
Perfect.</p>


Weeks 5–7
2019-07-16T00:00:00+00:00
Hello again! It’s been three weeks since my last post (well, one of those weeks I was on vacation, so more like two weeks), so there are many many updates to write about. If you recall from my last post, Coverity has been down for maintenance, so I started looking around for other things to do. Here is a list of some of the highlights:</p>

Coverity isn’t the only static analyzer we use—coreboot also runs nightly scans</a> using the Clang Static Analyzer</a>, an open source tool from the LLVM project. The results from these scans aren’t as detailed as Coverity and unfortunately contain many duplicates and false positives, but I was able to submit patches</a> for some of the issues. Usability of the analyzer is also still somewhat limited, since we have no way of recording fixes for past issues or ignoring false positives. (A framework like CodeChecker</a> might be able to help with this.)</li>
coreboot will (hopefully) soon be (almost) VLA free</a>! Variable length arrays</a> (VLAs) were a feature added in C99 that allows the length of an array to be determined at run time, rather than compile time. While convenient, this is also very dangerous, since it allows use of an unlimited amount of stack memory, potentially leading to stack overflow. This is especially dangerous in coreboot, which often has very little stack space to begin with. Fortunately, almost all of the VLAs in coreboot can be replaced with small fixed-sized buffers, aside from one tricky exception that will require a larger rewrite. This patch was inspired by a similar one</a> for Linux, and is currently under review.</li>
Similar to the above, coreboot will also soon be free of implicit fall throughs</a>! Switches in C have the unfortunate property of each case statement implicitly falling through to the next one, which has led to untold number of bugs over the decades and at least 38 in coreboot itself (by Coverity’s count). GCC however recently added</a> the -Wimplicit-fallthrough</code> warning, which we can use to banish them for good. As of this patch</a>, all accidental fall throughs have been eliminated, with the intentional ones marked with the /* fall through */</code> comment. Hopefully this is the last we see of this type of bug.</li>
coreboot now has a single implementation</a> of <stdint.h></code>. Previously, each of the six supported architectures (arm, arm64, mips, ppc64, riscv, and x86) had their own custom headers with arch-specific quirks, but now they’ve all been tidied up and merged together. This should make implementing further standard library headers easier to do (such as <inttypes.h></code></a>).</li>
Finally, many coreboot utilities and tools such as flashrom</a>, coreinfo</a>, nvramtool</a>, inteltool</a>, ifdtool</a>, and cbmem</a> have stricter compiler warnings, mostly from -Wmissing-prototypes</code> and -Wextra</code>. Fixing and adding compiler warnings like these are very easy to do, and make great first time contributions (-Wconversion</code> and -Wsign-compare</code> in particular can always use more work).</li>
</ul>
That’s it for this week! Thankfully Coverity is back online now, so expect more regular blog posts in the future.</p>


Week 4
2019-06-20T00:00:00+00:00
Hello again! If you recall from my last post, the schedule this week is to fix the issues in northbridge/via</code> and southbridge</code>. However, Coverity is going through a major internal upgrade, and so the issue tracker has been offline all week. Luckily though I was able to fix most of these issues last week, so assuming the upgrade finishes soon I won’t be behind schedule. In the mean time, I decided to try flashing coreboot onto my T500, since the last component I was waiting for arrived last week. Here is a little mini-guide to my (sometimes harrowing) flashing experience.</p>
Supplies</h3>

ThinkPad T500</li>
BeagleBone Black</li>
5V 2A power adapter for the BBB</li>
Jumper Wires</li>
Pomona 5252 Test Clip</li>
Atheros AR9462 Wireless Card</li>
</ul>
Updating the EC</h3>
It is generally recommended to update the embedded controller firmware before flashing coreboot, which can only be done during a Lenovo BIOS update. (Unlike Chromebooks, ThinkPads unfortunately do not have open source EC’s.) I was able to find a copy of the latest BIOS on the Lenovo EOL Portal</a>, and attempted to perform an update … which froze and crashed halfway through. Uh oh. This is OK, as long as I don’t restart the computer I can just try flashing it again, right? Wrong! The next time I tried it Windows ran into a fatal error and decided to force a restart for me (gah!). Upon booting it up again, I was met with absolutely nothing, because the screen wouldn’t even turn on. More than a little concerned that I had bricked it, I searched through online forums until I stumbled across the Crisis Recovery</a> tool. Apparently, old ThinkPads have a method to force-update the BIOS from an external USB stick or floppy (if you have one of those lying around). The recovery tool had to be run in Windows XP Service Pack 3 emulation mode, and seemed to format the USB correctly. My ThinkPad wasn’t so impressed, and obstinately refused to recognize the stick. As a last hope, I asked around on IRC what to do, and Nico Huber informed me that the ThinkPad was likely not</em> dead, and that I could just proceed with flashing coreboot anyway. Well, here goes nothing.</p>
Building Coreboot</h3>
So we’re going to flash coreboot, but what options do I pick when compiling it? I scoured around the internet to find tutorials for flashing coreboot onto a T500 and other related ThinkPads, but they all recommended different options, sometimes contradictory. Hmmmm. Once again going back to IRC, Angel Pons helped me configure a very minimal build.</p>
General setup ---> [*] Use CMOS for configuration values
              ---> [*] Allow use of binary-only repository
Mainboard ---> Mainboard vendor ---> Lenovo
          ---> Mainboard model ---> ThinkPad T500
Devices ---> Display ---> Linear "high-resolution" framebuffer
</code></pre>
Now, the T500 is a very special laptop, in that it can run coreboot without any binary blobs at all. However, I decided to enable microcode updates anyway, since they provide important stability improvements (like not crashing). This laptop also comes with an Intel ME which can be completely wiped, but I decided to leave that for later. (Now that I know coreboot works, there will be a follow-up post in several weeks when I do that.)</p>
Disassembly and Flashing</h3>
Like most laptops, the flash IC of the T500 is locked from the factory, and requires an initial external flash to install coreboot (afterwards, subsequent flashes can be done internally). This requires disassembling the laptop to access the SOIC-16, which is buried in the bowels of the T500 case and requires a complete tear-down to access. The Libreboot T500</a> page gives you a feel for the amount of work required to extract the motherboard, which along with the hardware maintenance manual</a> I referred to extensively.</p>

  
  SOIC-16 highlighted in red</figcaption>
</figure>
With the motherboard extracted from the case, the next step is to attach the Pomona 5252 to the SOIC-16 and jumper it to the BBB, which was all made very easy by this X200 guide</a>. Somewhat blithely following the previous guide, I set up an old ATX PSU to provide 3.3v to the flash chip. However, whenever I connected it to the test clip, it would always power itself off. Strange. Going back to IRC, Nico informed me that this is in fact A VERY BAD AND DANGEROUS THING TO DO. THE INTERNET IS LYING—DO NOT USE AN ATX PSU, YOU COULD FRY YOUR MOTHERBOARD! Oops. After puzzling over how to provide enough power to the chip without the PSU, Patrick Rudolph chimed in that a) the T500 motherboard is basically indestructible (whew!), and b) the flasher itself should be able to provide enough power. Hooking the 3.3v cable into the BBB instead, I tried reading the flash chip.</p>
$ flashrom -p linux_spi:dev=/dev/spidev1.0,spispeed=512
(a bunch of output that I forgot to write down)
</code></pre>
It works! Even with a bricked Lenovo BIOS, it is still recommended to keep a backup, so next we read the old factory ROM.</p>
$ flashrom -p linux_spi:dev=/dev/spidev1.0,spispeed=512 -r factory.rom
</code></pre>
Do this three times with three distinct images, and compare their SHAsums to make sure they are all identical (otherwise the connection might be faulty). If they all match, keep one as a backup.</p>
Now the moment of truth: writing coreboot.</p>
$ flashrom -p linux_spi:dev=/dev/spidev1.0,spispeed=512 --ifd -i bios -w coreboot.rom
</code></pre>
Note that because I left the ME as-is, it is important to only flash the BIOS region, not the entire chip.</p>
Reassembly and Testing</h3>
Sadly, no instant gratification here—I had to reassemble half the laptop before I could test booting it up. However, after doing so and gingerly pressing the power button, I was greeted by the lovely SeaBIOS boot menu. It actually worked! Huzzah! Finishing reassembly, I replaced the factory Intel wireless card with an Atheros AR9462, which can run without any binary firmware. After installing Debian, I now have a laptop running completely free and open source software, all the way from the BIOS up (well, except for the ME, but I’ll fix that later).</p>

  
  T500 running coreboot and Debian 9</figcaption>
</figure>
For the final icing on the cake, here is a fresh board status</a> report for the T500. Many thanks to everyone who helped me in this process.</p>


Week 3
2019-06-13T00:00:00+00:00
Hello again! This is a continuation of my posts about fixing the Coverity issues in coreboot. This week’s plan was to tackle the 28 issues in northbridge/intel</code>, which turned out to be much easier than I expected, since I’m already done! With that out of the way, I’m going to begin working on northbridge/via</code> and southbridge</code>. For the curious, here is the project timeline for entire summer.</p>
Week</th> Components</th> Issues</th></tr></thead>

May 6 to 10</td> util</code></td> 22</td></tr>
May 13 to 17</td> util, payloads</code></td> 22</td></tr>
May 20 to 24</td> arch, drivers</code></td> 20</td></tr>
May 27 to 31</td> commonlib, cpu, lib, mainboard</code></td> 22</td></tr>
June 3 to 7</td> northbridge/amd</code></td> 21</td></tr>
June 10 to 14</td> northbridge/intel</code></td> 28</td></tr>
June 17 to 21</td> northbridge/via, southbridge</code></td> 22</td></tr>
June 24 to 28</td> soc/intel</code></td> 21</td></tr>
July 1 to 5</td> soc/rockchip, soc/nvidia</code></td> 20</td></tr>
July 15 to 19</td> soc/misc, vendorcode/cavium</code></td> 26</td></tr>
July 22 to 26</td> vendorcode/amd</code></td> 21</td></tr>
July 29 to Aug 2</td> vendorcode/amd</code></td> 21</td></tr>
Aug 5 to 9</td> vendorcode/amd</code></td> 20</td></tr>
Aug 12 to 16</td> vendorcode/amd</code></td> 20</td></tr>
Aug 19 to 23</td> vendorcode/amd</code></td> 20</td></tr>
</tbody></table>
As you can see, there are a lot</em> of issues in the AMD vendorcode. This consists primarily of AGESA, AMD’s framework for initialization of their 64 bit platforms (somewhat similar to Intel’s FSP). This code is somewhat … dense (someone on IRC described it as a “sea of abstraction”), so I made sure to leave plenty of time for it. As always, you can keep up to date on my current progress on Gerrit</a>.</p>
PS: As an extra bonus, here is a picture of my new BeagleBone Black!</p>

  
</figure>
I recently got a ThinkPad T500 to practice installing coreboot on, and I needed some sort of external programmer to flash the SOIC. There are many options available (flashrom has a whole list</a>), but a single-board computer like this is one of the closest you can get to “plug-and-play.” There are many other popular boards (notably the Raspberry Pi), but the BBB doesn’t require any binary blobs to boot, and is open source hardware</a> too. The only thing I’m waiting for now is an Atheros ath9k wireless card, which runs without any binary firmware. (Hey, if you’re gonna go freedom, you gotta go all the way.)</p>


Introduction
2019-06-05T00:00:00+00:00
Hello everyone! My name is Jacob Garber, and I am a student in this year’s GSoC 2019! My project is on making coreboot Coverity clean. Coverity</a> is a free static-analysis tool for open source projects that searches for common coding mistakes and errors, such as buffer overruns, null pointer dereferences, and integer overflow. Coverity automatically analyzes the coreboot codebase and flags issues it finds, and my job is to classify them into bugs and false-positives and patch them if I can. You can check the Coverity overview</a> for coreboot, though seeing the issue tracker itself requires registration. At the beginning of the summer, coreboot had over 380 flagged issues, but it’s now down to 303, so we’re making progress! I plan to address 20–30 issues per week depending on the source component, which so far has gone surprisingly well. (Surprising, in the sense that coming into the summer I knew very little about coreboot or firmware development in general.) For the curious, you can see the history and progress of all my changes on Gerrit</a>. My mentors for this project are Patrick Georgi, Martin Roth, and David Hendricks, who have all been extremely helpful in guiding me through the development process, reviewing my patches, and answering my many questions. Thank you all.</p>
Now, fixing Coverity bugs isn’t the only thing I’d like to do this summer. As I said before, I’d like to learn more about coreboot, and what better way to do that than installing it on a laptop! My current laptop is an old 2011 Macbook Air, which is surprisingly close to getting coreboot support (many thanks to Evgeny Zinoviev</a>). However, I am (slightly) hesitant about installing yet-experimental firmware on my one and only development machine, so until then I picked up an old Thinkpad T500 to practice on. This laptop has the advantage of being able to run blob-free, and if in the very unlikely event I end up bricking it, who cares! (I mean, I’ll care, but it’ll be a worthy sacrifice.) I also bought a BeagleBone Black</a> to try out external flashing and was hoping to include a picture today, but the shipping was delayed. You’ll have to wait until next week!</p>


qsort vs. std::sort
2016-02-01T00:00:00+00:00
When writing high performance code, there are pretty well two options: C and C++.</p>

C is essentially a portable wrapper around assembly. You can tell exactly how much every action costs, where memory is allocated, and where it is freed. Nothing is hidden. However, because of this direct control, writing C code is very tedious and error prone.</li>
C++ is a bit higher level. You give up some control in exchange for ergonomics. For example, returning a vector</code> from a function is the difference between an O(1) and O(n) operation, depending on if the compiler implements return value optimization. However, C++ templates give the compiler more type information, which it can sometimes use to generate faster code.</li>
</ul>
A classic example of when C++ outperforms C is sorting. In item 46 of his book Effective STL</a>, Scott Meyers claims that std::sort</code> can outperform qsort</code> by up to 670% because C++ will inline the comparison function during template instantiation. Let's see how they compare with sorting a vector of a million random integers. Writing and measuring accurate benchmarks is difficult, so we're going to let the Nonius</a> framework do it for us.</p>
// sort.cpp
#define NONIUS_RUNNER
#include "nonius.h++"
#include <algorithm>
#include <cstdlib>
#include <random>

// qsort
static int comp(const void* const a, const void* const b) {
    const int arg1 = *static_cast<const int*>(a);
    const int arg2 = *static_cast<const int*>(b);

    // we can't simply return a - b, because that might under/overflow
    return (arg1 > arg2) - (arg1 < arg2);
}

// std::sort with no inlining
struct compare_noinline {
    __attribute__((noinline))
    bool operator()(const int a, const int b) const {
        return a < b;
    }
};

// std::sort with inlining
struct compare {
    // the compiler will inline this automatically
    bool operator()(const int a, const int b) const {
        return a < b;
    }
};

static std::vector<int> random_vector(const size_t size) {

    std::random_device seed{};
    std::default_random_engine engine{seed()};
    std::uniform_int_distribution<int> dist{std::numeric_limits<int>::min(),
                                            std::numeric_limits<int>::max()};

    std::vector<int> vec(size);
    std::generate(vec.begin(), vec.end(), [&]{ return dist(engine); });

    return vec;
}

// generate a vector of a million random integers
constexpr size_t size = 1'000'000;
static const auto rand_vec = random_vector(size);

NONIUS_BENCHMARK("qsort", [](nonius::chronometer meter) {

    // Nonius does multiple runs of the benchmark, and each one needs a new
    // copy of the original vector, otherwise we'd just be sorting the same
    // one over and over
    const auto runs = static_cast<size_t>(meter.runs());
    std::vector<std::vector<int>> vectors{runs, rand_vec};

    meter.measure([&](const size_t run) {
        auto& current_vec = vectors[run];
        std::qsort(current_vec.data(), current_vec.size(), sizeof(int), comp);
    });
});

NONIUS_BENCHMARK("std::sort noinline", [](nonius::chronometer meter) {

    const auto runs = static_cast<size_t>(meter.runs());
    std::vector<std::vector<int>> vectors{runs, rand_vec};

    meter.measure([&](const size_t run) {
        auto& current_vec = vectors[run];
        std::sort(current_vec.begin(), current_vec.end(), compare_noinline{});
    });
});

NONIUS_BENCHMARK("std::sort inline", [](nonius::chronometer meter) {

    const auto runs = static_cast<size_t>(meter.runs());
    std::vector<std::vector<int>> vectors{runs, rand_vec};

    meter.measure([&](const size_t run) {
        auto& current_vec = vectors[run];
        std::sort(current_vec.begin(), current_vec.end(), compare{});
    });
});
</code></pre>
Let's compile this bad boy.</p>
# Apple Clang 8.0.0
# Note that Nonius has a dependency on Boost
$ clang++ -std=c++14 -stdlib=libc++ -O3 -march=native sort.cpp -o sort
$ ./sort
</code></pre>
Nonius prints out quite a lot of information, so I'll summarize the most important bits.</p>
qsort                253 ms +/- 8 ms
std::sort noinline   137 ms +/- 6 ms
std::sort inline      88 ms +/- 5 ms
</code></pre>
With no inlining, std::sort</code> is already 1.8x faster than qsort</code>. I am unsure why this is, it could be from differences in algorithms. Inlining bumps this up to 2.9x faster: an impressive speedup.</p>

Week</th>	Components</th>	Issues</th></tr></thead>
May 6 to 10</td>	`util</code></td>`	22</td></tr>
May 13 to 17</td>	`util, payloads</code></td>`	22</td></tr>
May 20 to 24</td>	`arch, drivers</code></td>`	20</td></tr>
May 27 to 31</td>	`commonlib, cpu, lib, mainboard</code></td>`	22</td></tr>
June 3 to 7</td>	`northbridge/amd</code></td>`	21</td></tr>
June 10 to 14</td>	`northbridge/intel</code></td>`	28</td></tr>
June 17 to 21</td>	`northbridge/via, southbridge</code></td>`	22</td></tr>
June 24 to 28</td>	`soc/intel</code></td>`	21</td></tr>
July 1 to 5</td>	`soc/rockchip, soc/nvidia</code></td>`	20</td></tr>
July 15 to 19</td>	`soc/misc, vendorcode/cavium</code></td>`	26</td></tr>
July 22 to 26</td>	`vendorcode/amd</code></td>`	21</td></tr>
July 29 to Aug 2</td>	`vendorcode/amd</code></td>`	21</td></tr>
Aug 5 to 9</td>	`vendorcode/amd</code></td>`	20</td></tr>
Aug 12 to 16</td>	`vendorcode/amd</code></td>`	20</td></tr>
Aug 19 to 23</td>	`vendorcode/amd</code></td>`	20</td></tr> </tbody></table> As you can see, there are a lot</em> of issues in the AMD vendorcode. This consists primarily of AGESA, AMD’s framework for initialization of their 64 bit platforms (somewhat similar to Intel’s FSP). This code is somewhat … dense (someone on IRC described it as a “sea of abstraction”), so I made sure to leave plenty of time for it. As always, you can keep up to date on my current progress on Gerrit</a>.</p> PS: As an extra bonus, here is a picture of my new BeagleBone Black!</p> </figure> I recently got a ThinkPad T500 to practice installing coreboot on, and I needed some sort of external programmer to flash the SOIC. There are many options available (flashrom has a whole list</a>), but a single-board computer like this is one of the closest you can get to “plug-and-play.” There are many other popular boards (notably the Raspberry Pi), but the BBB doesn’t require any binary blobs to boot, and is open source hardware</a> too. The only thing I’m waiting for now is an Atheros ath9k wireless card, which runs without any binary firmware. (Hey, if you’re gonna go freedom, you gotta go all the way.)</p> Introduction 2019-06-05T00:00:00+00:00 Hello everyone! My name is Jacob Garber, and I am a student in this year’s GSoC 2019! My project is on making coreboot Coverity clean. Coverity</a> is a free static-analysis tool for open source projects that searches for common coding mistakes and errors, such as buffer overruns, null pointer dereferences, and integer overflow. Coverity automatically analyzes the coreboot codebase and flags issues it finds, and my job is to classify them into bugs and false-positives and patch them if I can. You can check the Coverity overview</a> for coreboot, though seeing the issue tracker itself requires registration. At the beginning of the summer, coreboot had over 380 flagged issues, but it’s now down to 303, so we’re making progress! I plan to address 20–30 issues per week depending on the source component, which so far has gone surprisingly well. (Surprising, in the sense that coming into the summer I knew very little about coreboot or firmware development in general.) For the curious, you can see the history and progress of all my changes on Gerrit</a>. My mentors for this project are Patrick Georgi, Martin Roth, and David Hendricks, who have all been extremely helpful in guiding me through the development process, reviewing my patches, and answering my many questions. Thank you all.</p> Now, fixing Coverity bugs isn’t the only thing I’d like to do this summer. As I said before, I’d like to learn more about coreboot, and what better way to do that than installing it on a laptop! My current laptop is an old 2011 Macbook Air, which is surprisingly close to getting coreboot support (many thanks to Evgeny Zinoviev</a>). However, I am (slightly) hesitant about installing yet-experimental firmware on my one and only development machine, so until then I picked up an old Thinkpad T500 to practice on. This laptop has the advantage of being able to run blob-free, and if in the very unlikely event I end up bricking it, who cares! (I mean, I’ll care, but it’ll be a worthy sacrifice.) I also bought a BeagleBone Black</a> to try out external flashing and was hoping to include a picture today, but the shipping was delayed. You’ll have to wait until next week!</p> qsort vs. std::sort 2016-02-01T00:00:00+00:00 When writing high performance code, there are pretty well two options: C and C++.</p> C is essentially a portable wrapper around assembly. You can tell exactly how much every action costs, where memory is allocated, and where it is freed. Nothing is hidden. However, because of this direct control, writing C code is very tedious and error prone.</li> C++ is a bit higher level. You give up some control in exchange for ergonomics. For example, returning a `vector</code> from a function is the difference between an O(1) and O(n) operation, depending on if the compiler implements return value optimization. However, C++ templates give the compiler more type information, which it can sometimes use to generate faster code.</li> </ul>`A classic example of when C++ outperforms C is sorting. In item 46 of his book Effective STL</a>, Scott Meyers claims that std::sort</code> can outperform qsort</code> by up to 670% because C++ will inline the comparison function during template instantiation. Let's see how they compare with sorting a vector of a million random integers. Writing and measuring accurate benchmarks is difficult, so we're going to let the Nonius</a> framework do it for us.</p> // sort.cpp #define NONIUS_RUNNER #include "nonius.h++" #include <algorithm> #include <cstdlib> #include <random> // qsort static int comp(const void* const a, const void* const b) { const int arg1 = static_cast<const int>(a); const int arg2 = static_cast<const int>(b); // we can't simply return a - b, because that might under/overflow return (arg1 > arg2) - (arg1 < arg2); } // std::sort with no inlining struct compare_noinline { __attribute__((noinline)) bool operator()(const int a, const int b) const { return a < b; } }; // std::sort with inlining struct compare { // the compiler will inline this automatically bool operator()(const int a, const int b) const { return a < b; } }; static std::vector<int> random_vector(const size_t size) { std::random_device seed{}; std::default_random_engine engine{seed()}; std::uniform_int_distribution<int> dist{std::numeric_limits<int>::min(), std::numeric_limits<int>::max()}; std::vector<int> vec(size); std::generate(vec.begin(), vec.end(), [&]{ return dist(engine); }); return vec; } // generate a vector of a million random integers constexpr size_t size = 1'000'000; static const auto rand_vec = random_vector(size); NONIUS_BENCHMARK("qsort", [](nonius::chronometer meter) { // Nonius does multiple runs of the benchmark, and each one needs a new // copy of the original vector, otherwise we'd just be sorting the same // one over and over const auto runs = static_cast<size_t>(meter.runs()); std::vector<std::vector<int>> vectors{runs, rand_vec}; meter.measure([&](const size_t run) { auto& current_vec = vectors[run]; std::qsort(current_vec.data(), current_vec.size(), sizeof(int), comp); }); }); NONIUS_BENCHMARK("std::sort noinline", [](nonius::chronometer meter) { const auto runs = static_cast<size_t>(meter.runs()); std::vector<std::vector<int>> vectors{runs, rand_vec}; meter.measure([&](const size_t run) { auto& current_vec = vectors[run]; std::sort(current_vec.begin(), current_vec.end(), compare_noinline{}); }); }); NONIUS_BENCHMARK("std::sort inline", [](nonius::chronometer meter) { const auto runs = static_cast<size_t>(meter.runs()); std::vector<std::vector<int>> vectors{runs, rand_vec}; meter.measure([&](const size_t run) { auto& current_vec = vectors[run]; std::sort(current_vec.begin(), current_vec.end(), compare{}); }); }); </code></pre> Let's compile this bad boy.</p> # Apple Clang 8.0.0 # Note that Nonius has a dependency on Boost $ clang++ -std=c++14 -stdlib=libc++ -O3 -march=native sort.cpp -o sort $ ./sort </code></pre> Nonius prints out quite a lot of information, so I'll summarize the most important bits.</p> qsort 253 ms +/- 8 ms std::sort noinline 137 ms +/- 6 ms std::sort inline 88 ms +/- 5 ms </code></pre> With no inlining, std::sort</code> is already 1.8x faster than qsort</code>. I am unsure why this is, it could be from differences in algorithms. Inlining bumps this up to 2.9x faster: an impressive speedup.</p>

Jacob Garber

Bootstrapping a Highly Available Container Registry

Does Kubernetes Work With SELinux?

Blocking Sudo Exploits with fapolicyd

Installing KOReader on a Kobo

The Crushing Complexity of the GPL3

A Basic C Makefile

Final Update

Off-by-One Errors</h3> There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors.</p> </blockquote>

Weeks 11–12

Weeks 8–10

Weeks 5–7

Week 4

Supplies</h3> ThinkPad T500</li> BeagleBone Black</li> 5V 2A power adapter for the BBB</li> Jumper Wires</li> Pomona 5252 Test Clip</li>

Week 3

Introduction

qsort vs. std::sort