Caius' Lab

Making a Digital Clock; An Attempt at Integrating Hardware, Software and 3D Printing

Caius Brindescu — Tue, 09 Jun 2026 00:00:00 +0000

For the last few years, we've been using a Comcast TV Box as a clock for the living room. It hasn't been used for its original purpose in years, but the clock was very useful, so it lived on for this reason alone. To save some space (and power), I decommissioned that box about a month ago to make room for a networking mini-rack.

However, my partner really missed the illuminted clock in the living room. Whatever clocks we had lying around were not illuminated as they were battery powered, so none were a good replacement because of this. To fix this, I decided to try and build one using off the shelf components, and a 3D printed case. The final result can be seen below.

Inspiration and design
The clock was inspired by this 3D Model</a>. However, I wanted a different look, and the project seemed easy enough to try out by myself.
The design goals were, as follows:

Illuminated</li>
Powered externally. I don't need another thing in my life that needs charging.</li>
Self adjusting, so I don't have to worry about setting the correct time and DST changes.</li>
Shows the local temperature; because why not.</li>
Decent aesthetic (at least as far as my design skills go).</li> </ol>
Parts
The display is a 4 module MAX7219, with red LEDs, giving me a resolution of 32x8 pixels. I bought a cheap set of 3 from Amazon</a>, so I had a few tries in case the assembly went wrong. To wire it all permanently, the idea was to use a protoboard to solder everything into one assembly.
Typeface
The goal was to have a typeface that could show all 10 digits, in a 3x6 size. This blog post</a> provides fantastic examples, and the 3x6 examples posted were a great starting point.
Here's the design for the 10 digits that I landed on:
The glyphs are legible, and they cannot be confused with one another. Two of the digits proved to be problematic, in particular 4 and 9. The original design was the following:
However, the 4 looks like a 9 that has lost a pixel, either because it's out, or because I messed up the encoding. The final design of a more traditional 4, and a 9 that's the same glyph as 6, rotated 180 degrees looks a lot better, and leaves no room for confusion. At this resolution, it works well enough.
The 5 used to be a 180 degree rotated variant of the 2 glyph. However, I didn't like the shape of it, so I added one more pixel, and it makes all the difference. You can see here the old and new 5 glyphs, side by side:
UI considerations
Layout
The temperature is left aligned, so under normal weather conditions in the Pacific Northwest, only the left 2 digits are used for the majority of the time. The left digit will be used for a minus sign if the temperatures go negative. Luckily, I don't have to worry about that over here, if I stick to Fahrenheit 1</a>.
Animation
Feedback and error conditions
To avoid having to manually set the time, and to compensate for the known not-so-great quality of the RTC on the board, the clock will synchronize with NTP once hourly 2</a>. When this is happening, the display will not update, as the update is synchronous.
This presents 2 challenges. First, any animation would "freeze" or not work while the sync is in progress. This is an easy fix, as I don't allow update to happen within 10 seconds from the top of the minute. This means that time updates can only happen between seconds 10 and 50 of every minute, avoiding a display freeze in the middle of an animation should the 2 coincide at the same time.
The second issue of the display freezing, is that I will not know if it's frozen because of a problem, or because it's synchronizing. This is fixed, by adding a synchronization symbol to the clock face. The symbol is a "double height" colon. It blends in nicely aesthetically, and doesn't interfere with the main function.
The double height colon also serves as an indicator that the time sync failed on the last attempt. The colon will blink every second (like the normal state). This still allows the clock to function, while signalling that the displayed time might not be accurate. This state will persist until a time sync is successful. When the sync fails, we'll keep retrying every 10 minutes, to minimize the time when we have "degraded time."
The final piece of the puzzle is the temperature. If the clock fails to fetch it, the temperature display will be removed from the clock face until we have data.
The full code for the firmware is up on GitHub</a>.
Case design #</a> </h1>
The case design was probably the more challenging part of the project. First, this is the first integrated electronics project that I have tackled from scratch. Secondly, and crucially, I'm still learning CAD. Having a "full" assembly was a good way to learn a few tricks.
The electronics will be retained in the clock through friction. A few test prints showed that the diffuser part fits snugly enough that this would be fine. Also, since I was using off-the-shelf components in ways they probably never were designed to integrate, coming up with a different mounting solution looked quite daunting.
To aid with 3D printing, the case should also require minimal, to no supports. The end result requires no supports, so I'm happy with the result. The final designed is shared on Printables</a>.
Display Diffuser #</a> </h2>
For the diffuser, one solution was to 3D print a thin 2-3 layer cover that goes over the display. The gaps in the layers would still allow the digits to be seen, while aiding readability by improving contrast. I did a few test prints, however the end result was too dim for my liking, and I had to power the LEDs at full brightness for the contrast to be decent. So I needed something more transparent.
Luckily I had some transparent PETG that I bought years ago, and never opened. Printing a 3 layer diffuser over the display removed the background "noise" of the turned off pixels and allowed most of the light to shine through. The display is still legible in bright daylight, even at the lowest setting, without turning the room red at night.
Integration #</a> </h1>
The final integration was mostly around getting the electronics all sorted out. The most challenging part was removing the existing 90 degree header from the display module, so I could use a straight header to connect to the protoboard that was carrying the MCU.
Once that hurdle was over, the wiring was done on a piece of proto board, and then everything was soldered together as one unit. I didn't have any Kapton tape, so electrical tape had to do to avoid any shortcircuits between the stacked headers.
Finally, at the last minute, I added a 10K potentiometer to control the brightness, without having to reupload the firmware, together with the code to make this work. This will allow me to easily tune the brightness when it's installed.
Below are 2 images: one of the final electronics assembly, and one with the electronics assembled inside the case.
Final results
Conclusions
It was a fun build. It pushed my coding skills, as it's been a while since I've written C++, or worked extensively with microcontrollers 3</a>. Coming up with a parametric design that had 4 total pieces (front bezel, diffuser, main body, and the rear cover) was challenging. It worked out in the end, and only one hole was misaligned!
The end product meets 100% of my original design criteria, and is a fully functional and setup-free clock.
Errata

2026-06-11: Updated the face layout images to use the correct 8 glyph.</li> </ul>

Negative temperatures below 99 degrees (F or C) are intentionally not supported. I think I'll have other things to worry about if that's ever needed. ↩</a> </li>

The once hourly part has been arbitrarily chosen. I did not measure the precision of the on-board RTC, but it seems reasonable that it keep to within a minute for one hour. ↩</a> </li>

An LLM was used to speed up the coding part. ↩</a> </li> </ol> </section>

Backing up Persistent Volume Claims with k8up

Caius Brindescu — Sun, 03 May 2026 00:00:00 +0000

Previously, I wrote about backing up Postgres to S3 using CNPG and Barman Cloud</a>. This is enough if all the data you want to persist is in Postgres. However, if you have any other volumes, this will not be enough. In this post, I'll go over setting up backups with k8up</a>, and what the restore process looks like.
Why k8up?
k8up (pronounced "ketchup") is a Kubernetes Operator distributed via a Helm chart. By default, it backs up all Persistent Volume Claims (PVCs) marked as ReadWriteMany</code>, ReadWriteOnce</code>, or with a certain label. It usesrestic</a>, so it can write to object storage. For this blog post, I'm using Amazon S3 as the destination, but self-hosted options like Garage</a> should also work.
As to why I chose it, it's more mature than VolSync</a>, and I'm interested only in backing up PVCs. The cluster definition is managed with Helm</a>, so Velero</a> seems a bit too heavy for my limited use case.
Finally, there's Longhorn</a>, which can also do backups. My cluster will be quite resource-constrained, so I'm not sure the extra overhead of Longhorn will be worth it in the long run. I'm OK with an "outage" if I need to restore a PVC from S3, and the extra redundancy doesn't seem worth the overhead.
Test service#</a> </h1> To evaluate this, I've created a very simple service that writes the current time to a PVC once a second. It will give me an idea of how backups perform, and how recovery works. Here is the Helm template for this service: apiVersion: v1 kind: PersistentVolumeClaim metadata: name: time-writer-pvc namespace: backup-test labels: must-backup: "true" spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Mi --- apiVersion: v1 kind: Pod metadata: name: time-writer namespace: backup-test spec: containers: - name: time-writer image: busybox:1.37 imagePullPolicy: IfNotPresent command: ["/bin/sh", "-c"] args: - while true; do date >> /data/time.log; sleep 1; done volumeMounts: - name: data mountPath: /data volumes: - name: data persistentVolumeClaim: claimName: time-writer-pvc</code></pre>Backup setup#</a> </h1> Once we have this setup, we can start with the backups. First, we need to create 2 secrets: one for the restic encryption key, and one for the AWS credentials. apiVersion: v1 kind: Secret metadata: name: {{ .Values.clusterName }}-restic-secret-key namespace: backup-test type: Opaque stringData: secretKey: {{ .Values.secrets.resticSecretKey }} --- apiVersion: v1 kind: Secret metadata: name: {{ .Values.clusterName }}-s3-credentials namespace: backup-test type: Opaque stringData: access-key-id: {{ .Values.secrets.awsAccessKey }} secret-key-id: {{ .Values.secrets.awsSecretKey }}</code></pre> Now, we can define the backup we want to take. For s3</code>, I'll use the regional S3 endpoint, s3.us-west-2.amazonaws.com</code>. For the bucket name, we can also specify a prefix. In this example, the restic repo lives under the test</code> prefix. apiVersion: k8up.io/v1 kind: Backup metadata: name: backup-test namespace: backup-test spec: failedJobsHistoryLimit: 2 successfulJobsHistoryLimit: 2 backend: repoPasswordSecretRef: name: {{ .Values.clusterName }}-restic-secret-key key: secretKey s3: endpoint: https://s3.us-west-2.amazonaws.com bucket: bucket-name/test accessKeyIDSecretRef: name: {{ .Values.clusterName }}-s3-credentials key: access-key-id secretAccessKeySecretRef: name: {{ .Values.clusterName }}-s3-credentials key: secret-key-id</code></pre> Once this is done, we can run helm upgrade</code> and we get our backup. Checking the jobs, they have completed: $ kubectl get jobs -n backup-test NAME STATUS COMPLETIONS DURATION AGE backup-schedule-test-backup-c9wz6-0 Complete 1/1 9s 12m</code></pre> And looking at the logs, we get confirmation that it backed up our only file: 2026-05-03T19:45:35Z INFO k8up.restic.restic.backup starting backup 2026-05-03T19:45:35Z INFO k8up.restic.restic.backup starting backup for folder {"foldername": "time-writer-pvc"} 2026-05-03T19:45:35Z INFO k8up.restic.restic.backup.command restic command {"path": "/usr/local/bin/restic", "args": ["backup", "--host", "backup-test", "--json 2026-05-03T19:45:35Z INFO k8up.restic.restic.backup.command Defining RESTIC_PROGRESS_FPS {"frequency": 0.016666666666666666} 2026-05-03T19:45:37Z INFO k8up.restic.restic.backup.progress backup finished {"new files": 1, "changed files": 0, "errors": 0} </code></pre> For the final confirmation, we can see the restic repo structure in S3: $ aws s3 ls s3://bucket-names/test/ PRE data/ PRE index/ PRE keys/ PRE snapshots/ 2026-05-03 12:45:35 155 config</code></pre>Recovery#</a> </h1> A backup is only useful if we can recover our data from it. So let's see how this would work. First, let's create a PVC that's the destination for our data: kind: PersistentVolumeClaim apiVersion: v1 metadata: name: restore-test namespace: backup-test annotations: # Setting this to false to exclude from future backups k8up.io/backup: "false" spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Mi</code></pre> Then, we can define our restore: apiVersion: k8up.io/v1 kind: Restore metadata: name: restore-test spec: restoreMethod: folder: claimName: restore-test backend: repoPasswordSecretRef: name: {{ .Values.clusterName }}-restic-secret-key key: secretKey s3: endpoint: https://s3.us-west-2.amazonaws.com bucket: bucket-name/test accessKeyIDSecretRef: name: {{ .Values.clusterName }}-s3-credentials key: access-key-id secretAccessKeySecretRef: name: {{ .Values.clusterName }}-s3-credentials key: secret-key-id</code></pre> This will create a new job, and checking the logs, we can see that it succeeded: $ kubectl logs -n backup-test jobs/restore-restore-test -f <...truncated...> 2026-05-03T20:04:04Z INFO k8up.restic.restic.restore.command restic command {"path": "/usr/local/bin/restic", "args": ["restore", "be8f37f009f0c910ad5ce8aa067d3dbd4050e7de9c41fde14394555a73adee06:/data/time-writer-pvc", "--target", "/restore"]} 2026-05-03T20:04:04Z INFO k8up.restic.restic.restore.command Defining RESTIC_PROGRESS_FPS {"frequency": 0.016666666666666666} 2026-05-03T20:04:05Z INFO k8up.restic.restic.restore.restic.stdout restoring snapshot be8f37f0 of [/data/time-writer-pvc] at 2026-05-03 19:45:35.978401924 +0000 UTC by @backup-test to /restore 2026-05-03T20:04:05Z INFO k8up.restic.restic.restore.restic.stdout Summary: Restored 1 files/dirs (3.144 KiB) in 0:00</code></pre> Finally, we can check the PVC directly, and confirm we have data until 19:45:36. The backup was started at 19:45:35 and finished by 19:45:37. This gives us a very good RPO of a few seconds, which is great for my use case! $ kubectl exec -n backup-test -it pvc-inspect -- tail /data/time.log Sun May 3 19:45:27 UTC 2026 Sun May 3 19:45:28 UTC 2026 Sun May 3 19:45:29 UTC 2026 Sun May 3 19:45:30 UTC 2026 Sun May 3 19:45:31 UTC 2026 Sun May 3 19:45:32 UTC 2026 Sun May 3 19:45:33 UTC 2026 Sun May 3 19:45:34 UTC 2026 Sun May 3 19:45:35 UTC 2026 Sun May 3 19:45:36 UTC 2026</code></pre>Scheduling#</a> </h1> The above example only covers a "one-time" backup. For this to be useful, I need a regular schedule. This is where the Schedule</code> comes in. I'll start with the example provided in the documentation</a>. We'll back up every 5 minutes, and every hour we'll do checking and pruning. Data retention can be tuned depending on the use case. apiVersion: k8up.io/v1 kind: Schedule metadata: name: schedule-test spec: backend: repoPasswordSecretRef: name: {{ .Values.clusterName }}-restic-secret-key key: secretKey s3: endpoint: https://s3.us-west-2.amazonaws.com bucket: bucket-name/test accessKeyIDSecretRef: name: {{ .Values.clusterName }}-s3-credentials key: access-key-id secretAccessKeySecretRef: name: {{ .Values.clusterName }}-s3-credentials key: secret-key-id backup: schedule: '*/5 * * * *' failedJobsHistoryLimit: 2 successfulJobsHistoryLimit: 2 check: schedule: '0 * * * *' prune: schedule: '30 * * * *' retention: keepLast: 5 keepDaily: 7</code></pre> This will result in one job every 5 minutes: $ kubectl get jobs -n backup-test NAME STATUS COMPLETIONS DURATION AGE backup-schedule-test-backup-7l7sj-0 Complete 1/1 10s 3m17s backup-schedule-test-backup-l7jkq-0 Complete 1/1 10s 8m17s</code></pre> And for each job, the backup was successful: 2026-05-03T21:40:29Z INFO k8up.restic.restic.backup starting backup 2026-05-03T21:40:29Z INFO k8up.restic.restic.backup starting backup for folder {"foldername": "time-writer-pvc"} 2026-05-03T21:40:29Z INFO k8up.restic.restic.backup.command restic command {"path": "/usr/local/bin/restic", "args": ["backup", "--json", "--host", "backup-test", "/data/time-writer-pvc"]} 2026-05-03T21:40:29Z INFO k8up.restic.restic.backup.command Defining RESTIC_PROGRESS_FPS {"frequency": 0.016666666666666666} 2026-05-03T21:40:31Z INFO k8up.restic.restic.backup.progress backup finished {"new files": 0, "changed files": 1, "errors": 0} 2026-05-03T21:40:31Z INFO k8up.restic.restic.backup.progress stats {"time": 1.727354075, "bytes added": 51912, "bytes processed": 50837}</code></pre>Conclusion#</a> </h1> Overall, k8up seems to meet my use case. The setup is straightforward and easy to reason about. The restore process is also straightforward. While the example here is very limited, it gives me confidence that the basics work. The real test will come once it's operating on real data, but regular restores and testing should give me the confidence I need there.
Setting up a Homelab: Step 1 Caius Brindescu — Sun, 19 Apr 2026 00:00:00 +0000 I am building a Kubernetes cluster out of second-hand Lenovo ThinkCenter machines. As part of this project, I want to streamline the setup as much as possible, and I'll try to automate most steps. The goal is to have a reproducible setup that will simplify modifications, adding more hardware, etc. In this post, I will go over what I call the "bootstrap step." How do you start with a fresh install of an OS, as easily as possible without having to "manually" install it from a USB stick on each machine? The solution is to create a pre-configured NVMe drive, from a cloud OS image, and set up cloud init</a> to do the configuration that we need. Theory of Operation#</a> </h1> I am running this process on macOS. This comes with a few extra complications, compared to running this from a Linux machine1</a>. Should this be run from Linux, as it makes life easier? Probably, but where's the fun in that. </blockquote> Before I dive into the details, let's look at what needs to be done: First, we need to write the image on the NVMe drive. That's easy enough. However, I want the machines to be headless, so I don't have any inputs or output connected. Ideally when they boot, they are accessible from the network, and ready to set up. This is where cloud-init comes in. Cloud-init runs on a cloud VM during first boot, to handle machine-specific configuration. That's our exact use case! It reads its configuration from a fairly straightforward yaml configuration format</a>. While we aren't running in "the cloud," the abstraction makes sense, and cloud-init is flexible enough to work on a bare-metal machine. The goal of the setup is to do 2 things: Set up SSH key for remote access</li> Set up a hostname, so we can easily find it on the network using DNS.</li> </ol> Step 1: Writing the image#</a> </h1> First, we need to choose an OS that we want to run. For my setup I chose to run Ubuntu 25.10, the latest at the time of writing. You can find releases on this page.</a> Once we have that, we can't write that directly to the NVMe drive. They are in the QEMU Qcow2 image format (they are designed for cloud use). We'll need to convert them to raw. We'll do this with qemu2</a>. qemu-img convert -f qcow2 -O raw ubuntu-25.10-server-cloudimg-amd64.img ubuntu-25.10-server-cloudimg-amd64.raw</code></pre> Now we have an image we can write to the NVMe drive. We'll use Ansible</a> to automate the whole process, so it's easily repeatable. Doing any of these following steps on the wrong disk WILL wipe out that disk, and you will lose any and all stored data! Double check all paths, and your assumptions! Proceed at your own risk! </blockquote> We'll assume that the following variables are defined: target_disk: /dev/disk2</code> - the buffered block device that our NVMe is mounted to</li> target_raw_disk: /dev/rdisk2</code> - the unbuffered block device for faster writes using dd</code></li> </ul> First, we need to make sure that the drive is unmounted and ready to be written to: - name: Unmount disk ansible.builtin.command: cmd: diskutil unmountDisk {{ target_disk }}</code></pre> Then we can copy the image to the NVMe drive: - name: Write image to disk ansible.builtin.command: cmd: dd if={{ image_src }} of={{ target_raw_disk }} bs=4m</code></pre> We now have an NVMe drive that will boot on our machine. However, it doesn't have the configuration needed for headless remote access. So we'll have to tackle that next, and this is where cloud-init comes into play. Step 2: Cloud-init configuration#</a> </h1> Cloud-init has different ways in which it can get its initial scripts. On a cloud machine, it's usually through some kind of metadata service. For example, on AWS it's from http://169.254.169.254/latest/</code>. However, this isn't a cloud machine, so we'll have to do something different. Luckily, Cloud-init supports the NoCloud</code> datasource</a>. This allows us to get the user data from a specific partition called cidata</code> or CIDATA</code>. This is the option we'll go with next3</a>. Creating the partition#</a> </h2> Since I'm managing a Linux file system, I decided to use gptfdisk</code>4</a> for all the partition management. First, we need to make sure that we see the whole disk, otherwise we'll have no new space to create the new partition. - name: Make GPT see the full disk ansible.builtin.command: cmd: sgdisk --move-second-header {{ target_disk }}</code></pre> The default root (/</code>) partition will expand to fill all the available space, so we don't want to create our new partition immediately after. This would prevent the partition from growing. So we'll have to create the partition at the end. For this example, I chose a size of 50MB. This is probably overkill for the 2 tiny files we're writing, but it gives room to grow if we need to. - name: "Add CIDATA partition at the end" ansible.builtin.command: cmd: sgdisk --new=0:-51M:+50M --typecode=0:0700 --change-name=0:CIDATA {{ target_disk }}</code></pre> Next, we'll format it as FAT32, and mount it: - name: "Format CIDATA to FAT32" ansible.builtin.command: cmd: diskutil eraseVolume MS-DOS CIDATA {{ target_disk }}s2 - name: Mount CIDATA partition ansible.builtin.command: cmd: diskutil mount {{ target_disk }}s2</code></pre> At the end, if you run diskutil</code>, you'll get something like this, if all went well. /dev/disk2 (external, physical): #: TYPE NAME SIZE IDENTIFIER 0: GUID_partition_scheme *256.1 GB disk2 1: BC13C2FF-59E6-4262-A352-B275FD6F7172 1.1 GB disk2s13 2: Bios Boot Partition 4.2 MB disk2s14 3: EFI UEFI 111.1 MB disk2s15 4: Linux Filesystem 254.8 GB disk2s1 5: Microsoft Basic Data CIDATA 52.8 MB disk2s2</code></pre>Writing the cloud-init config#</a> </h2> Now, we're ready to write our cloud-init files. We'll need two of them: meta-data</code> for setting the hostname</li> user-data</code> for setting up SSH access.</li> </ul> - name: Write cloud-init meta-data ansible.builtin.copy: content: | local-hostname: {{ target_hostname }} dest: /Volumes/CIDATA/meta-data - name: Write cloud-init user-data ansible.builtin.copy: content: | #cloud-config users: - name: ubuntu ssh_authorized_keys: {% for key in public_keys -%} - {{ key }} {% endfor -%} sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/bash dest: /Volumes/CIDATA/user-data become: true</code></pre> We'll unmount, eject, and we're done. - name: Unmount disk ansible.builtin.command: cmd: diskutil unmountDisk {{ target_disk }}s2 - name: Eject disk ansible.builtin.command: cmd: diskutil eject {{ target_disk }}</code></pre> To create our new configured boot disk, all we need to do is run: ansible-playbook playbooks/ubuntu-start.yml --ask-become-pass -e "target_hostname=machine1"</code></pre> This makes setting up multiple machines for the same cluster really easy, as we only need to change the target_hostname</code> variable when running the playbook. Conclusion#</a> </h1> In this post, I've detailed how to create a bootable drive for a new PC, that will boot up, and configure itself with the correct hostname and SSH keys for remote access. Using cloud-init allows us to specify a custom bootstrap step for each machine, so we can use the same "base" image in all cases. This has greatly simplified adding new machines to the cluster, or reprovisioning old ones if we replace the SSD. To state the obvious, this would be a lot simpler if done using Linux, as we could mount the root file system, and make our changes there. In that case, using cloud-init is probably superfluous, and this post would be 1/4 of the size it is now. However, using cloud-init gives us more flexibility on what's run. More importantly, it presented a great opportunity for experimenting with a novel (to me) way of bootstrapping Linux on a bare-metal headless machine. Appendix#</a> </h1> The full playbook: - name: Flash Ubuntu disk image hosts: localhost become: true vars: image_src: /tmp/ubuntu-25.10-server-cloudimg-amd64.raw target_raw_disk: /dev/rdisk2 target_disk: /dev/disk2 target_hostname: ubuntu public_keys: - ssh-rsa AAAAB3N.... tasks: - name: Check running on macOS ansible.builtin.assert: that: ansible_facts['os_family'] == "Darwin" fail_msg: "This playbook can only be run on macOS" - name: Unmount disk ansible.builtin.command: cmd: diskutil unmountDisk {{ target_disk }} - name: Write image to disk ansible.builtin.command: cmd: dd if={{ image_src }} of={{ target_raw_disk }} bs=4m - name: Make GPT see the full disk ansible.builtin.command: cmd: sgdisk --move-second-header {{ target_disk }} - name: "Add CIDATA partition at the end" ansible.builtin.command: cmd: sgdisk --new=0:-51M:+50M --typecode=0:0700 --change-name=0:CIDATA {{ target_disk }} - name: "Format CIDATA to FAT32" ansible.builtin.command: cmd: diskutil eraseVolume MS-DOS CIDATA {{ target_disk }}s2 - name: Mount CIDATA partition ansible.builtin.command: cmd: diskutil mount {{ target_disk}}s2 - name: Write cloud-init meta-data ansible.builtin.copy: content: | local-hostname: {{ target_hostname }} dest: /Volumes/CIDATA/meta-data - name: Write cloud-init user-data ansible.builtin.copy: content: | #cloud-config users: - name: ubuntu ssh_authorized_keys: {% for key in public_keys -%} - {{ key }} {% endfor -%} sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/bash dest: /Volumes/CIDATA/user-data become: true - name: Unmount disk ansible.builtin.command: cmd: diskutil unmountDisk {{ target_disk }}s2 - name: Eject disk ansible.builtin.command: cmd: diskutil eject {{ target_disk }}</code></pre>Errata#</a> </h1> 2026.06.09 - Updated Ansible snippet and appendix playbook to fix hard-coded reference to /dev/disk2</code> in the "Mount CIDATA partition" task.</li> </ul> You can't reliably mount an ext4 file system</a>, and using cloud-init gets around this limitation. ↩</a> </li> On a Mac, you can install the right package with brew install qemu</code> if you're using Homebrew</a>. On a Debian-based Linux distro, you can use apt install qemu-utils</code>. ↩</a> </li> There's also the option of passing in kernel arguments, and using a service on the local network. However, since mounting ext4</code> partitions is not a feasible option, I've decided against this approach. ↩</a> </li> You can install it with brew install gptfdisk</code> ↩</a> </li> </ol> </section> Why I Self Host Caius Brindescu — Sat, 28 Mar 2026 00:00:00 +0000 I have been running my own servers, on and off since college. They have been a great experimentation platform, and given me resources to tap into for my professional life. It wasn't always smooth sailing, but here's some of my takes, and why it's always been worth the hassle, at least for me. A short history#</a> </h1> The early days#</a> </h2> The first "server" I built was during college when I took an old desktop, put Linux on it, and used it mostly as a network file store. No redundancy, but it was fun. Later on during grad school, I had to setup the research group server1</a>. Using DynDNS</a>2</a>, it was publicly accessible. That worked fine for a while, until a vulnerable Apache module turned it into a spam bot. I only found out when I got an annoyed email from a sysadmin in Bucharest, Romania, that it was spamming their network. Took it down for the day, cleaned it up, and made a note to apply patches more often in the future. Today#</a> </h2> Currently, I am running a Pi Cluster with 4 Raspberry Pi 4 Nodes, with 4 GB each, running K3s</a>. While not a very powerful cluster, and only using SD Card for storage, it's enough to serve a static website, grafana, prometheus, and any random experiments. I also have a 1 node Framework Desktop that will run more "critical" systems, like a Postgres database for services that need it. Why?#</a> </h1> A lot of the services I run (or plan to) are usually available for free (or a reasonable subscription) else where. You can host a website cheaply on AWS with S3 and Cloudfront (assuming low traffic). You can host your 3D prints on Printables and the like, and I do. But there are a few advantages to having control of where your data ends up. Learning#</a> </h2> This has been the primary reason for this. The Pi Cluster in particular was a great learning experience. Testing a repeatable Ansible set of playbooks, getting K3S to work and deploying a few services using both plain Kubernetes templates and Helm is not something I get to do in my day job. I find that I learn best when working on something practical. This is true from programming languages to infrastructure and networking. Having a "disposable" setup makes this a lot easier when learning. Enshittification#</a> </h2> It's an unfortunate reality of today's world. Free services get more restrictive, open source alternatives get yanked3</a>, or the prices get constantly increased, with "AI" as the excuse4</a>. Having your own services protects from this reality, and you don't need to worry as much about increased costs, lost access, etc. Private access#</a> </h2> From the grad school days, I've learned to be wary when exposing anything over the internet. To this end, the cluster is currently private and I access it from anywhere using Tailscale. The free tier5</a> is enough for me and my partner, and we can access the services from anywhere. Of course, locally we can always use the local network. Using Cert Manager</a>, I have publicly valid certificates and DNS that points to the Tailscale IP address of a node in the cluster. This works great, and we don't get any annoying browser warnings. Making everything public is only a matter of updating the DNS records. Your data, your rules#</a> </h2> Keeping the data private means it's not going to be used to train AI models, and target ads at you. I'd rather have my data truly private, not private with an asterisk and a disclaimer, as the service can still read it but promises to play nice. The downsides#</a> </h1> Reliability#</a> </h2> My setup is only as reliable as my internet and power are. A UPS helps with the power, but no backup to the internet connection. If Comcast is being its usual self, we'll get a few periods of downtime a week. For now this is not an issue, as the total downtime is less than 10 minutes a week, and mostly at night. Data safety#</a> </h2> If anything were to happen to the hardware, well, the data is gone. That is solved by having automatic backups to S36</a>, and having a good 3-2-1 backup policy</a> is critical. Also, test your backups regularly. You don't want to find that your backups are broken when you really need them. Cost#</a> </h2> In particular, the upfront costs of getting the hardware. This is only getting more expensive these days7</a>. However, if you've got some old hardware lying around, or go the used option, it's going to be less of a concern. Security#</a> </h2> If the spambot incident taught me anything, it's that if it's open to the internet it's a target. Data leaks could be more serious if it has private data, and you're the only one in charge and responsible for securing it. Keeping up to date with patches becomes even more critical the more sensitive the stored data. A solution like Tailscale sidesteps the issue by keeping everything private. Conclusion#</a> </h1> Is this for everyone? No. But if you want to learn, or care about where data ends up living, this is something worth looking into. In our living room, because the university policies for a "hard-wired" server were beyond draconian, and made the hardware useless for the research we were doing ↩</a> </li> It was free in those days ↩</a> </li> Minio archiving their open source repo</a> ↩</a> </li> Google Workspace price increase due to new AI features</a> ↩</a> </li> I am aware that this could enshittify in the future. However, Headscale</a> is an option, so I'm confident there's an offramp if I need it ↩</a> </li> For data durability, I haven't found anything that beats S3, or other key store offerings by a large cloud vendor. ↩</a> </li> RAM price hikes</a> ↩</a> </li> </ol> </section> Self Hosted Postgres in Kubernetes with PITR Recovery Caius Brindescu — Wed, 31 Dec 2025 00:00:00 +0000 Introduction#</a> </h1> I would like to selfhost a few services, and they require persistence, in the form of a PostgreSQL database. Since my Kubernetes cluster runs on 4 Raspberry Pi 4, each with an SD card for disk, it's only a matter of time until one of them gets corrupted. To avoid the innevitable and predictable data loss, I'll need automated backups, idealy with Point-In-Time Recovery (PITR). If I'm going to reinvent the wheel, why not go for the whole wheel? Solution#</a> </h1> CloudNativePG</a> seems to be most mature and feature complete out there. It provides an operator that does a lot of the heavy lifting. Backups are handled by the Barman Cloud</a> plugin. It handles aspects like WAL Log archiving, taking regular snapshots and uploading them a cloud object storage, like S31</a>. Seems to meet all the needs I need for my project. Setup#</a> </h1> Setting up a Cloud Native PG database#</a> </h2> Using the instructions on their website</a>, we'll need to install the operator. We can do it with the following command: $ kubectl apply --server-side -f https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.28/releases/cnpg-1.28.0.yaml</code></pre> We can now create our database. The Barman Cloud Plugin we'll be using later assumes that the DB is in the cnpg-system</code> namespace. To simply this experiment, we'll work with that assumption, and we'll start by creating the namespace: $ kubectl create namespace cnpg-system</code></pre> Then we can define our cluster configuration, with one user so we can connect to it later: apiVersion: v1 type: kubernetes.io/basic-auth kind: Secret metadata: name: postgres-secret namespace: cnpg-system stringData: username: user1 password: supersecretpassword --- apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: test namespace: cpgn-system spec: instances: 1 storage: size: 1Gi</code></pre> And when we apply this, we have a new cluster with 1 instance. We can forward the port, and connect to it like we normally would to a Postgres instance running locally. $ kubectl port-forward -n cnpg-system service/test-restore-rw 5432:5432</code></pre>Backups#</a> </h2> Installing#</a> </h3> Now, we can configure backups for our new Postgres cluster. First, let's install the prerequisites that the Barman Cloud plugin requires. We will need to install Certificate Manager2</a>. $ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.2/cert-manager.yaml</code></pre> Then, we can install the plugin: $ kubectl apply -f https://github.com/cloudnative-pg/plugin-barman-cloud/releases/download/v0.9.0/manifest.yaml</code></pre>Configuration#</a> </h3> We can now define the backup configuration. For this example, I'll use S3 as the storage target for the backups. First, we'll need to store a set AWS credentials with access to the right bucket3</a>. We'll use an Opaque Secret</a> to store the access and secret keys: apiVersion: v1 kind: Secret metadata: name: s3-credentials namespace: cnpg-system type: Opaque stringData: access-key-id: <redacted> secret-key-id: <redacted></code></pre> Next, we'll define the storage configuration to our S3 Bucket: apiVersion: barmancloud.cnpg.io/v1 kind: ObjectStore metadata: name: s3-store namespace: cnpg-system spec: configuration: destinationPath: s3://<bucket>/postgres/ s3Credentials: accessKeyId: name: s3-credentials key: access-key-id secretAccessKey: name: s3-credentials key: secret-key-id wal: compression: gzip</code></pre> Finally, we'll need to tell our Postgres cluster to use this configuration, and enable WAL archiving. We'll add the following to the Cluster spec</code> field: plugins: - name: barman-cloud.cloudnative-pg.io isWALArchiver: true parameters: barmanObjectName: s3-store</code></pre> The final piece of the puzzle is setup regular "base" backups. These will backup the entire dataset, and give us a "base" for the WAL logs to be applied to in order to get our Point-In-Time Restore. apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: pg-backup namespace: cnpg-system spec: cluster: name: test schedule: '0 0 * * * *' # hourly backupOwnerReference: self method: plugin pluginConfiguration: name: barman-cloud.cloudnative-pg.io</code></pre>Results#</a> </h3> Now that everything is configured, we can list our S3 bucket, and sure enough, we have backups. The top level structure has our base and WAL logs: $ aws s3 ls s3://<redacted>/postgres/test/ PRE base/ PRE wals/</code></pre> Digging in deeper, we have our base backups, nicely named by timestamp, one every hour, as we'd expect: $ aws s3 ls s3://<redacted>/postgres/test/base/ PRE 20251227T030000/ PRE 20251227T040000/ PRE 20251227T050000/ PRE 20251227T060000/ PRE 20251227T070000/ ...</code></pre> And finally, we have our WAL logs archived: $ aws s3 ls s3://<redacted>/postgres/test/wals/0000000100000000/ 2025-12-26 21:00:02 16944 000000010000000000000013.gz 2025-12-26 21:00:04 210 000000010000000000000014.00000028.backup.gz 2025-12-26 21:00:03 16513 000000010000000000000014.gz 2025-12-26 21:05:03 17139 000000010000000000000015.gz 2025-12-26 21:30:03 17276 000000010000000000000016.gz 2025-12-26 22:00:02 16416 000000010000000000000017.gz 2025-12-26 22:00:05 209 000000010000000000000018.00000028.backup.gz 2025-12-26 22:00:04 16510 000000010000000000000018.gz ...</code></pre> So we should have all the pieces needed to perform a PITR. We'll tacke this in the next section. Restoring#</a> </h2> For testing the restore, I've created a simple table, with timestamps for easy reasoning: CREATE TABLE test( id int, created_at timestamp(6) );</code></pre> And we inserted different values, and this is the end state of the table: test=> select * from test; id | created_at ----+---------------------------- 16 | 2025-12-27 03:00:17.601392 17 | 2025-12-27 03:00:20.438822 18 | 2025-12-27 03:28:06.64914 19 | 2025-12-27 20:42:25.563207 20 | 2025-12-27 20:42:32.992066 21 | 2025-12-27 20:45:58.634257 (6 rows)</code></pre> We'll restore the table to 2025-12-27, at 20:43:00 UTC. For this, we'll create a new cluster, and we'll point it at the backups we have. We'll also need to give it the target time we want the cluster restored to. We arrive at this configuration: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: test-restore namespace: cnpg-system spec: instances: 1 imagePullPolicy: IfNotPresent bootstrap: recovery: source: source externalClusters: - name: source plugin: name: barman-cloud.cloudnative-pg.io parameters: barmanObjectName: s3-store serverName: test targetTime: "2025-12-27T20:43:00Z" storage: size: 1Gi</code></pre> Once the restore is done (it was pretty much instant for this example), we con connect and check our test table: test=> select * from test; id | created_at ----+---------------------------- 16 | 2025-12-27 03:00:17.601392 17 | 2025-12-27 03:00:20.438822 18 | 2025-12-27 03:28:06.64914 19 | 2025-12-27 20:42:25.563207 20 | 2025-12-27 20:42:32.992066 (5 rows)</code></pre> As we'd expect, we're missing row with id</code> 21, as it's after the target restore time. Conclusions and Final Remarks#</a> </h1> All in all, this was an easy setup, and it works fine for at least the basic use case. The true test is once this sees some "production" loads, and testing with an actual live data base. A note on S3 cots#</a> </h2> For this example, I used S3 as the backup target destination. The amount of data stored is small, however, WAL archiving could end up writing a lot of objects. This could incur significant S3 API charges, so it's something I'm keeping an eye on. With hourly base backups, I didn't see any cost increases for my AWS account. But inserting 21 records, and deleting a few is not exactly a representative use case, but at least the "baseline" cost is not absurd. Retention policies#</a> </h2> Barman has the option of specifying retention policies</a>. However, for this experiment, I've gone with specifying a lifecycle policy on the S3 bucket. Everything under postgres/</code> will be deleted after 7 days. This will give me a one week recovery window. Using the Barman retention policy will cause the plugin to list S3, and then delete the objects. This also will incur some S3 API charges, and I think that using the S3 lifecycle rule is probably good enough. The plugin also supports Google Cloud Storage, or Azure Blobs or some other services that implementation a compatible API</a>. ↩</a> </li> You can skip this step if your cluster already has it installed on the cluster. My test cluster did not. ↩</a> </li> For reference, I granted the user full access to the S3 bucket where the backups are stored. ↩</a> </li> </ol> </section> Setting up the DX Commander Expedition Antenna Caius Brindescu — Mon, 21 Apr 2025 00:00:00 +0000 Recently I got a new DXCommander Expedition Antenna</a> for Parks on the Air (POTA) use. After some experimentations, this week I decided to finish the tuning, and make a few contacts with it Initial thoughts#</a> </h2> It's a large antenna! See in context with a large pine tree. It took a while to set up, with a couple of saw horses it's not too bad. Having the antenna at a reasonble height avoids all be bending over to cut and measure things. Once it was clear where everything went, and had the clamps trimmed to size, the assembly went smooth. I also tried tryting to extend the antenna while it's vertical. I don't recomment this approach. It doesn't work and it will come crashing down and "colapse" quite fast. The best option for deploying in the field is to lay it down, assemble everything, then walk it up. It's light enough that it's easy to do with a single person. The good#</a> </h2> The tuning! 40 and 20 meters are spot on, with barely any noticeable movement of the SWR meter while transmitting. 20 meters is well below 2.0 SWR across the entire band. For 40 meters, the story is the same. The 40 meters element ended up a lot longer than I expected. So long, in fact, that I don't need any shockcord at the top. It reaches the top of the pole, and I use the little clear platic tubing to hold the element in place. A loop of shockcork keeps me from loosing the little plastic tube. I opted for a "perfect" 40 meter tune, and this probably lead to the long element, and the not so great 15 meter performance (more on that later). The OK#</a> </h2> Tuning the 10 meter band was a bit more finicky than I liked. I was able to get it under 3.0 across the whole band, and I was able to use it fine without using the internal ATU of the radio. (Sorry, no image of the 10 meter turning. I closed the software and didn't save the image, and I only realized this after the antenna was down. I'm not going to stand it back up, in the rain, for one image. You'll have to trust me on this one) The meh#</a> </h2> 15 meters. It's tuning is not great, see below. In practice, I get under 2.0 with the Icom 7300 (very brief testing). It's usable without an ATU, and well within ATU limits if I'm using a radio that has one built in. It's possible the "overtunning" the 40 meter band got me worse performance here. But it's a compromise I made on purpose. I use 40 meters a lot more often than 15, and I'm happy with the overall result. Also, the elements are a bit loose than I'd like. The tensioning loops ended up very close to the spreader plates. This might be my error in assembly, but for future reference, having the loops a few inches down is OK, as there's pleny of shockcord to accomodate this. Conclusions#</a> </h2> It's a great antenna! It's larger than I expected. But it did deliver the performance I was expecting. It's now in the back of the truck, and I'm looking forward to trying out in the field on the next POTA activation! Decoding Meshtastic Channel Links Caius Brindescu — Sat, 25 Jan 2025 00:00:00 +0000 Introduction#</a> </h2> If you don't know what Meshtastic is, this video has a great introduction</a>. The TL;DR: is that it's a communication protocol using 900 MHz (in the US) LORA Radios</a>. Communication is done either via direct messages (DMs), or via channels. The channels are encrypted, so for 2 parties to communicate, they need to know the encryption key (pre-shared key or PSK, for short) for that channel. Sharing the chaneel name, key etc. is done via QR codes or URLs</a>. In this blog post, we'll look at what information is encoded when sharing channel settings, and we can decode it using a "simple" Python script. The channel link#</a> </h2> We'll use the following link as an example: https://meshtastic.org/e/#CjQSIOsfCkgIpGY_8iW02ad-4QPaCSBISJqzIVoZKHdqKXd8GgtCbG9nQ2hhbm5lbCUEAAAAEg4IATgBQANIAVAeWBRoAQ</code></pre> The part that we are interested in is the fragment: CjQSIOsfCkgIpGY_8iW02ad-4QPaCSBISJqzIVoZKHdqKXd8GgtCbG9nQ2hhbm5lbCUEAAAAEg4IATgBQANIAVAeWBRoAQ</code></pre> This is a URL safe base64</a> encoded string that has the information we are looking for. Structure#</a> </h2> The message is encoded using protobuf</a>. The structure we are interested in is defined in this file</a>. It's reasonably well documented, so I will not get into all the details here. But the main properties we are interested in are the channel name, and the PSK. They are encoded as the psk</code> and name</code> properties in the top level ChannelSettings</code> message. For the rest of this post, we'll ignore the remainder of the properties, but they can be extracted using the same approach. Protobuf and Python#</a> </h2> Generating Python code from the protobuf structure#</a> </h3> The first setup is installing all the prerequisites. There are 2 parts: the protobuf compiler and the Python protobuf library. sudo apt install protobuf-compiler pip install protobuf==3.20.1</code></pre> Note: I recommend using a virtual enviroment</a> for your python setup. Here's a good resource for setting it up</a>. </blockquote> Next, we'll need to get the protobuf definition "compiled" into python code, so we can the deserialize the configuration from the URL. The protobuf definition lives in GitHub</a>, so we'll need to clone it. git clone git@github.com:meshtastic/protobufs.git</code></pre> We'll need to compile a total of four proto files to get this working. First, let's create a new folder for our generated files to live in: mkdir meshtastic</code></pre> Then, cd</code> into the protobuf</code> folder, and run the following commands. (You can compile all of them if you want to, but it's not needed for this use case.) protoc --python_out=.. meshtastic/apponly.proto protoc --python_out=.. meshtastic/channel.proto protoc --python_out=.. meshtastic/device_ui.proto protoc --python_out=.. meshtastic/config.proto</code></pre> Finally, we need to make the meshtastic</code> folder a python "module", we we'll create an empty __init__.py</code> file inside: touch meshtastic/__init__.py</code></pre>Decoding#</a> </h3> How that have all the prerequisites, let's get decoding! To start with we'll need to import the required pacakges: from meshtastic.apponly_pb2 import ChannelSet import base64</code></pre> We'll see the string we want to decode as a constant (for now): TO_DECODE = 'CjESIMC70tNI5vkpQpHZGeg0WV7y6KqEoD0_t74fM1_jCaMkGghCbG9nVGVzdCUEAAAAEg4IATgBQANIAVAeWBRoAQ'</code></pre> Next, we'll create a new, empty object to populate with the deserialized data: channelSet = ChannelSet()</code></pre> Finally, let's decode and display the channel name and PSK: channelSet.ParseFromString(base64.urlsafe_b64decode(TO_DECODE + "==")) for s in channelSet.settings: print("Channel name: " + s.name) print("Psk: " + base64.b64encode(s.psk).decode())</code></pre> A few notes on the implementation. The settings are encoded in a "URL Safe" base64 encoding. However, Python still expects the padding to be there, so we're adding the maximum of 2 padding characters to get around this. If that's too many, any extras will be ignored by the decoder. "Proper" URL Safe encoding explictly omits padding, but the Python library still requires it, so here we are. Results#</a> </h2> Running the above code, we'll get the settings we expect: $ python decode.py Channel name: BlogTest Psk: wLvS00jm+SlCkdkZ6DRZXvLoqoSgPT+3vh8zX+MJoyQ=</code></pre> We can extract other properties if we want to, using the same approach. The End#</a> </h2>

Conclusion

Step 2: Cloud-init configuration
`Luckily, Cloud-init supports the` `NoCloud</code> datasource</a>. This allows us to get the user data from a specific partition called cidata</code> or CIDATA</code>. This is the option we'll go with next3</a></sup>.</p>`

Cost
In particular, the upfront costs of getting the hardware. This is only getting more expensive these days 7</a></sup>. However, if you've got some old hardware lying around, or go the used option, it's going to be less of a concern.</p>

Conclusions

Results
`$ python decode.py </span></span> Channel name: BlogTest</span></span> Psk: wLvS00jm+SlCkdkZ6DRZXvLoqoSgPT+3vh8zX+MJoyQ=</span></span></code></pre> We can extract other properties if we want to, using the same approach.</p>`
The End

The End

Caius' Lab

Making a Digital Clock; An Attempt at Integrating Hardware, Software and 3D Printing

Parts
The display is a 4 module MAX7219, with red LEDs, giving me a resolution of 32x8 pixels. I bought a cheap set of 3 from Amazon</a>, so I had a few tries in case the assembly went wrong. To wire it all permanently, the idea was to use a protoboard to solder everything into one assembly.</p>

Backing up Persistent Volume Claims with k8up

Setting up a Homelab: Step 1

Why I Self Host

Self Hosted Postgres in Kubernetes with PITR Recovery

Setting up the DX Commander Expedition Antenna

Decoding Meshtastic Channel Links

Caius' Lab

Making a Digital Clock; An Attempt at Integrating Hardware, Software and 3D Printing

Backing up Persistent Volume Claims with k8up

Setting up a Homelab: Step 1

Writing the cloud-init config#</span></a> </h2> Now, we're ready to write our cloud-init files. We'll need two of them:</p> meta-data</code> for setting the hostname</li>

Why I Self Host

Self Hosted Postgres in Kubernetes with PITR Recovery

Setting up the DX Commander Expedition Antenna

Decoding Meshtastic Channel Links