Wake Me Up When September Ends

September, 2026

New User

Well, about two years in, and it has finally happened… Dorsal has its first external user!

Yes, he’s a friend of mine, and I’m providing the compute for free, but that’s the nature of a test user. In exchange for his questions, feedback, and patience in testing the system, he gets free access.

Apart from some lingering Pulumi Kubernetes Operator workspace pod bugs that continue to haunt me, his initial application rollout more or less went off without a hitch! I’ll be honest, I was quite shocked. I expected an awkward back-and-forth, where I eventually just have to admit that it isn’t really working yet.

There is still an endless list of things to do, but I’ve been pleasantly surprised with the outcome so far.

Deployment Status Updates

I have a handful of top priority features to tackle, outside of egregious bugs:

August

In the first half of August, I finished knocking-out some of the worst application-layer bugs. There are still a few in my backlog, but they’re less offensive.

I also did that manifest generation consolidation that I mentioned in the previous Dev Log, which has been working splendidly! There is now a relatively simple middleware that regenerates manifests based on the path and HTTP method. It’s working so well, in fact, that I almost forgot about having done it in early August!

Here’s how it works…

For example, if a stack environment ID query parameter was present in a PUT request, the associated stack environment manifest should be regenerated. If there are no differences before and after the regeneration, there is no new git commit, so there is no new Argo CD deploy.

In this way, I can lean slightly on unnecessary manifest regeneration to drastically simplify the code. By always having an idempotent manifest generation process, the only cost is a bit of I/O and CPU when users are actively modifying their stacks. This should, realistically, be an irrelevant increase in compute.

The latter half of August was spent in the UK and France. The eagle-eyed amongst you may have noticed the lack of a Dev Log for last month, and that is why. It didn’t feel necessary to provide an update on a fortnight’s worth of bug bashing.

September

September, on the other hand, has proven to be a surprisingly productive month. Most recently (last night), I rolled out basic functionality for infrastructure deployment status updates:

In Progress

Complete

I have no illusions of being a wizard UI designer, and I’m mostly just thinking out loud these days by throwing things into the Console to see what feels right, but I think this gets the job done!

Architecturally, this is something that I’m pretty happy with.

It’s backed by two simple tables in the database (runs and steps), which are written to by an internal API that the Pulumi Kubernetes Operator workspace pods post back to.

Each run begins by registering with the internal API and getting back a Run ID. The Pulumi program then registers the steps against that run, and simply marks them as started/completed/failed as the program progresses.

If there are no active runs, the latest deployment time is shown instead of the progress bar.

I think I need to add a placeholder, such as “Not Yet Deployed,” for an application that doesn’t have any deployment history. Other than that, this simple implementation should do the job until I decide to attach more metadata to the runs, should that be necessary in the future.

My plan for application deployment status and debugging tools is one that I’m excited for, but I won’t discuss that until it’s actually rolled out. So, stay tuned!

CI Pipelines Go Brr

Automated Testing

Apart from bug bashing and the deployment status feature, I’ve also been doing some meta work on the development workflow.

Because DX can be an endless rabbit hole, I’m trying to take little bites out of it each month or so, rather than sinking large blocks of time into it.

This month, I did two interrelated things:

Because my environment could already be run entirely isolated within a Docker Compose network (with the exception of the login flow via AWS Cognito), moving the tests into CI was relatively simple.

Github Actions ships with docker compose already installed, so all that was necessary was ensuring that the environment variables and secrets were set.

To get things working, I created a new “CI” profile that I could fully reproduce locally. Anything non-sensitive could go into .env.ci files, which were committed to git. Comparable ones already existed for the local development environment anyway.

I then just added the secrets to Github Actions, did a little bit of debugging with the file system access (Docker for Mac vs Github Actions), and we were in business!

So far, my e2e tests have been incredibly reliable. The simple HTML and tooling, I think, really makes a difference. Given that it’s a serverside-rendered application, with no virtual DOM, and no “magic” component libraries, the Playwright runner continues to be very reliable across Chrome, Firefox, and Webkit.

There are also the unit tests, which I currently only have covering the internal API. The reason is that this is actually the easiest way to develop those endpoints. Because they are for inter-service communication, it feels very natural to define the API contract with tests, make them pass, and then consume the endpoints in the other service.

The aforementioned deployment status updates were built in this exact manner.

In the last few days, I’ve also added the unit tests to CI.

Dependabot

Dependabot PRs were the main reason for adding the tests to Github Actions.

Every time a Dependabot PR appeared, I’d either just let Jesus take the wheel and merge it, or I’d clone, run the tests, and merge if everything passed.

It quickly became obvious that I could skip those steps where I’m the middleman, and just have Dependabot auto-merge PRs when they passed CI.

Unfortunately, Dependabot doesn’t have an auto-merge feature built-in. So, I’ll briefly explain how I got this working.

First, I created a Fine-Grained Access token with content: write, pull requests: write, and commit status: read. I added this token to the Dependabot Secrets in Github.

Then, I added a step at the end of the Tests workflow that looks like this:

- name: "Dependabot: Approve and Enable Auto-Merge"
  if: github.actor == 'dependabot[bot]'
  env:
	  GH_TOKEN: $
	  PR_URL: $
  run: |
	  gh pr review "$PR_URL" --approve -b "E2E tests passed successfully."
	  gh pr merge "$PR_URL" --auto --squash

That’s it!

Now, when Dependabot opens a PR, CI runs the tests. If they pass, the PAT (acting on my behalf) approves the PR and enables auto-merge.

For extra fun, I added branch protection rules that require the tests to pass in CI, and I require all branches to be up-to-date before merging. Because Dependabot automatically rebases to resolve conflicts, this ensures that CI is always run with the latest version of all packages before a PR is merged.

If I eventually make a business out of Dorsal, with real customers, I’ll have a proper Development / Staging environment with continuous delivery. The Production environment will be stamped with a release version, which I’ll cut whenever I’m ready to promote the current master branch.

For now, we just let the automation do its thing…

NOTE: I have a 7-day delay for Dependabot updates. This is typically a good idea for helping to avoid supply chain attacks.

Snap Back to Reality

I haven’t discussed real life much in today’s update, but holy smoking biscuits… it’s been more than just a couple weeks of international travel, let me tell you.

Real life comes first, and I’m so grateful to have many big, exciting things happening in it, but I’m still happy to have had such a productive month with Dorsal.

As far as real life goes, though, I’ll say this… Regardless of what happens with it, even if nobody ever pays me to use Dorsal, having this project on the go is keeping me sane.

If you’re also burnt-out at work, tired of wading through AI slop, losing the “spark” from programming… whatever it is that’s bumming you out about tech, I highly recommend a side project.

Sometimes, real life means just having a job. It pays the bills. Right now, I need those bills to be paid, but I also need to keep my sanity.

I consider Dorsal, the time and the money spent on it, to be an investment in not quitting my day job.

There’s no guarantee that it’ll work for you, but it has certainly worked for me.

Until next month,

Nathan