Using systemd as user-level inetd

Sun 03 December 2023 by pj Tagged as linux systemd

So, I wanted to jump on the AI-dev-assistant bandwagon. Preferably with an open-source self-hosted system so I don't have to worry about usage limits or the system going away, etc.

A bit of research later, and my dev-assistant of choice is currently Tabby, which is happy to run under docker and then talk to neovim via a socket. So now I just have to set it up to auto-start somehow.

Systemd has socket activation, which is almost the same as inetd, but docker isn't nice enough to accept filehandles via a socket or pipe (which is what inetd - and thus systemd - wants)... but the systemd guys realized that lots of things wouldn't do so and thus wrote systemd-socket-proxyd to bridge the gap.

I mostly followed the method and details as (well) written up in a blog post by an atlassian dev, resulting in:

tabby-docker.sh

#!/bin/bash
exec docker run -t --rm \
  --name tabby \
  --pull always \
  --gpus all \
  -p 8151:8080 \
  -v $HOME/.tabby:/data \
  tabbyml/tabby \
  serve --model TabbyML/StarCoder-1B --device cuda

~/.config/systemd/user/tabby-docker.service

[Unit]
Description=TabbyML/tabby container

[Service]
ExecStart=/home/pj/bin/tabby-docker.sh
ExecStartPost=/home/pj/bin/waitforport localhost 8151
ExecStop=/usr/bin/docker stop tabby

~/.config/systemd/user/tabby-docker-proxy.service

[Unit]
Description=unix socket proxy to tabby-docker service

BindsTo=tabby-docker.service
After=tabby-docker.service

[Service]
ExecStart=/lib/systemd/systemd-socket-proxyd 127.0.0.1:8151

~/.config/systemd/user/tabby-docker-proxy.socket

[Socket]
ListenStream=8150

[Install]
WantedBy=sockets.target

The only tweak I found necessary was to change the Requires= in the myservice-proxy.service file to be BindsTo= so that if I manually stop tabby (via docker stop or systemctl --user stop tabb-docker), it will be restarted when a connection is next made.

The only annoyance factor is the use of two sockets: one for systemd to listen on and one for the proxy to talk to. This has two potential fixes: 1. Docker should be able to accept a connection from systemd somehow: a unix socket, a file descriptor... something. 2. Systemd should have a shortcut syntax to do all of the above. inetd and xinetd had redirect; something similar in the .socket config could obviate the need for the proxy service.

Hmm. Now that I think about it, I wonder if there's a way to wrap the proxy inside the docker container so that the unix socket could be passed via a filesystem mapping. Maybe I'll experiment another time.


George Floyd and the failure of the police

Mon 08 June 2020 by pj Tagged as blm commentary
I just read Neil Degrasse Tyson's _Reflections on the Color of My Skin_ and was struck by one of the last suggestions he made: that George Floyd should get a full dress funeral, like any police officer shot in the line of duty. And I heartily agree - civilians wrongly killed … read more

Scalable Rice Pudding

Sun 10 May 2020 by pj Tagged as breakfast dessert

...if the amounts seem strange it's to make it easy to scale up. Got 2 cups of leftover rice from chinese food delivery? multiply by two. Making rice for dinner? Make an extra 4 or 6 (or 15 if you're my family) cups of rice and use the leftovers.

The …

read more

Putting My Home Into Git

Tue 28 August 2018 by pj Tagged as linux

So I currently have two laptops: one for work and one for personal use. There is, however, a fair bit of overlap; I'd like to have a bunch of config files (.bashrc, .vimrc, ~/.config/i3/config, etc) and a bunch of useful scripts (in ~/bin) available in both places. I …

read more

Auto Wrench Belt Replacement?

Wed 13 September 2017 by pj

Some years ago I received a gift of a Black and Decker AAW100 Type 1 'Auto Wrench'. It's nice for the lazy man (which is definitely me).

I recently pulled it out of the toolbox, only to find that it didn't work. A screwdriver and 5 minutes later and it …

read more

Ubiquiti Controller Password reset

Fri 28 July 2017 by pj Tagged as software wifi

I really like my Ubiquiti AP; it's very set-and-forget.

But apparently I take that too literally, because I've forgotten the password at least twice in the 3 months I've owned it.

So here's a note (somewhat to myself) of the one-liner to reset the password to password:

(echo 'use ace …
read more

Putting Puppy Linux on a Raspberry Pi

Fri 21 July 2017 by pj Tagged as linux raspi

I like the Raspberry Pi machines.... mostly. It's reasonably cheap, reasonably reliable, very widespread hardware. I just can't get comfortable running from an SD card - I've dealt with corrupted OSs on them waaaay too often in the past. So I went looking for a linux distro that would load from …

read more

dCore notes v0.1

Mon 17 August 2015 by pj Tagged as dcore linux raspi

What's dCore?

dCore is a TinyCoreLinux derivative that's based on Debian. TinyCoreLinux has the nice property of, once booted, running from RAM instead of scribbling on your USB stick for ephemeral things like log writes and pid files and etc.

Installing

I was lazy so I took the TinyCoreLinux installer …

read more

OSMC and hostapd

Tue 19 May 2015 by pj Tagged as kodi linux raspi wifi

I recently hooked up a Raspberry Pi running Kodi to my Living Room television, and it was a hit with the kids (their favorite shows and moving more easily accessible than by swapping DVDs) and the wife (easy to run via a remote app on her phone).

Separately, trips with …

read more

Packaging sites with Aspen

Tue 05 May 2015 by pj Tagged as aspen software www

As the co-maintainer of Aspen, I've focused mainly on making it easier to use, but have, myself, had few opportunities to build sites using it. One such was granted to me the other day, and I took the time to figure out how to build a python package that used …

read more