PJ's Ponderingshttps://place.org/~pj/blog/2023-12-03T00:00:00-05:00Using systemd as user-level inetd2023-12-03T00:00:00-05:002023-12-03T00:00:00-05:00pjtag:place.org,2023-12-03:/~pj/blog/using-systemd-as-user-level-inetd.html<p>So, I wanted to jump on the AI-dev-assistant bandwagon. Preferably with
an open-source self-hosted system so I don't have to worry about usage limits
or the system going away, etc. </p>
<p>A bit of research later, and my dev-assistant of choice is currently
<a href="https://github.com/TabbyML/Tabby">Tabby</a>, which is happy to run under docker …</p><p>So, I wanted to jump on the AI-dev-assistant bandwagon. Preferably with
an open-source self-hosted system so I don't have to worry about usage limits
or the system going away, etc. </p>
<p>A bit of research later, and my dev-assistant of choice is currently
<a href="https://github.com/TabbyML/Tabby">Tabby</a>, which is happy to run under docker and
then talk to neovim via a socket. So now I just have to set it up to auto-start somehow.</p>
<p>Systemd has socket activation, which is <em>almost</em> the same as inetd, but docker isn't
nice enough to accept filehandles via a socket or pipe (which is what inetd - and
thus systemd - wants)... but the systemd guys realized that lots of things
wouldn't do so and thus wrote
<a href="https://www.freedesktop.org/software/systemd/man/latest/systemd-socket-proxyd.html">systemd-socket-proxyd</a> to bridge the gap.</p>
<p>I mostly followed the method and details as (well) written up in
<a href="https://blog.developer.atlassian.com/docker-systemd-socket-activation/">a blog post by an atlassian dev</a>,
resulting in:</p>
<h2>tabby-docker.sh</h2>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="nb">exec</span><span class="w"> </span>docker<span class="w"> </span>run<span class="w"> </span>-t<span class="w"> </span>--rm<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--name<span class="w"> </span>tabby<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--pull<span class="w"> </span>always<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--gpus<span class="w"> </span>all<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>-p<span class="w"> </span><span class="m">8151</span>:8080<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>-v<span class="w"> </span><span class="nv">$HOME</span>/.tabby:/data<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>tabbyml/tabby<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>serve<span class="w"> </span>--model<span class="w"> </span>TabbyML/StarCoder-1B<span class="w"> </span>--device<span class="w"> </span>cuda
</code></pre></div>
<h2>~/.config/systemd/user/tabby-docker.service</h2>
<div class="highlight"><pre><span></span><code><span class="k">[Unit]</span>
<span class="na">Description</span><span class="o">=</span><span class="s">TabbyML/tabby container</span>
<span class="k">[Service]</span>
<span class="na">ExecStart</span><span class="o">=</span><span class="s">/home/pj/bin/tabby-docker.sh</span>
<span class="na">ExecStartPost</span><span class="o">=</span><span class="s">/home/pj/bin/waitforport localhost 8151</span>
<span class="na">ExecStop</span><span class="o">=</span><span class="s">/usr/bin/docker stop tabby</span>
</code></pre></div>
<h2>~/.config/systemd/user/tabby-docker-proxy.service</h2>
<div class="highlight"><pre><span></span><code><span class="k">[Unit]</span>
<span class="na">Description</span><span class="o">=</span><span class="s">unix socket proxy to tabby-docker service</span>
<span class="na">BindsTo</span><span class="o">=</span><span class="s">tabby-docker.service</span>
<span class="na">After</span><span class="o">=</span><span class="s">tabby-docker.service</span>
<span class="k">[Service]</span>
<span class="na">ExecStart</span><span class="o">=</span><span class="s">/lib/systemd/systemd-socket-proxyd 127.0.0.1:8151</span>
</code></pre></div>
<h2>~/.config/systemd/user/tabby-docker-proxy.socket</h2>
<div class="highlight"><pre><span></span><code><span class="k">[Socket]</span>
<span class="na">ListenStream</span><span class="o">=</span><span class="s">8150</span>
<span class="k">[Install]</span>
<span class="na">WantedBy</span><span class="o">=</span><span class="s">sockets.target</span>
</code></pre></div>
<p>The only tweak I found necessary was to change the <code>Requires=</code> in the
<code>myservice-proxy.service</code> file to be <code>BindsTo=</code> so that if I manually stop tabby
(via <code>docker stop</code> or <code>systemctl --user stop tabb-docker</code>), it will be restarted
when a connection is next made.</p>
<p>The only annoyance factor is the use of two sockets: one for systemd to listen on
and one for the proxy to talk to. This has two potential fixes:
1. Docker should be able to accept a connection from systemd somehow: a unix socket,
a file descriptor... something.
2. Systemd should have a shortcut syntax to do all of the above. inetd and xinetd
had <code>redirect</code>; something similar in the <code>.socket</code> config could obviate the need for
the proxy service.</p>
<p>Hmm. Now that I think about it, I wonder if there's a way to wrap the proxy <em>inside</em> the docker container so that the unix socket could be passed via a filesystem mapping. Maybe I'll experiment another time.</p>George Floyd and the failure of the police2020-06-08T12:37:20-04:002020-06-08T12:37:20-04:00pjtag:place.org,2020-06-08:/~pj/blog/george-floyd-and-the-failure-of-the-police.html
I just read <a href="https://www.haydenplanetarium.org/tyson/commentary/2020-06-03-reflections-on-color-of-my-skin.php">Neil Degrasse Tyson's _Reflections on the Color of My Skin_ </a>
and was struck by one of the last suggestions he made: that George Floyd should
get a full dress funeral, like any police officer shot in the line of duty. And
I heartily agree - civilians wrongly killed …
I just read <a href="https://www.haydenplanetarium.org/tyson/commentary/2020-06-03-reflections-on-color-of-my-skin.php">Neil Degrasse Tyson's _Reflections on the Color of My Skin_ </a>
and was struck by one of the last suggestions he made: that George Floyd should
get a full dress funeral, like any police officer shot in the line of duty. And
I heartily agree - civilians wrongly killed by police should be given this
recognition, to point out that the police, supposed guardians of civility and
civilization, failed in their duty. As with the death of any officer in the
line of duty, they should, for some period of time, wear black armbands,
symbols of their shame and dishonor - they're supposed to protect civilians,
not harm them.
Scalable Rice Pudding2020-05-10T00:00:00-04:002020-05-10T00:00:00-04:00pjtag:place.org,2020-05-10:/~pj/blog/scalable-rice-pudding.html<p>...if the amounts seem strange it's to make it easy to scale up. Got 2 cups
of leftover rice from chinese food delivery? multiply by two. Making rice
for dinner? Make an extra 4 or 6 (or 15 if you're my family) cups of rice
and use the leftovers.</p>
<h2>The …</h2><p>...if the amounts seem strange it's to make it easy to scale up. Got 2 cups
of leftover rice from chinese food delivery? multiply by two. Making rice
for dinner? Make an extra 4 or 6 (or 15 if you're my family) cups of rice
and use the leftovers.</p>
<h2>The Recipe</h2>
<p>Per 1-cup of cooked white rice (leftover, whatever)
add:</p>
<ul>
<li>1 cup milk</li>
<li>2/3 Tablespoon butter</li>
<li>1/3 teaspoon vanilla</li>
<li>1/3 teaspoon cinnamon</li>
</ul>
<p>Bring all of the above to a boil. Then add:</p>
<ul>
<li>2/9 cups sugar</li>
<li>dried fruit to taste (raisins, craisins, whatever sounds good)</li>
</ul>
<p>Let simmer for 30 minutes.</p>
<p>Serve warm or cold. (I like it warm, with a splash of cold milk)
Store in a closed container in the refrigerator.</p>Putting My Home Into Git2018-08-28T00:00:00-04:002018-08-28T00:00:00-04:00pjtag:place.org,2018-08-28:/~pj/blog/putting-my-home-into-git.html<p>So I currently have two laptops: one for work and one for personal use. There
is, however, a fair bit of overlap; I'd like to have a bunch of config files
(<code>.bashrc</code>, <code>.vimrc</code>, <code>~/.config/i3/config</code>, etc) and a bunch of useful
scripts (in <code>~/bin</code>) available in both places. I …</p><p>So I currently have two laptops: one for work and one for personal use. There
is, however, a fair bit of overlap; I'd like to have a bunch of config files
(<code>.bashrc</code>, <code>.vimrc</code>, <code>~/.config/i3/config</code>, etc) and a bunch of useful
scripts (in <code>~/bin</code>) available in both places. I looked at a couple solutions,
including <a href="http://dotfiles.github.io/">dotfiles</a> and such, but a post on
StackOverflow had the <a href="https://stackoverflow.com/a/27336975/8002">outline of a
solution</a> that I'm fairly happy
with.</p>
<p>I put an alias into my <code>.bashrc</code>:</p>
<ul>
<li><code>alias homegit='git --work-tree=$HOME --git-dir=$HOME/.home.git'</code></li>
</ul>
<p>And wrote a little script to automate setup on new machines:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span>-d<span class="w"> </span><span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/.home.git"</span><span class="w"> </span><span class="o">]</span><span class="p">;</span><span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">"homegit already set up."</span>
<span class="w"> </span><span class="nb">exit</span><span class="w"> </span><span class="m">1</span>
<span class="k">fi</span>
<span class="nv">homegit</span><span class="o">=</span><span class="s2">"git --work-tree=</span><span class="nv">$HOME</span><span class="s2"> --git-dir=</span><span class="nv">$HOME</span><span class="s2">/.home.git"</span>
<span class="nv">$homegit</span><span class="w"> </span>init
<span class="nv">$homegit</span><span class="w"> </span>remote<span class="w"> </span>add<span class="w"> </span>origin<span class="w"> </span><span class="nv">$MY_HOMEDIR_REPO</span>
<span class="nv">$homegit</span><span class="w"> </span>fetch<span class="w"> </span>origin
<span class="nv">$homegit</span><span class="w"> </span>reset
<span class="nv">$homegit</span><span class="w"> </span>checkout<span class="w"> </span>-t<span class="w"> </span>origin/master
</code></pre></div>
<p>and then ran it. Then I wrote an <code>info/exclude</code> file: </p>
<div class="highlight"><pre><span></span><code><span class="gh">#</span> exclude everything
/*
<span class="gh">#</span> ...except the following
!/bin
!/.home.git/info
!/.xscreensaver
!/.git_template
!/.gitconfig
!/.screenrc
!/.bashrc
!/.vimrc
!/.xinitrc
!/.config
/.config/*
!/.config/i3/
</code></pre></div>
<p>So only the files I wanted to save were listed in a 'homegit status'. Then I
could just <code>homegit add</code> them and could go to town!</p>
<p>I imagine maintaining the <code>info/exclude</code> file is going to be the main issue,
but it's probably good for me to keep up with that minor skill anyway.</p>Auto Wrench Belt Replacement?2017-09-13T19:41:00-04:002017-09-13T19:41:00-04:00pjtag:place.org,2017-09-13:/~pj/blog/auto-wrench-belt-replacement.html<p>Some years ago I received a gift of a Black and Decker AAW100 Type 1 'Auto Wrench'.
It's nice for the lazy man (which is definitely me). </p>
<p>I recently pulled it out of the toolbox, only to find that it didn't work.
A screwdriver and 5 minutes later and it …</p><p>Some years ago I received a gift of a Black and Decker AAW100 Type 1 'Auto Wrench'.
It's nice for the lazy man (which is definitely me). </p>
<p>I recently pulled it out of the toolbox, only to find that it didn't work.
A screwdriver and 5 minutes later and it seems that the issue is that the belt is broken.
Searching on the internet, it's a common complaint, with no real good solution.</p>
<p>First job is to figure ou what needs replacement:
Some specs:</p>
<ul>
<li>Tooth width: .05" = 20</li>
<li>Belt width: .1160"</li>
<li>Belt length : 1.1165 + 1.2440 = 2.3605"</li>
<li>grooves = teeth = 19
*</li>
</ul>Ubiquiti Controller Password reset2017-07-28T16:05:00-04:002017-07-28T16:05:00-04:00pjtag:place.org,2017-07-28:/~pj/blog/ubiquiti-controller-password-reset.html<p>I really like my Ubiquiti AP; it's very set-and-forget.</p>
<p>But apparently I take that too literally, because I've forgotten the password
at least twice in the 3 months I've owned it.</p>
<p>So here's a note (somewhat to myself) of the one-liner to reset the password to
<code>password</code>:</p>
<div class="highlight"><pre><span></span><code>(echo 'use ace …</code></pre></div><p>I really like my Ubiquiti AP; it's very set-and-forget.</p>
<p>But apparently I take that too literally, because I've forgotten the password
at least twice in the 3 months I've owned it.</p>
<p>So here's a note (somewhat to myself) of the one-liner to reset the password to
<code>password</code>:</p>
<div class="highlight"><pre><span></span><code>(echo 'use ace' ; echo 'db.admin.update({}, { $set: {"x_shadow" : "$6$9Ter1EZ9$lSt6/tkoPguHqsDK0mXmUsZ1WE2qCM4m9AQ.x9/eVNJxws.hAxt2Pe8oA9TFB7LPBgzaHBcAfKFoLpRQlpBiX1" }})') | mongo --port 27117
</code></pre></div>Putting Puppy Linux on a Raspberry Pi2017-07-21T00:00:00-04:002017-07-21T00:00:00-04:00pjtag:place.org,2017-07-21:/~pj/blog/putting-puppy-linux-on-a-raspberry-pi.html<p>I like the Raspberry Pi machines.... mostly. It's reasonably cheap,
reasonably reliable, very widespread hardware. I just can't get comfortable
running from an SD card - I've dealt with corrupted OSs on them waaaay too
often in the past. So I went looking for a linux distro that would load from …</p><p>I like the Raspberry Pi machines.... mostly. It's reasonably cheap,
reasonably reliable, very widespread hardware. I just can't get comfortable
running from an SD card - I've dealt with corrupted OSs on them waaaay too
often in the past. So I went looking for a linux distro that would load from
flash but run from RAM. Enter Puppy Linux.</p>
<p>Puppy has been around awhile, and it shows in the hodgepodge of information
available online. After much reading of wikis, opening of issues on github,
and asking on IRC, here's how to build a recent version of Puppy-raspbian:</p>
<p>If you're not running Puppy already, you'll want to clone <code>run_woof</code>, which
sets up a Puppy-chroot from which you can run <code>woof-CE</code> which will build your
image. So:</p>
<div class="highlight"><pre><span></span><code>git clone https://github.com/puppylinux-woof-CE/run_woof.git
git clone https://github.com/puppylinux-woof-CE/woof-CE.git
</code></pre></div>
<p>Now you need a recent .iso to turn into a chroot. I used:</p>
<div class="highlight"><pre><span></span><code>wget http://distro.ibiblio.org/puppylinux/puppy-tahr/iso/tahrpup%20-6.0-CE/devx_tahr_6.0.6.sfs
wget http://distro.ibiblio.org/puppylinux/puppy-tahr/iso/tahrpup%20-6.0-CE/tahr64-6.0.6-uefi.iso
</code></pre></div>
<p>Then run run-woof:</p>
<div class="highlight"><pre><span></span><code>cd run_woof
./run_woof_helper
</code></pre></div>
<p>Note that I got a complaint about a lack of a value for SUDO_ASKPASS, which
I remedied by doing:</p>
<div class="highlight"><pre><span></span><code><span class="n">sudo</span><span class="w"> </span><span class="n">apt</span><span class="w"> </span><span class="n">install</span><span class="w"> </span><span class="n">ssh</span><span class="o">-</span><span class="n">askpass</span><span class="o">-</span><span class="n">gnome</span><span class="w"> </span>
<span class="k">export</span><span class="w"> </span><span class="n">SUDO_ASKPASS</span><span class="o">=/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">ssh</span><span class="o">-</span><span class="n">askpass</span>
</code></pre></div>
<p>after which <code>run-woof_helper</code> ran fine. I chose the above tahr64 iso from the
list, and after a few minutes was dumped into a root prompt in the chroot.</p>dCore notes v0.12015-08-17T00:00:00-04:002015-08-17T00:00:00-04:00pjtag:place.org,2015-08-17:/~pj/blog/dcore-notes-v01.html<h1>What's dCore?</h1>
<p><a href="http://wiki.tinycorelinux.net/dcore:welcome">dCore</a> is a TinyCoreLinux derivative that's based on Debian. TinyCoreLinux has the nice property of, once booted, running from RAM instead of scribbling on your USB stick for ephemeral things like log writes and pid files and etc.</p>
<h1>Installing</h1>
<p>I was lazy so I took the TinyCoreLinux installer …</p><h1>What's dCore?</h1>
<p><a href="http://wiki.tinycorelinux.net/dcore:welcome">dCore</a> is a TinyCoreLinux derivative that's based on Debian. TinyCoreLinux has the nice property of, once booted, running from RAM instead of scribbling on your USB stick for ephemeral things like log writes and pid files and etc.</p>
<h1>Installing</h1>
<p>I was lazy so I took the TinyCoreLinux installer CD and used it to make a bootable USB stick,
and then copied over the <code>dCore-jessie.gz</code> and <code>vmlinuz-jessie</code> from the
<a href="http://tinycorelinux.net/dCore/x86/release/dCore-jessie/">download site</a> files into
<code>/boot</code>, and then modified <code>/boot/extlinux/extlinux.conf</code>.</p>
<p>It was easier to do than to describe :)</p>
<h1>Configuring</h1>
<ul>
<li>Setting the tc user password: <code>passwd</code> didn't work (for unknown reasons it just returned immediately without prompting for a new password), so (as root) I worked around it by doing <code>echo 'tc:mynewpassword' | chpasswd</code>. And then of course put <code>/etc/shadow</code> into my <code>/opt/.filetool.lst</code>.</li>
</ul>OSMC and hostapd2015-05-19T00:00:00-04:002015-05-19T00:00:00-04:00pjtag:place.org,2015-05-19:/~pj/blog/osmc-and-hostapd.html<p>I recently hooked up a Raspberry Pi running Kodi to my Living Room television,
and it was a hit with the kids (their favorite shows and moving more easily accessible than
by swapping DVDs) and the wife (easy to run via a remote app on her phone). </p>
<p>Separately, trips with …</p><p>I recently hooked up a Raspberry Pi running Kodi to my Living Room television,
and it was a hit with the kids (their favorite shows and moving more easily accessible than
by swapping DVDs) and the wife (easy to run via a remote app on her phone). </p>
<p>Separately, trips with the kids in the car where they'd like to watch a video are currently
a bit annoying because it means keeping a DVD library in the car. </p>
<p>Clearly, Something Must Be Done.</p>
<p>My solution: Install OSMC on a Raspberry Pi and then hack it to 1) be a wireless AP and 2) use a GPIO
to signal an <a href="https://learn.adafruit.com/adafruit-powerboost-500-plus-charger">Adafruit PowerBoost 500C</a> that
it's time to shut down in an orderly fashion so as not to trash the filesystem.</p>
<h1>On OSMC vs OpenELEC</h1>
<p>OpenELEC is my usually-preferred distro for a dedicated raspi box, but... it's not very easily hackable. OSMC used to be raspbmc,
which was based on raspbian, so it's a nice familiar debian base with scads of software in the standard repos. OSMC it is.</p>
<h1>To get hostapd to work:</h1>
<ul>
<li><code>systemctl disable connman.service</code> - disable connman, which OSMC uses for some reason</li>
<li><code>apt-get install ifupdown hostapd dnsmasq wireless-tools dhcpcd5 ...</code> - I lost track of all the network tools I had to install,
but that's a good start.</li>
<li>configure /etc/hostapd/hostapd.conf</li>
<li>configure /etc/network/interfaces</li>
<li>configure /etc/dnsmasq.conf to serve dhcp and some minimal amount of DNS</li>
<li>For some reason I had to blacklist the bluetooth module for my zd1211 driver to come up without a soft rfkill flag set</li>
</ul>
<p>...with the result being a box that boots up to be both a wireless AP <em>and</em> a Kodi machine.</p>Packaging sites with Aspen2015-05-05T16:05:00-04:002015-05-05T16:05:00-04:00pjtag:place.org,2015-05-05:/~pj/blog/packaging-sites-with-aspen.html<p>As the co-maintainer of <a href="http://aspen.io">Aspen</a>, I've focused mainly on making it easier to use, but have, myself,
had few opportunities to build sites using it. One such was granted to me the other day, and I took the time to
figure out how to build a python package that used …</p><p>As the co-maintainer of <a href="http://aspen.io">Aspen</a>, I've focused mainly on making it easier to use, but have, myself,
had few opportunities to build sites using it. One such was granted to me the other day, and I took the time to
figure out how to build a python package that used aspen and exported a website created in the package.</p>
<p>So with a file tree like:</p>
<div class="highlight"><pre><span></span><code>README.md
setup.py
myapp
myapp/__main__.py
myapp/__init__.pyc
myapp/wsgi.pyc
myapp/__init__.py
myapp/www
myapp/www/index.spt
myapp/wsgi.py
</code></pre></div>
<p>$ more myapp/wsgi.py </p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">os.path</span>
<span class="kn">from</span> <span class="nn">aspen.website</span> <span class="kn">import</span> <span class="n">Website</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
<span class="n">website</span> <span class="o">=</span> <span class="n">Website</span><span class="p">()</span>
<span class="n">website</span><span class="o">.</span><span class="n">www_root</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">dirname</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">abspath</span><span class="p">(</span><span class="vm">__file__</span><span class="p">)),</span> <span class="s1">'www'</span><span class="p">)</span>
<span class="k">return</span> <span class="n">website</span>
</code></pre></div>
<p>$ more myapp/<strong>main</strong>.py </p>
<div class="highlight"><pre><span></span><code><span class="c1"># Called via python -m myapp</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="kn">from</span> <span class="nn">wsgiref.simple_server</span> <span class="kn">import</span> <span class="n">make_server</span>
<span class="kn">from</span> <span class="nn">myapp.wsgi</span> <span class="kn">import</span> <span class="n">main</span>
<span class="n">website</span> <span class="o">=</span> <span class="n">main</span><span class="p">()</span>
<span class="n">port</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'PORT'</span><span class="p">,</span> <span class="s1">'8080'</span><span class="p">))</span>
<span class="n">server</span> <span class="o">=</span> <span class="n">make_server</span><span class="p">(</span><span class="s1">'0.0.0.0'</span><span class="p">,</span> <span class="n">port</span><span class="p">,</span> <span class="n">website</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Greetings, program! Welcome to port </span><span class="si">{0}</span><span class="s2">."</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">port</span><span class="p">))</span>
<span class="n">server</span><span class="o">.</span><span class="n">serve_forever</span><span class="p">()</span>
</code></pre></div>
<p>$ more setup.py</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">os</span>
<span class="kn">from</span> <span class="nn">setuptools</span> <span class="kn">import</span> <span class="n">setup</span><span class="p">,</span> <span class="n">find_packages</span>
<span class="k">def</span> <span class="nf">spt_files</span><span class="p">(</span><span class="n">topdir</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""find .spt files under 'topdir'"""</span>
<span class="n">found</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">root</span><span class="p">,</span> <span class="n">dirs</span><span class="p">,</span> <span class="n">files</span> <span class="ow">in</span> <span class="n">os</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">topdir</span><span class="p">):</span>
<span class="n">found</span> <span class="o">+=</span> <span class="p">[</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">f</span><span class="p">)</span> <span class="k">for</span> <span class="n">f</span> <span class="ow">in</span> <span class="n">files</span> <span class="k">if</span> <span class="n">f</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s1">'.spt'</span><span class="p">)</span> <span class="p">]</span>
<span class="n">found</span> <span class="o">=</span> <span class="p">[</span> <span class="n">f</span><span class="p">[</span><span class="nb">len</span><span class="p">(</span><span class="n">topdir</span><span class="p">):]</span> <span class="k">if</span> <span class="n">f</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">topdir</span><span class="p">)</span> <span class="k">else</span> <span class="n">f</span> <span class="k">for</span> <span class="n">f</span> <span class="ow">in</span> <span class="n">found</span> <span class="p">]</span>
<span class="k">return</span> <span class="n">found</span>
<span class="n">setup</span><span class="p">(</span>
<span class="n">name</span><span class="o">=</span><span class="s1">'myapp'</span><span class="p">,</span>
<span class="n">version</span><span class="o">=</span><span class="s1">'0.1'</span><span class="p">,</span>
<span class="n">description</span><span class="o">=</span><span class="s1">'My app description'</span><span class="p">,</span>
<span class="n">url</span><span class="o">=</span><span class="s1">'https://github.com/pypa/sampleproject'</span><span class="p">,</span>
<span class="c1"># Author details</span>
<span class="n">author</span><span class="o">=</span><span class="s1">'The Python Packaging Authority'</span><span class="p">,</span>
<span class="n">author_email</span><span class="o">=</span><span class="s1">'pypa-dev@googlegroups.com'</span><span class="p">,</span>
<span class="c1"># Choose your license</span>
<span class="n">license</span><span class="o">=</span><span class="s1">'GPLv3'</span><span class="p">,</span>
<span class="c1"># See https://pypi.python.org/pypi?%3Aaction=list_classifiers</span>
<span class="n">classifiers</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'Development Status :: 3 - Alpha'</span><span class="p">,</span>
<span class="s1">'Intended Audience :: Developers'</span><span class="p">,</span>
<span class="s1">'Programming Language :: Python :: 2.7'</span><span class="p">,</span>
<span class="p">],</span>
<span class="n">keywords</span> <span class="o">=</span> <span class="s1">'myapp'</span><span class="p">,</span>
<span class="n">packages</span> <span class="o">=</span> <span class="n">find_packages</span><span class="p">(),</span>
<span class="n">package_data</span> <span class="o">=</span> <span class="p">{</span> <span class="s1">''</span><span class="p">:</span> <span class="n">spt_files</span><span class="p">(</span><span class="s1">'myapp'</span><span class="p">)</span> <span class="p">},</span>
<span class="n">install_requires</span> <span class="o">=</span> <span class="p">[</span> <span class="s1">'aspen'</span> <span class="p">]</span>
<span class="p">)</span>
</code></pre></div>
<p>The key here is the package_data parameter, which pulls in .spt files under the specified directory, making them part of the package.</p>Vundle Rocks!2014-08-02T01:01:00-04:002014-08-02T01:01:00-04:00pjtag:place.org,2014-08-02:/~pj/blog/vundle-rocks.html<p>So I just replaced a very complicated git repo with numerous submodules with a much simpler one.</p>
<p>This gift was granted to me by the creator of <a href="https://github.com/gmarik/Vundle.vim">Vundle</a>, which is a vim package manager
that lets you specify desired vim packages in your .vimrc and then handles the download and …</p><p>So I just replaced a very complicated git repo with numerous submodules with a much simpler one.</p>
<p>This gift was granted to me by the creator of <a href="https://github.com/gmarik/Vundle.vim">Vundle</a>, which is a vim package manager
that lets you specify desired vim packages in your .vimrc and then handles the download and installation of same, automatically.</p>
<p>Since I'm likely to copy my .vimrc around (another reason it was in a git repo was to keep it sync'd across multiple machines),
I added an autoinstall block to the top, so that it will autoinstall vundle if it needs to. Yay for automation!</p>Laptop Overheating2014-05-12T00:00:00-04:002014-05-12T00:00:00-04:00pjtag:place.org,2014-05-12:/~pj/blog/laptop-overheating.html<p>This morning, my laptop was, as usual, showing an average
idle running temp of around 76C, which is fairly hot, but didn't
seem too bad... until I put it under load, at which point it shut
down because of the temp limit of 97C. Much searching of the
Internet commenced …</p><p>This morning, my laptop was, as usual, showing an average
idle running temp of around 76C, which is fairly hot, but didn't
seem too bad... until I put it under load, at which point it shut
down because of the temp limit of 97C. Much searching of the
Internet commenced, trying to find out what might make a W510
overheat and shut down. Perhaps the heatsink on the CPU wasn't
seated properly - but you'd think that would have evinced itself
sometime in the previous two years of ownership!</p>
<p>Over lunch, acting on a hunch, I removed the keyboard so as to
get at the internals, and applied massive amounts of compressed air,
sending much dust flying. </p>
<p>New idle temp at bootup: 46C. </p>
<p>This reminds me of a friend's '$20 home repair of your desktop or your money
back' idea: such a service would do nothing more than show up, take the machine outside,
crack the case and apply an air compressor. Either you make $20 for <5min of work,
or you give them their money back - no matter what, it didn't take much time, so on
to the next one!</p>Bundle of Holding 3+ Reviews2013-07-08T10:20:00-04:002013-07-08T10:20:00-04:00pjtag:place.org,2013-07-08:/~pj/blog/bundle-of-holding-3-reviews.html<p>My friend <a href="http://allenvarney.com">Allen Varney</a> just spearheaded
<a href="http://www.bundleofholding.com/index/ended">Bundle of Holding +3</a>, and
I just finished reading them all, so here are some brief reviews:</p>
<ul>
<li>
<p><em>Oathbreaker</em> (Books 1 and 2) by Colin McComb - nice high fantasy, with lots of backstabbing and intrigue, though a bit heavy on backstory. And while the backstory …</p></li></ul><p>My friend <a href="http://allenvarney.com">Allen Varney</a> just spearheaded
<a href="http://www.bundleofholding.com/index/ended">Bundle of Holding +3</a>, and
I just finished reading them all, so here are some brief reviews:</p>
<ul>
<li>
<p><em>Oathbreaker</em> (Books 1 and 2) by Colin McComb - nice high fantasy, with lots of backstabbing and intrigue, though a bit heavy on backstory. And while the backstory was certainly interesting, I enjoyed the latest bits of the timeline the most. (Note the careful phrasing to avoid spoilers)</p>
</li>
<li>
<p><em>Tales of the Far West</em> by Gareth-Michael Skarka - a great genre mashup that I'd love to read more of! That said, some of the short stories were <em>too</em> short, more like vignettes than actual stories.</p>
</li>
<li>
<p><em>Switchflipped</em> by Greg Stolze - Unlike <em>Oathbreaker</em>, there's not <em>enough</em> backstory here - I was left wanting more information on the world in general, the Bells and the Fords and other highly idiomatic clans, and the end seemed more like it should be the midpoint of the actual story than the end. Is this part of a series and I missed that note?</p>
</li>
<li>
<p><em>Delta Green: Strange Authorities</em> by John Scott Tynes - Reminds me a bit of Larry Correria or P.N.Elrod, with an extra dose of espionage. Also left lots of world-details out that I would've liked to read.</p>
</li>
<li>
<p><em>Dangerous Games 1: How To Play</em> by Matt Forbek - How can a book where Allen dies be bad? A nice straightforward murder mystery. </p>
</li>
<li>
<p>The Afterlife Series by Mur Lafferty - Great fun! Being a god isn't all it's cracked up to be! Also: unintended consequences with a chaser of steampunk!</p>
</li>
<li>
<p><em>New Tales of the Yellow Sign</em> by Robin D. Laws - Some horrific, some sick-making. Overall, I'm not a huge fan of either of those, so this one got a solid "okay" from me.</p>
</li>
<li>
<p><em>A Prayer for Dead Kings and Other Tales</em> by Scott Fitzgerald Gray - One part Fafrd and the Grey Mouser, one part Saberhangen's Swords with a dash of Moorcock. Fun but not amazingly memorable.</p>
</li>
<li>
<p><em>Doc Wilde and the Frogs of Doom</em> by Tim Byrd - If Indiana Jones and Lara Croft had Tom Swift as a kid, their present-day descendants might be the Wildes! A family-friendly adventure romp that I've put on the list of books to read to my kids... with notes to go look for more.</p>
</li>
</ul>Blogging by Pelican2013-05-12T00:00:00-04:002013-05-12T00:00:00-04:00pjtag:place.org,2013-05-12:/~pj/blog/blogging-by-pelican.html<p>I stumbled across <a href="http://getpelican.com">Pelican</a> and was
quite stricken - I've wanted something similar for a long time.
So! Since my current blog is an ancient mess, but the content
was salvagable (thanks to it all being in text files), I converted
them all to Pelican and will maybe start blogging again …</p><p>I stumbled across <a href="http://getpelican.com">Pelican</a> and was
quite stricken - I've wanted something similar for a long time.
So! Since my current blog is an ancient mess, but the content
was salvagable (thanks to it all being in text files), I converted
them all to Pelican and will maybe start blogging again. Maybe.</p>Transcoding mpeg2 to mp4 with avconv2013-01-22T00:00:00-05:002013-01-22T00:00:00-05:00pjtag:place.org,2013-01-22:/~pj/blog/transcoding-mpeg2-to-mp4-with-avconv.html<p>I'm using mediatomb to share my media files across my home LAN, and a friend pointed me at
<a href="https://github.com/NfNitLoop/stream-chan">stream-chan</a> which would
stream live TV from my HDHomeRun via DLNA. Way cool! Except that the
main endpoints in my network are phones and tablets, which are none too
happy about trying …</p><p>I'm using mediatomb to share my media files across my home LAN, and a friend pointed me at
<a href="https://github.com/NfNitLoop/stream-chan">stream-chan</a> which would
stream live TV from my HDHomeRun via DLNA. Way cool! Except that the
main endpoints in my network are phones and tablets, which are none too
happy about trying to play mpeg2 streams, to say nothing of my poor
gasping wifi network which ends up crippled due to the 30+ Mbps that a
single stream wants to sustain. So clearly some transcoding was called
for. I chose mp4 as most devices have hardware assist when decoding
x264+mp3, so even the oldest and slowest would have at least a chance to
play it.</p>
<p>The incantations for this were gleaned from all over the net and also avconv manpages. This being so, and with many non-obvious problems along the way, I figured sharing it here might be helpful to the world at large.</p>
<div class="highlight"><pre><span></span><code>avconv -i - -codec:v libx264 -s vga -preset ultrafast -f mpegts -codec:a libmp3lame -ac 2 -y OUTFILE
</code></pre></div>
<p>So now, an explanation:</p>
<p>-i - means 'input from stdin'
-codec:v libx264 means use the libx264 codec for video output
-s vga means make vga-sized output (in a bid to lower computation cost)
-preset ultrafast is an argument to the libx264 codec to tell it to be fast rather than accurate
-f mpegts means output format of mpegts
-codec:a libmp3lame means transcode to mp3 output
-ac 2 means use the first two channels and drop the rest because x264 doesn't handle >2 audio channels
-y means overwrite</p>
<p>After all that, it seems my ancient little NAS is too slow a machine to do realtime A/V transcoding. Ah well, at least I know <em>how</em> now - new hardware is a different issue. And maybe this will help someone!</p>Strawberry Chili Shake2012-06-04T00:00:00-04:002012-06-04T00:00:00-04:00pjtag:place.org,2012-06-04:/~pj/blog/strawberry-chili-shake.html<p>Provided to me by some friends several years ago.</p>
<h1>Recipe</h1>
<p>In a bowl, whisk together 2 cups yogurt, 1/4 cup orange juice, 1 cup
chopped mint leaves and 1 tablespoon sugar until mint is fragrant;
strain and discard solids. In another bowl, combine 2 cups sliced
strawberries , 2 tablespoons …</p><p>Provided to me by some friends several years ago.</p>
<h1>Recipe</h1>
<p>In a bowl, whisk together 2 cups yogurt, 1/4 cup orange juice, 1 cup
chopped mint leaves and 1 tablespoon sugar until mint is fragrant;
strain and discard solids. In another bowl, combine 2 cups sliced
strawberries , 2 tablespoons lime juice, 2 tablespoons sugar and 1
teaspoon chili powder; refrigerate bowls for 2 hours, stirring once. To
serve, spoon yogurt onto strawberries and stir. Garnish: Whipped cream
and mint.</p>
<h1>Translation:</h1>
<h2>Blend:</h2>
<ul>
<li>2c yogurt </li>
<li>2c strawberries </li>
<li>1c mint leaves </li>
<li>3T sugar</li>
<li>.25c OJ</li>
<li>2T lime juice</li>
<li>1t chili powder</li>
</ul>
<p>or:</p>
<h1>Strawberry-Orange Soup</h1>
<p>(Scott and Christa's version)</p>
<h2>Ingredients:</h2>
<ul>
<li>2 c Yogurt</li>
<li>2c Orange juice or Orange/Mango juice or Orange/Pineapple juice</li>
<li>1 bunch Mint leaves</li>
<li>3 Tbsp Sugar</li>
<li>1 lb Strawberries, destemmed</li>
<li>1 Lime, juiced</li>
<li>1 tsp Chili powder</li>
<li>Whipped cream (optional topping)</li>
</ul>
<h2>Preparation</h2>
<p>Blend all ingredients in Vitamix until smooth. Top with whipped cream if desired.</p>Fixing APT Corruption2006-11-15T17:18:26-05:002006-11-15T17:18:26-05:00pjtag:place.org,2006-11-15:/~pj/blog/fixing-apt-corruption.html
A debian machine at work had some RAM go bad... in service. Which ended up corrupting something in /var/cache/apt, (I think pkgcache.bin). Which caused apt-get to segfault repeatedly. I had traces like
<pre>
gettimeofday({1163625872, 123448}, NULL) = 0
gettimeofday({1163625872, 123572}, NULL) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0 …</pre>
A debian machine at work had some RAM go bad... in service. Which ended up corrupting something in /var/cache/apt, (I think pkgcache.bin). Which caused apt-get to segfault repeatedly. I had traces like
<pre>
gettimeofday({1163625872, 123448}, NULL) = 0
gettimeofday({1163625872, 123572}, NULL) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
</pre>
and gdb traces like:
<pre>
Reading Package Lists... Done
Building Dependency Tree... 50%
Program received signal SIGSEGV, Segmentation fault.
0xb7ec84d7 in pkgDepCache::CheckDep ()
from /usr/lib/libapt-pkg-libc6.3-5.so.3.3
(gdb) bt
#0 0xb7ec84d7 in pkgDepCache::CheckDep ()
from /usr/lib/libapt-pkg-libc6.3-5.so.3.3
#1 0xb7ec918c in pkgDepCache::DependencyState ()
from /usr/lib/libapt-pkg-libc6.3-5.so.3.3
#2 0xb7ec95c7 in pkgDepCache::Update ()
from /usr/lib/libapt-pkg-libc6.3-5.so.3.3
#3 0xb7ec825f in pkgDepCache::Init ()
from /usr/lib/libapt-pkg-libc6.3-5.so.3.3
#4 0xb7eea054 in pkgCacheFile::Open ()
from /usr/lib/libapt-pkg-libc6.3-5.so.3.3
#5 0x0805e277 in std::vector<pkgsrcrecords::parser::builddeprec, std::allocator<pkgsrcrecords::parser::builddeprec> >::_M_insert_aux ()
#6 0x08055069 in ?? ()
#7 0xb7ebcf3c in CommandLine::DispatchArg ()
from /usr/lib/libapt-pkg-libc6.3-5.so.3.3
#8 0x0805d706 in ?? ()
#9 0xb7c8f974 in __libc_start_main () from /lib/tls/libc.so.6
#10 0x0804c5b1 in ?? ()
</pre>
so if you run into this, just:
<pre>
mv /var/cache/apt/ /var/cache/apt-
mkdir -p /var/cache/apt/archives/partial
</pre>
and then 'apt-get update' and things will be fixed.
War on Terror Jingoism2006-09-21T12:40:44-04:002006-09-21T12:40:44-04:00pjtag:place.org,2006-09-21:/~pj/blog/war-on-terror-jingoism.html
<p>
A relative forwarded me an email spewing a bunch of jingoistic
garbage, and I responded:
<p>
Radical Muslims:Islam :: KKK:Christianity
<p>
I'm not always an Andrew Sullivan fan, but the letter he
<a href="http://dish.andrewsullivan.com/2006/09/15/what_weve_lost/">
reprints</a> vividly demonstrates where attitudes like the below
will get us - where they've already gotten us. "Liberty and Justice …</p></p></p>
<p>
A relative forwarded me an email spewing a bunch of jingoistic
garbage, and I responded:
<p>
Radical Muslims:Islam :: KKK:Christianity
<p>
I'm not always an Andrew Sullivan fan, but the letter he
<a href="http://dish.andrewsullivan.com/2006/09/15/what_weve_lost/">
reprints</a> vividly demonstrates where attitudes like the below
will get us - where they've already gotten us. "Liberty and Justice
for all"... be careful how you redefine 'all' lest it be redefined to
exclude _you_ sometime in the future.
<p>
The way to fight the war on terror is to not be afraid, to not act out
of fear, and to treat terrorists like the criminals they are instead of
lowering yourself into their world of hate and lies.
<p>
SWAMI and pyward2005-03-18T16:44:48-05:002005-03-18T16:44:48-05:00pjtag:place.org,2005-03-18:/~pj/blog/swami-and-pyward.html
So the other day I went looking for something to manage .htaccess files
via a CGI. They're a mild pain to edit manually, so I figured someone
had run into this problem before. I was right - but everyone wanted
$25ish for their solution. So I decided it couldn't be that …
So the other day I went looking for something to manage .htaccess files
via a CGI. They're a mild pain to edit manually, so I figured someone
had run into this problem before. I was right - but everyone wanted
$25ish for their solution. So I decided it couldn't be that hard and set
off to 'scratch my own itch' as most programmers do.
<p>
Managing an entire .htaccess seemed a bit big to bite off at once
(since I didn't remember that much about the format), so I decided to
get htpasswd/htgroup-file managment out of the way first. A simple
little webapp emerged, named <a href="/~pj/software/swami">SWAMI</a> (Simple Web
Authorizaion Management Interface).
<p>
As I was writing it, I decided it would be cool to make it nice and
RESTful, because one of the side effects of doing so is that it
makes sure I don't forget anything. So I reorganized the code some
and made the URI space and actions RESTful. As I was doing so, I
found it annoying to rewrite a bunch of the same if-then statements
over and over, so I decided to refactor the code into a framework.
I've done this before in a different context, and I spent some time
looking at <a href="http://www.mnot.net/tarawa">Tarawa</a>, so I
soon had something simple and flexible.
This framework then got moved into its own library, which I called
<a href="/~pj/software/pyward/pyward.py">pyward</a> (Python Web App RESTful
Dispatcher).
<p>
So now I've got this RESTful framework which at the moment deals with being
a CGI but could easily be bolted onto the side of an HTTPServer class.
Politics-Oriented Software Development2005-02-01T01:00:00-05:002005-02-01T01:00:00-05:00pjtag:place.org,2005-02-01:/~pj/blog/politics-oriented-software-development.html
Originally found <a href="https://web.archive.org/web/20050225100608/http://www.kuro5hin.org/story/2005/1/28/32622/4244">here</a>
<pre>
By TheophileEscargot
Fri Jan 28th, 2005 at 11:53:03 AM EST
Software
A brief guide to software development in the real world. Aimed mainly
at new developers: experienced programmers already know most of this.
This guide is for hands-on programmers, not managers.
</pre>
BREAK
<pre>
Introduction
Most textbooks …</pre>
Originally found <a href="https://web.archive.org/web/20050225100608/http://www.kuro5hin.org/story/2005/1/28/32622/4244">here</a>
<pre>
By TheophileEscargot
Fri Jan 28th, 2005 at 11:53:03 AM EST
Software
A brief guide to software development in the real world. Aimed mainly
at new developers: experienced programmers already know most of this.
This guide is for hands-on programmers, not managers.
</pre>
BREAK
<pre>
Introduction
Most textbooks and methodology guides start with statistics proving
that software projects are disastrously prone to failure. I'll take that
for granted.
They then propose complicated rules designed for a fantasy business
world where cooperative workers idealistically work towards the common
goal of maximizing shareholder value.
A couple of weeks in a typical office should disabuse you of that.
Real world companies are made up of individuals working for their own
goals: career advancement, more money, or just the chance to slack off.
Occasionally it may be in someone's interest to build a successful
project. Usually it's more useful to sabotage it.
This leads to the real reason for the perpetual software crisis:
1. Most software fails because it is designed to fail
Why you should want your project to succeed
A developer rarely benefits from the failure of his project. There
are exceptions, such improving your resumé by choosing bleeding-edge
technology that may also cripple the project. However, few developers
escape the consequences: they may be stuck maintaining an inadequate
system long after the project manager has moved on to better things.
In addition, since management is a less specialized skill, it is
easier for a manager to change jobs than it is for a developer.
Sabotage
There are two kinds of sabotage: deliberate and incidental.
The direct manager of a project has an incentive to see the project
succeed. As we will see later, he may also have incentives to make the
project fail, but generally he wants to get it working, at least in the
short term until he gets promoted.
In most companies, promotion is competitive. This means that the
direct manager's competitors have an incentive to make him fail:
deliberate sabotage. This tendency is most prevalent in non-software
houses that have several teams of in-house developers. It obviously
doesn't arise if there is only one software development manager, and in
software houses the chance of expansion renders the competition less
fierce.
Because everyone has to pay lip service to the good of the company,
deliberate sabotage has to be disguised as incidental sabotage.
Nevertheless, be aware that it's a good idea to keep sensitive
information, such as estimates and necessary purchases away from
potential rivals. Beware of casual conversations with the enemy.
2. Loose lips sink projects
Incidental sabotage is more widespread. Anyone who has input into
the project will be working for their own ends. Examples: the legal
department want to make things safe for them by insisting on huge legal
disclaimers at the expense of usability. Designers want to maintain an
impressive portfolio by using flash displays at the expense of bandwidth
and small footprints.
Analysis
Some early software methodologies advocated a 40-20-40 model: spend
the first 40% of time on analysis and design, 20% of the time building
the system, 40% of the time on testing. Modern methodologies tend to
focus on incremental development: build a prototype and continuously
improve.
Neither of these are acceptable to the business commissioning the
project. Accounting want to know the cost of the project when planning
the fiscal year. HR want to know how many people to hire or fire. No-one
is going to authorise 40% of a project's likely budget to find out what
it will cost to build. No-one is going to sell the idea of building a
system first then finding out exactly what it will do later.
Therefore, you will continue to develop the traditional way: take a
guess based on inadequate analysis and pray that you've padded it with
enough contingency.
The project will probably overrun, but see incidental sabotage:
that's not the accountants' problem.
3. Don't trust the analysis
Descoping
As the system is being built, the consequences of the poor analysis
will show up: requirements will become more complicated than you
thought.
Fortunately, due to incidental sabotage, much of the functionality
will be unnecessary. At some point, you will need to descope: reduce the
functionality so you can finish the system. There are two aspects to
doing this successfully:
4a. Descope early
As soon as possible in the project, try to work out which features
will be cancelled. For every requirement, get some explanation of why
it's there. From this you can often identify the important bits: build
those first, and structure the system so the others can be added if
necessary.
4b. Admit descoping late
While you should actually descope as early as possible, you should
admit you've descoped as late as possible.
First, the person who identifies a problem is usually blamed for
that problem: kill the bearer of bad news is the principle. Second,
admit it too early and people have the chance to add more pointless
requirements: it's safer to wait until there's panic in the air. Only
propose descoping when the project is already behind schedule.
Also remember that someone who points out a problem early is a
troublemaker; someone who fixes a problem at the last minute is a hero.
Architecture
It can be useful to make your architecture reflect organizational
lines. If one team is likely to be unreliable, try to make them
responsible for an isolated module of the system. If possible, make the
rest of the system work independently of it. In any case, make sure a
failure there cannot be blamed on something else.
Be aware that architecture charts can give management incorrect
ideas about where the workload lies. Consider this diagram:
+----------+ +-----------+ +-----------+ +------------+ +----------+
| GUI | | Docs | | I/O | | Data | | Utils | | | | | | | | | |
| | Alice | | Alice | | Alice | | Bob | | Alice | +----------+
+-----------+ +-----------+ +------------+ +----------+
On showing this breakdown to any given manager, it will be
clear that Alice has taken on the vast bulk of the project work,
while Bob is basically slacking. Bob cannot explain to the manager
that designing data structures for this project is the most difficult
component, and the rest is trivial. Additionally, any undue progress
made by Bob on the data structure can be sabotaged by Alice at any time
by changes to project modules under her control
5. Make sure architecture assigns blame clearly
Managing Management
The manager of a project usually wants it to succeed, but not
necessarily in the same way as the developers. The project manager
only needs it to work in the short term: he doesn't care if it's easy
to maintain. However, an unmaintainable project can be a trap for a
developer, who might spend years making tiny changes.
There are also circumstances where the project manager benefits
from doing things the wrong way. A manager gains status from the size of
his team: it may be advantageous to enlarge the project.
It can help to keep an eye on any magazine subscriptions held
by the manager. Try to disparage technologies you don't want to work
with, encourage useful technologies.
The goal of meetings, as far as the manager is concerned, is
to appear in control and abreast of the project. Your goal is to make
sure that they think they are in control and abreast of the project,
while keeping all details from them. Most managers will happily collude
in this, since they have no desire to know what is happening, so long
as they are seen to be so aware and are rigorously kept blissfully
ignorant of their own ignorance. Remember that managers are essentially
secretaries who can fire you.
6. Managers don't want to know the truth: keep it from them.
Documentation
Documentation is an essential tool in the twin goals of
ass-covering and of managing management. It also provides dangerous
traps for potential enemies.
The political trick is to strike the correct tone of opaque
vagueness and unshakeable authority. The true use of documentation is
to bridge the inevitable gap between what the project is supposed to do
and what it actually does. Politically written documentation bridges
this gap by appearing to claim the former without actually denying the
latter. On close examination, it will be found to say nothing at all.
This is the true origin of the fallacy that code is its own
best documentation: The code itself is the only document which may be
relied upon in any way in order to find out how the software might
actually behave under this or that set of circumstances. However, anyone
actually saying so only reveals themselves as a troublemaker.
There is also a three-way conflict between the interested
parties:
1. The people specifying the documentation want it to be
as detailed as possible, in as much volume as possible. The more
documentation, the better, as this increases their importance.
2. The people writing the documentation want it as effortless
as possible
3. The people eventually using the documentation want it as
accurate as possible.
The usual compromise between 1 and 2 is to create vast amounts
of inaccurate, out-of-date documentation. Since it's never updated, it's
usually useless to 3.
The best way to sneak useful documentation past the system
is to create a brief "Overview" document, small enough to be kept
up-to-date. This will at least give maintenance programmers an idea of
what the system is supposed to do.
7. Keep documentation brief enough to be kept up-to-date
If possible, try to get accurate documentation created:
otherwise you will find yourself doing endless maintenance and support
on a system that only you understand.
Ass-covering
Document everything. Preserve your old emails. If possible,
keep them in a personal archive so that you can use them, but they
cannot be used against you.
Especially if you're busy, keep a record of what you've done
every day: either in your official timesheets or in private notes.
Especially when under pressure it can be tempting to leave them to
later: but this leaves you dangerously vulnerable when the blamestorming
begins.
When asked incredulously "how on earth did it take you so long
to do this", you need to be able to say first: "Because I had to do
X, Y and Z and then undo them", and point to your record as evidence.
When asked "why on earth did you do that?" you need to be able to say
"because you told me to in this email sent on this date."
In turn, the emails you sent will be used in evidence against
you. Keep a professional tone: before sending any sensitive email take a
moment to think how it would look at an industrial tribunal.
The chief difficulty is reaching a satisfactory compromise
between ass-covering and not appearing too negative. If you know
something is going to fail, make sure you point it out and have a
record, but try to present it in a positive way. Say that it is a
"major risk", rather than a certain failure. Try to request additional
resources or time even when you know they will be denied.
8. Preserve records privately
Error messages and logfiles
As well as being later than you expect, the system will be less
reliable than you expect. Make sure your debug and logfiles give you
plenty of information. As with architecture, make sure that your error
messages assign blame appropriately.
Overtime
As the deadlines whoosh past, the pressure will grow to do
unpaid overtime. Reduced concentration and the unavailability of other
people mean that generally an hour of overtime achieves much less than
an hour of normal time, so it's not very effective.
The thing to remember is that your line manager wants you to
do overtime because he's being pressured by his managers. In many cases
he doesn't want you to do overtime, since he knows that the extra bugs
aren't worth it, but to be seen to be doing overtime. The key rule is:
9. Overtime only counts when people see it
It's usually better to do overtime in the evening than the
morning, if there are more people around to see it. Weekend overtime is
usually pointless since there are rarely witnesses.
Remember that you're not just being visible to your boss, but
the office as a whole. Your overtime is useless to him unless it's
visible to his boss, or else other people who contribute to your team's
reputation.
Keep to the minimum possible. Remember that the earliest part
is most valuable since there are more witnesses: better to do half an
hour Monday to Thursday than two hours on Wednesday. It also sounds
better to say: "I've worked late four nights this week." No-one will be
keeping track that closely anyway.
Finally, remember that with deadlines pressing, the last thing
your line manager wants is to fire you. Threats are likely to be empty
in the short term, and quickly forgotten when the project is over. As
part of his empire, you are contributing to his status: he doesn't want
to get rid of you even if you're useless. You're only likely to be fired
if the company as a whole is reducing numbers, and in that case no
amount of work guarantees safety.
This article was created as a collaboration on ko4ting the K5
wiki by clover kicker, motty, R Mutt, Scrymarch and TheophileEscargot.
</pre>
*solutions2004-03-25T12:54:48-05:002004-03-25T12:54:48-05:00pjtag:place.org,2004-03-25:/~pj/blog/solutions.html
Reading <a href="http://www.freedom-to-tinker.com/archives/000559.html">this entry</a> on Freedom to Tinker about why Fraunhaufer's 'Lightweight DRM' is doomed made me think, 'Yet again, well meaning people are championing ideas long shot down,' and reminded me of Cory Doctorow's <a href="http://www.craphound.com/spamsolutions.txt">form letter</a> for people who think they've solved spam. So I decided to take a first …
Reading <a href="http://www.freedom-to-tinker.com/archives/000559.html">this entry</a> on Freedom to Tinker about why Fraunhaufer's 'Lightweight DRM' is doomed made me think, 'Yet again, well meaning people are championing ideas long shot down,' and reminded me of Cory Doctorow's <a href="http://www.craphound.com/spamsolutions.txt">form letter</a> for people who think they've solved spam. So I decided to take a first pass at <a href="http://www.place.org/~pj/drmsolutions.html">one</a> for people who think they've solved DRM. Suggestions welcome!
Winer on Techies2003-10-13T13:06:01-04:002003-10-13T13:06:01-04:00pjtag:place.org,2003-10-13:/~pj/blog/winer-on-techies.html
I can't believe Dave Winer thinks that <a href-"http: blogs.law.harvard.edu bloggercon notjustgeeks">this opinion</a> is news.
<p>
Business has always been about find a need (ie. a customer's desire) and answering it. However, I think he oversimplifies and doesn't give the techies their due when he says that 'Vision isn't coming from [them]"; customers have needs …</p>
I can't believe Dave Winer thinks that <a href-"http: blogs.law.harvard.edu bloggercon notjustgeeks">this opinion</a> is news.
<p>
Business has always been about find a need (ie. a customer's desire) and answering it. However, I think he oversimplifies and doesn't give the techies their due when he says that 'Vision isn't coming from [them]"; customers have needs, but the way they envision them being filled is limited by their understanding of the technologies involved (a farmer with no knowledge of internal combustion just wants more oxen to plow his fields), just as the way techies envision filling them is limited by the tech's understanding of the need involved (plowing a field in under an hour is no real advantage if it still takes days to actually plant the field). Working together is essential to get the best result (and what the farmer really needs is a way to speed up the entire field-readying, planting, harvesting process).
<p>
None of this should come as a suprise to anyone who's done any amount of consulting or tried to run a business. If it's news to Mr. Winer, well, I'm glad he finally figured it out.
Photo Album software2003-10-07T05:48:53-04:002003-10-07T05:48:53-04:00pjtag:place.org,2003-10-07:/~pj/blog/photo-album-software.html
I'm looking for a CGI to run locally that will turn my current mishmash of digital photos in a nice album.
Some features I'd like:
<ul>
<li> the fewer actual CGIs the better
<li> no MySQL necessary - it just seems like overkill, not to mention too much fragility
<li> automatic generation of thumbnails and …</li></li></li></ul>
I'm looking for a CGI to run locally that will turn my current mishmash of digital photos in a nice album.
Some features I'd like:
<ul>
<li> the fewer actual CGIs the better
<li> no MySQL necessary - it just seems like overkill, not to mention too much fragility
<li> automatic generation of thumbnails and indexes
<li> bonus points for slideshow functionality
<li> some amount of permissions - I'd like to keep some pics private and control access to other ones
</ul>
I intend to update this if I find anything that matches. If *you* find anything that you think matches, please email me or comment here.
LiveJournal enabled2003-07-24T13:52:50-04:002003-07-24T13:52:50-04:00pjtag:place.org,2003-07-24:/~pj/blog/livejournal-enabled.html
Well, I finally caved and got a <a href="http://www.livejournal.com/users/bitjuggler">LiveJournal Account</a>, just so I could comment on other people's stuff.
Open letter to John Harell2003-06-12T06:12:58-04:002003-06-12T06:12:58-04:00pjtag:place.org,2003-06-12:/~pj/blog/open-letter-to-john-harell.html
To: <a href="http://homepage.mac.com/jharrell/blog/entry-54.html">John Harell</a><br>
Subject: absolute copyright<br>
<p>
> When I create something, that thing is mine.
<p>
I'm a programmer, so I know what you mean. I write software and bring it
into the world with much blood, sweat and tears along the way. And yet...
<p>
I didn't create the language the software …</p></p></p>
To: <a href="http://homepage.mac.com/jharrell/blog/entry-54.html">John Harell</a><br>
Subject: absolute copyright<br>
<p>
> When I create something, that thing is mine.
<p>
I'm a programmer, so I know what you mean. I write software and bring it
into the world with much blood, sweat and tears along the way. And yet...
<p>
I didn't create the language the software was written in.
<p>
I didn't design the standard interface my software uses to communicate
with the user.
<p>
I didn't design the operating system my software runs on.
<p>
I didn't design or produce the hardware on which it runs.
<p>
I didn't independently teach myself all the skills it takes to write
software - sure, I tried my best to learn, but people from my parents to
my co-workers, and not a few authors (and publishers, retailers, etc) of
books, have impacted my experience and therefore my skills along the way.
<p>
So if all those people and things contributed to my being able to write
software, should not they also have a claim on my works? After all,
they created me.
<p>
Hopefully food for thought,
<p>
--pj
Word unter alles2003-04-24T04:26:17-04:002003-04-24T04:26:17-04:00pjtag:place.org,2003-04-24:/~pj/blog/word-unter-alles.html
<a href="http://www.goldmark.org/netrants/no-word/">Here</a> is a great essay on one of my pet peeves, using Word (or excel, though there are fewer open alternatives) as a document exchange format. It used to be that you could advocate a neutral format by pointing out 'what if I don't have Word, but instead use WordPerfect …
<a href="http://www.goldmark.org/netrants/no-word/">Here</a> is a great essay on one of my pet peeves, using Word (or excel, though there are fewer open alternatives) as a document exchange format. It used to be that you could advocate a neutral format by pointing out 'what if I don't have Word, but instead use WordPerfect or Wordstar or AMIPro', but those products are victims of the microsoftopoly, so you have to mention <a href="http://www.openoffice.org">OpenOffice</a> or <a href="http://www.lyx.org">LyX</a> or something, at which point people look at you funny (though I've never understood why someone else's ignorance causes <em>them</em> to look at <em>me</em> funny if I point out something they didn't know about... I don't look at <em>them</em> funny just because they're ignorant) . *sigh* Note that this entry is here at least somewhat to remind me to point people at the above link the next time I get a word doc in email.
Boot Scripts2003-04-10T14:15:53-04:002003-04-10T14:15:53-04:00pjtag:place.org,2003-04-10:/~pj/blog/boot-scripts.html
Talking to a friend the subject of SysV-init vs BSD-style init
came up. I like SysV's /etc/init.d component because it gives
you simple on/off controls for all the system daemons, all in
the same place, and named predictably. I don't particularly care
for the 'runlevels' idea because …
Talking to a friend the subject of SysV-init vs BSD-style init
came up. I like SysV's /etc/init.d component because it gives
you simple on/off controls for all the system daemons, all in
the same place, and named predictably. I don't particularly care
for the 'runlevels' idea because they seem to be different on
different systems despite an 'official' definition, and they also
seem... inflexible. I remembered running across Richard Gooch's
<a href="http://web.archive.org/web/20091006063658/http://www.atnf.csiro.au/people/rgooch/linux/boot-scripts/">Linux Boot Scripts</a>
awhile back on <a href="http://web.archive.org/web/20050719011007/http://sweetcode.org/">s weetcode</a>
(if you're a sysadmin or programmer and you haven't been
there, go now, explore the archives, and finish reading this when you're
done... which means I won't expect you to finish reading this until
maybe tomorrow, or the day after, or perhaps next week) and wondered
if there was a <a href="http://www.debian.org">debian</a> package for
'em. Sure enough, there wasn't - despite debian using the 'util-linux'
package, they also use the separate 'sysvinit' package instead of the
simpleinit that comes with util-linux. So, disappointingly, there's no
way for me to try out the LBS dependency-based init scheme... unless I
build it myself. But if I'm going to build it myself, I might as well
package it up so others can use it too, right? and submit the patch to
the maintainer...
Texas Action Network2003-04-07T03:51:55-04:002003-04-07T03:51:55-04:00pjtag:place.org,2003-04-07:/~pj/blog/texas-action-network.html
I wish there were more things like the
<a href="http://www.texasep.org/texan/">Texas Action Network</a>; It
seems a useful feedback into the legislative process. But I also wish that it didn't supply form-letters - they're more likely to be ignored. Paradox: make it easy to show a unified front on an issue (via form letters …
I wish there were more things like the
<a href="http://www.texasep.org/texan/">Texas Action Network</a>; It
seems a useful feedback into the legislative process. But I also wish that it didn't supply form-letters - they're more likely to be ignored. Paradox: make it easy to show a unified front on an issue (via form letters) and you get ignored as spam. Make it hard, and no one participates.
Geeks & Politics2003-03-31T05:08:43-05:002003-03-31T05:08:43-05:00pjtag:place.org,2003-03-31:/~pj/blog/geeks-politics.html
<a href="http://cyberlaw.stanford.edu/lessig/blog/archives/2003_03.shtml#001039">Lawrence Lessig's blog</a> says:
<blockquote>
Ed Felten has a wonderful piece about the idiocy in the mini-DMCA's being considered by a number of state "governments." What is so frustrating about this business is not the people (like these governments) who disagree with you. But that their disagreement reveals that they have …</blockquote>
<a href="http://cyberlaw.stanford.edu/lessig/blog/archives/2003_03.shtml#001039">Lawrence Lessig's blog</a> says:
<blockquote>
Ed Felten has a wonderful piece about the idiocy in the mini-DMCA's being considered by a number of state "governments." What is so frustrating about this business is not the people (like these governments) who disagree with you. But that their disagreement reveals that they have
not done anything to understand the issue. We are over 5 years into this battle, yet these laws look like they have been drafted by people who have lived on another planet these past 5 years.
</blockquote>
which echoes Cory Doctorw's statement in a recent <a href="http://www.strangehorizons.com/2003/20030331/doctorow.shtml">interview</a>:
<blockquote>
Turing's Universal Machine is an incredibly powerful concept; so powerful, in fact, that as a culture we still haven't gotten our heads around it, which is why we now have people calling for the design of computers that can't be used to commit infringement, or can't be used to convey bad speech. . . . The idea that you can build a general purpose computer that can execute all instructions save for ones that are infringing is kind of touchingly naive, but profoundly ignorant of what a general purpose computer is.
</blockquote>
I think these provide some insight into the fundamental geek disconnect
from the political/law process: Geeks get it, so to them trying to
legislate things like DRM is equivalent to the silly bill that tried
to make pi = 3.2: It's nonsense. And the only real solution that I
can think of, in the face of the willful ignorance of entities like the
recording industry, is to let the ripples of education get far enough
out that everyman, in the form of the people involved in the lawmaking
process, is educated enough to respond to such proposals in the same way
the would to a proposal to make a law that the sun rise in the west.
In the meantime... do what you can, but realize you're fighting an
educational battle, not an ideological one.