#060

Another developer on the team has been looking into using Learning to Rank to improve our search results, as it’s what all the cool people are doing. We had a long pairing session on Friday getting the code into a more production-ready state so we can investigate it more properly.

I finished off my spelling suggestions investigation and came up with a few candidates to try out. One of those is getting the top 5 spelling suggestions from Elasticsearch, and doing our own ranking of them—because Elasticsearch seems to give far too much weight to term frequency, compared to term similarity. We want to swing the balance in the other direction, but figuring out how much will be tricky.

I killed off a legacy search endpoint which was only used by whitehall. All the tech debt big enough to track which has been fixed this quarter has been search-related: either we’re super-productive, or we’ve got a lot of free time.

I read A Crown of Swords (by Robert Jordan), the seventh book in the Wheel of Time series.

I decided to learn some Kubernetes, and how better to learn a container orchestration system than with a distributed system I already know well? So my weekend project has been getting GOV.UK running on Kubernetes, though all I’ve managed so far is finder-frontend…

My plan is to get a bunch of the frontend services running against the public GOV.UK APIs then, once those all seem to be working, progressing backwards through the stack: getting content-store and MongoDB working, search-api and Elasticsearch, and so on.

I wrote a memo on using GADTs for alternative datatype representations.