Meet some of the Cipher Challenge community …

There is something particularly satisfying about participating in the Cipher Challenge. Perhaps this is because of that moment of relief when the incomprehensible gibberish of the ciphertext finally resolves into a readable plaintext; or perhaps it is because every year there is the chance to make your cracking techniques more refined, your programs more efficient, your solving times a little faster than before. While the ability to write a Python program which can automatically solve Vigenere ciphers may not (in general) be the most useful skill, the Cipher Challenge encouraged me to keep on looking for ways to experiment and improve on my own previous work, trying even the most unlikely solutions to fix increasingly minor problems. Although I am, sadly, now too old to officially compete, the Cipher Challenge remains an excellent way to take a break from regular academic work; and the fact that I, among others, continue to be persistently hanging around on the forums and leaderboards long after leaving school is a clear testament to what an enjoyable and stimulating experience the Cipher Challenge is.

Some of my original methods for facing the final challenge in 2017 were a bit strange! This was mainly a result of being on a school trip when it came out, so having a limited amount of time to do it and an even more limited amount of time with a computer available to do it. This meant that I had to rely a lot less on programs than I would have liked – although I don’t think that this actually worked out too badly for me in the end.
Initially, without access to a computer I looked at the cipher on my phone (ignoring part A, on the logic it would be easy with my programs). However, when I finally had access to a laptop, about three hours after the challenge was released, I put the text into my ioc program (Ioc.py) and discovered that it wasn’t a digraphic substitution cipher at all. At this stage, I fairly easily managed to do 8A, only to discover that it didn’t really tell me anything helpful.
I ran a lot of other tests on the cipher then, including frequency analysis (letters, digraphs, trigrams, words) (Frequencycount2+.py) and ioc (every 2nd-20th letter, digraphs, both of these after simple transpositions), but didn’t find anything helpful – apart from confirmation that both ‘QN’ and ‘LA’ were common overall. I briefly considered the autokey cipher, as a cipher I had come across when researching the beaufort cipher for an earlier challenge; but after trying ‘agricola’ as the first word and receiving nonsense, I gave up on that approach. I then had a few hours when I couldn’t do the cipher. Later that evening, I had more time to look at it, so ran yet more fruitless data analysis, taking photos of all the results so that I could look at them even without the computer (unfortunately I have since deleted these photos, so cannot remember all the random things I tried). One of the things I did try was finding all the one letter words and looking at the letters before and after them, and their position in the text (Letterfind.py). I seemed to find nothing conclusive with this, although I did find some patterns between the three letters – for a bit of text ‘…A B C…’, sometimes A+B-C seemed to give me the letter ‘i’, sometimes A+C-B seemed to give me sense – but these patterns were inconsistent and didn’t give me anything truly helpful. Nothing I tried early the next morning seemed to work either.

Somehow I then managed to ignore the challenge until about half past three, when I was on a coach.

I don’t really know what made me try it – possibly I’d never truly let go of the idea of autokey – but I tried shifting the word ‘the’ one place relative to itself and adding the letters, e.g. t+?, h+t, e+h, e+?. With some surprise I discovered this gave me ?AL? – the reverse of the ‘LA’ which I had thought was ‘an’. Trying the same thing with ‘and’, I discovered that it gave me ?NQ?, the other common pair I had noted. At this stage, I knew I was getting somewhere – it appeared that ciphertext = plaintext + key, with the key being the plaintext shifted by one letter, as in an autokey cipher with a one-letter keyword, and so plaintext = ciphertext – key (key = previous plaintext character) – just, sadly, I’d made this discovery half an hour into a fourteen hour coach journey, for at least eleven hours of which I would not have access to my laptop.
Obviously, as my digraphs were reversed, I needed to reverse the ciphertext – unsurprising as 8A had been reversed, which I had thought might be a hint. Wanting to be able to decode from one end, I began with the fourth word, ‘LAX’ (forgetting, I think, that this was actually going to be the end of the text) and successfully decoded this as ‘eht’. However, when I tried to progress onto the previous word, ‘BXHWEI’, I got nonsense (‘?mlwae’). Assuming I’d made a mistake, I tried reversing the end to give ‘PKFCO ENPKMRG YAL GT OOBQHVJ’, and turned ‘YAL’ into ‘the’, but again, trying to do the other words I only got nonsense. Although ‘LAX’ looked like it could be somehow coded within itself, as it need the key ‘hte’, the same letters as the plaintext, I knew that the words could not be independently coded as ‘XAL’ and ‘YAL’ both translated as ‘the’. This was when I thought that reversing the whole text was probably too easy, and thought that maybe individual words were reversed – which also explained giving us the word spacing, which otherwise seemed far too nice (my original thought had been, ‘Oh no, it’s so hard that even with spaces we won’t get it’). This new idea was successful, and allowed me to decrypt the end of the text, ‘of the ancient world’. However, I encountered yet another problem with the start; I managed to decode from ‘the’ back to ‘??spite the’, but was then getting ‘iaspite’. I guessed that the correct word was either ‘despite’ or ‘inspite’; ‘despite’ seeming better, I chose to just accept the anomaly with the e and continued decoding, discovering that the phrase was in fact ‘yearning for respite the’. After continuing to decode manually, marking every letter which seemed to decode oddly, I noticed that it was consistently every twelfth letter of the text, which was effectively encoded with the letter ‘a’ regardless of what the key letter should have been. I was lucky to make the discovery then, as it meant that when I eventually wrote a program I could account for this.
At this stage, I still had about ten hours until I would be able to use my laptop, so I proceeded to spend the night trying to decode it manually – which, on a coach, working by the light of my phone in the middle of the night, was not at all fun. At the end of the night, I had only decoded about a third of the text, but at this point I was able to spend an hour and a half on the ferry using my laptop. In this time I wrote a (very glitchy) program (2017 8Bx) to solve the cipher, and wasted more time trying to solve the glitches; when I got off the ferry I realised that I could use the glitchy program to solve the cipher, in conjunction with a vigenere program (Vigenere.py) I already had.
Therefore, when I got home I ran the program – the first major problem was that it could not run to completion, but I simply told it to print the plaintext after every iteration, and I had already manually decoded the final phrase; and for some inexplicable reason, the even characters had been moved six down (I had made the false assumption that ord(“a”) and ord(“A”) were 26 apart), which was solved using my vigenere program. I had only been able to tell what had happened to alternate letters due to knowing how the plaintext should start: in this way having already decoded the start was helpful, although I did not need anywhere near as much correct text as I had.
I then only had to find the secret message. I was suspicious of the use of the word ‘yearning’, as it seemed a bit of an odd choice, so I assumed that the ‘y’ was the first letter. I wasted more time than I should have done looking for evenly spaced letters, before it hit me that punctuation must have been given for a reason. After comparing my deciphered, punctuationless text with the punctuated original, I found the message, and realised that I’d finished, far quicker than I’d expected. At no stage did I think of the 12×12 hill cipher; I was treating the cipher as a variant of the autokey, as this seemed to make sense with the other ‘imperial’ ciphers.