There is a secret war, and it’s taking place all around you. It’s being fought through the entertainment you enjoy, through the money you spend, even through the air that you breathe. In this war, most heroes will die unrecognised, and sides switch by the day. Although the world’s greatest military powers are involved, even an unarmed individual can make a huge difference to the battlefield, given sufficient ingenuity.
Taking a quick breather from all this post-millennial starry-eyed hyperbole, what we’re talking about is the ever-escalating war between cryptographers and cryptanalysts – that’s roughly code makers and code breakers. This war occasionally spills over into the ‘real’ world (Stuxnet exploited certain encryption techniques to spread and, ultimately, disrupt Iran’s nuclear program), but even when limited to the ‘virtual’ world of financial transactions and personal details, it is of immense importance.
Codes And Ciphers
Image by Eivind Lindbråten
Cryptography is usually introduced with the Caesar cipher, followed by a direct link to modern ciphers and cryptography. That’s because ciphers, unlike codes, are most relevant to the vast majority of modern cryptography. The difference between ciphers and codes is important, and we can show it most easily with a familiar example:
The quick brown fox jumps over the lazy dog.
Using a cipher, we might shift the letters of the alphabet by a particular number (let’s choose the number three, as that’s what Caesar supposedly used), and end up with:
WKH TXLFN EURZQ IRA MXPSV RYHU WKH ODCB GRJ.
Using a code, we might instead use special words known only to us to denote key terms and events:
The quick brown HAWK FLIES over the lazy EAGLE.
When ciphers dominate the field of cryptography, cryptanalysis becomes more about mathematics and statistics. When codes dominate the field of cryptography, cryptanalysis becomes more about meaning and language. In both cases, unusual levels of skill across both fields are required.
Ciphers are easy to use for encoding and decoding, simply requiring some form of algorithm, and are hard to break even if you know the way they are created. However, once a flaw is found the cipher is no longer usable for serious purposes, while codes are a little more flexible and may be altered or adapted. This is especially true of so-called ‘idiot codes’, such as we have used here – we’ve presumably arranged with the recipient to read “fox” for any instance of “hawk”, “dog” for any instance of “eagle” and “flies” for any instance of “jump”. Without the arrangement, the recipient can probably guess at the meaning, but they can’t work out specifics.
While ciphers are more widespread in modern cryptography (e.g. scrypt, PBKDF2, and SHA-256 in Bitcoin), codes can still be used to add an extra ‘layer of cryptography’ to secret messages, potentially rendering some sorts of attack useless. The reason that codes are not frequently used in much computer-based cryptography is that if you know what the code is based on, the code is useless – but you can know the exact process used to create a good cipher and still be unable to decipher it.
The history of cryptology (cryptography and cryptanalysis together) has much in common with those sequences in old cartoons in which animals pulled increasingly outsized weapons on each other until they reached cosmic proportions.
The first weapon to be used by cryptanalysts against ciphers such as the Caesar cipher was frequency analysis, a basic statistical approach that you’ve used if you’ve ever tried to solve a ‘cross-reference’ or ‘code breakers’ puzzle.
This cryptanalytic technique was discovered in roughly the 9th century, and meant that a majority of known ciphers were decipherable until the 15th and 16th centuries, when polyalphabetic ciphers became widely appreciated. The most famous of these was known as the Vigenère cipher, which is still often referred to as le chiffre indéchiffrable despite the fact it was déchiffred in the 19th century.
This is when the first hints of changes to come came to light. Machines had already been an optional extra for cryptography, initially as crude as sticks and strips of leather but increasingly sophisticated and even beautiful.
Henri II Cipher Machine by ‘Uploadalt’
The man who first solved a polyalphabetic cipher, Charles Babbage, is perhaps better known as the father of the computer. Although analytical engines played no known part in the deciphering of the polyalphabetic cipher, from this point on computers and cryptology would only become more and more intertwined. Not long after Babbage’s success, a general method for attacking polyalphabetic ciphers was proposed by Kasiski, and improved by Kerckhoff, effectively ending the reign of the so-called ‘indecipherable cipher’.
Cryptographers were generally on the back foot during the late 19th and early 20th century, with a majority of ciphers being crackable until World War 1. During World War 1 itself, difficult codes and complex combinations of cipher techniques started to come in to use. These, too, had their problems. It was the interception and decryption of the infamous Zimmerman telegram* that led to America joining the war, and arguably the breaking of the ADFGX cipher that enabled the Allied Powers to halt the 1918 Spring Offensive and cement their final victory.
The period following 1918 greatly resembled the future predicted by Babbage’s involvement with the field. Analogue technologies, and the increasingly complex algorithms that determined their operation dominated the era. Even the most crankish of these inventions, such as the Chaocipher, held up relatively well if the base algorithm was hidden, but of course the most famous and one of the toughest to crack was the Enigma machine.
Recently this incredible machine has been brought back to public attention by dramatizations of the important work carried out at Bletchley Park. In chronological order and, coincidentally, in order of increasing accuracy, we have seen major films such as U-571 (which failed to acknowledge the British), Enigma (which failed to acknowledge Alan Turing), and Bletchley Park (which failed to acknowledge the Polish) all taking on the telling of the story of the Enigma machine since the year 2000.
Automatic & Enigmatic
The Enigma machine was first designed, along with many other cryptographic machines, in the late 1910s and early 1920s. The initial design was somewhat limited compared to the complex machine used in World War II, being a relatively simple rotor machine. A rotor machine is basically an automated polyalphabetic system, with additional possible features such as changing the rotor position when certain conditions are met. Even a relatively simple three-rotor design could be equivalent to using 17,568 substitution alphabets! Using simpler mechanical cipher systems such as Jefferson disks, such a feat would require a cipher disk for each alphabet, resulting in a machine of between 88 and 176 metres in height. Alternatively, it would require an extremely diligent and accurate human operator to use the cylinders in such a way as to simulate a rotor machine.
The Enigma machine, with its interchangeable rotors, had a further six configurations – leading to a potential 105,456 possible arrangements. In practice, it had even more – an incredibly large number. Polyalphabetic systems with so many alphabets were just not practical prior to rotor machines, as speed of encryption was as much a factor as it is for modern commercial encryption algorithms.
You can check out an approximation of the Enigma machine’s workings here, if the site is still working.
Previously, the ingenuity of ciphers was limited to the ability of the human mind to reliably implement such ciphers. Now, the limitations on what could be achieved were mostly physical. Some early Enigma machines with many rotors were tried, but jammed because of the complexity and number of moving parts, for example.
The Enigma machine was not necessarily the biggest cryptographic stick around on release. However, by constantly adding figurative nails and shards of glass, it quickly became a brutally effective weapon of war. The Reflector was added in 1926, doubling the number of times each letter would pass through the rotor system. British cryptanalysts cracked this version, but were initially thwarted by the addition of a plugboard. In 1932, this version was cracked by Polish cryptanalysts, but the process of continuous improvement (unlike the sudden and catastrophic breakthrough seen in the film versions) meant that Enigma needed to be cracked over and over again as, for instance, the Navy added an extra rotor, or expanded the set of possible rotors that could be used.
Dawn Of The Computer Age
Colossus Computer by Antoine Tavenaux
The cracking of the Enigma cipher was made possible by the use of brute-force machines derived from Polish-built Bombas, Bombes. These were electromechanical machines, and certainly not general computers. A more programmable machine, that was much more like what we think of as a computer today, was built in December 1943, and given the suitably awe-inspiring name “Colossus”. This was used to decrypt ciphers produced by the high-security Lorenz machine, and incidentally was the first electronic digital computer.
With computers came the concept of feasible brute-force attacks on a scale that was hitherto unprecedented. However, it also permitted the encoding of messages with equally high levels of encryption. The scale and scope of these new methods of encryption and cryptanalysis are far beyond the remit of this article – it should be enough to note that the vast majority of the field of cryptology is now based on rigorous, high level mathematics.
Cryptography And The Internet
Security on the Internet is still a poorly understood area. Well-intentioned ‘grey hat’ security researchers take great risks by doing what they do, as small breaches in protocol can result in legal action from certain companies.
Data access is an especially vexed area for many companies, as the lines between what should and should not be publicly available become difficult to draw.
Not so long ago, even innocuous habits such as deep linking (effectively, linking to pages of a website other than the homepage or certain category folders, a practice so commonplace now that people generally forget it has a name) have been considered problematic. By contrast, today, it has been suggested that any data that is not deliberately restricted by code could be fair game. The line between what is made available and what is accidentally available has been shown to be blurry, making the work of computer security experts more important than ever.
As data in general becomes more and more important to business, we can see a change in how very large businesses protect their (your) data. Big companies like Facebook, Microsoft and Google are definitely keen on solid cryptography, making them hard to get information from, assuming you’ve not got insider access. That doesn’t mean your data is completely private, but at least you can assume that it’s as private as the company wants it to be!
This all makes cryptography extremely important to companies, as concerns about privacy, the value of data, and the law are all changing to make encryption more important as an explicit layer of protection around your company’s data.
Cryptographic Research And Development
Image by Emmanuel Boutet
Cryptographic (and cryptanalytic) research and development is currently operating beyond the cutting-edge. While a quantum cryptographic scheme has yet to be implemented on a grand scale, research into post-quantum cryptography is already in full flow. This means that as long as the mind-blowing work of academics continues to outpace technology, there is ample room for commercial research and development to resolve scientific and technical uncertainties in their wake.
For example, while theoretical optimums are reached in terms of security, encryption efficiency, and other concerns, security researchers in the field may be able to find best-in-practice compromises between theoretical concerns. Similarly, while an encryption technique (such as MD5) might be theoretically broken early on, it is generally a little while longer before it is practical to break it, and even longer before it is actually broken ‘in the wild’. Each of these stages requires a great deal of work from researchers, usually making multiple small breakthroughs at first before the system is completely broken. Even after the system is broken, attackers may continue to use it – and researching malware to identify the cryptographic system used continues to occur, and could easily qualify as ‘resolution of technological uncertainty’.
Even more practically, there may be research and development involved in the work of a company that just handles the implementation of cryptosystems. The majority of successful decryptions are the result of side-channel attacks (attacks based on implementation details), social engineering or brute force rather than a direct attack on the cipher being used. Research into minimising the viability of side-channel attacks, especially timing attacks and power-usage attacks, is just as important in practice as the most cutting-edge research into theoretical cryptosystems.
Computer security specialists and computer security companies often engage in research and development of this nature, and may be eligible for R&D tax credits from HMRC.
Security Research And Development
Keeping up to date with cryptology barely scratches the surface of what a good security company will offer, and the amount of research they need to put in to remain at the forefront of their industry. Cryptology just makes a particularly good example of the constant research and development required to stay on top, as the constant development of new methods of encrypting and attacking data displays.
From the work we’ve done with dedicated security experts such as CNS Group in the past, we know first hand that there’s a vast amount of work they do that qualifies for HMRC Research & Development Tax Relief. This is not limited by theoretical research; a lot of hands-on research and analysis of the latest attacks may involve qualifying research and development as well.
Non-standard Encryption and Cryptography Technologies
Development projects adopting standard methods of encryption and routine cryptography techniques would be unlikely to qualify for R&D Tax Relief if the implementation could be seen as routine or standard. The development and implementation of encryption or security techniques that do not follow established methodologies however, would most certainly be of interest. If non-standard ‘cryptography’ has been implemented, incorporating the use of proprietary or unpublished cryptographic functionality, then it is highly likely that some of the costs incurred can be recovered from HMRC. The focus is typically, but not solely restricted to, the development of encryption algorithms or protocols which have yet to be recognised by an international standards body, such as The GSMA, IEEE, ISO, ITU, 3GPP, IETF or The TIA.
If you work in information security, and the work you do could be described as anything but ‘standard’, then the work you undertake could qualify for R&D tax relief. For an SME, there is the potential to recover up to 1/3rd of qualifying expenditure from HMRC, so the security headaches you’ve faced could result in a sizeable cash injection. Get in touch to see if we can help and find out more about your options!
* The Zimmerman Telegram was actually encoded with two methods, primarily a high security naval code called “code 0075”, a “lotteriechiffre”, which was partly decoded first, and elsewhere “code 13040”, a less secure cipher which was tracked down and decrypted so that the German military wouldn’t realise “code 0075” was compromised and possibly filled in the gaps left by the partial translation of “code 0075”.