Skip to main content

Following on from last week’s show I look at how the intelligence services who were causing mayhem in Northern Ireland were protected by a culture of institutional secrecy in the British government. Via a little-known government file that was declassified in 2009 I tell the story of government policy on avowing the existence of the security services.

This episode focuses in on the late 1970s when a major shift in that policy – admitting MI6’s existence after 30 years of denial – was contemplated by the Prime Minister. The tale of how this shift was averted involves the Cabinet Secretary Sir John Hunt and a woman who would herself soon be Prime Minister, Margaret Thatcher. By reconstructing the timeline of events I offer a different view of how this potentially major shift in policy was diverted, which resulted in the existence of MI6 not being officially admitted until 1994.


I decided to do an episode on institutional secrecy, which comes up time and again in the comments and in almost any meaningful discussion about anything, these days. I could point you to various examples but there is one that I think is particularly telling, one that you almost certainly won’t already know about and that ties in with the last show because it was going on at the same time. Just as the struggle for Irish independence was turned into an all out ethnic war by the British security services, the Prime Minister and others were working to keep their very existence a secret.

We had four different prime ministers in this country in the 1970s – Ted Heath, Harold Wilson, James Callaghan and Margaret Thatcher. A Tory either end of the decade with Labour occupying the years in the middle, and the government file on this story I’m going to tell you today spans all of those but is mainly from the Callaghan Labour government. But before we get into the file we should establish a few things. The existence of MI5 and MI6 was not officially admitted until 1989 for MI5 and 1994 for MI6. As absurd as that sounds, and as absurd as the clip I played at the top of the show is, that was the official government position during the Cold War.

Now, as that clip from the sitcom Yes, Minister shows, everyone knew that they existed and that the government refusal to admit their existence was a farce, but nonetheless that was the position. The heads of MI5 and MI6 were never named in public, they did not hold press conferences, there were no public reports into their activities, none of that. So it is no surprise that they got up to an awful lot of highly criminal behaviour, and still do. The institutions of oversight and public disclosure haven’t had a chance to catch up, and of course never will.

I highlight this because it is quite different to the American system, where the CIA’s existence was never really a secret, and they’ve always had to participate in the overt government machinery, congressional committees, whatever. Ultimately there’s little difference in the way the two operate, don’t get me wrong, they are very similar institutions, which is no surprise because the CIA is sort of based on MI6, but in this element there is something a difference. Exactly why that is, I don’t have an answer. Or not one that is brief enough to crowbar into this show.

So, the upshot of this is that in this country we have something called the 30 year rule, whereby all government files are turned over to the public records office, now known as the national archives, after 30 years. That is supposed to be coming down to a 20 year rule but progress is slow and while we’re currently supposed to be using a 25 year rule we’re actually, in terms of documents being released, operating at a 28 1/2 year rule. I am being both facetious and deadly serious. I follow all this with great amusement, and I get a lot of good stuff via the national archives. They periodically release a bunch of files and some of them are digitised in PDF format and you can download them from the site. But they are only free to download for a few weeks, after that you have to pay the arbitrary sum of £3.30 per file, whether it is 8 pages or 200 pages. So I keep abreast of what’s coming out and try to download everything while it is still free and dump it in a folder on my hard drive so it is there if I need it later. Indeed, if you want any digitised file from the national archives released in the last three years or so, I’ve probably got a copy so feel free to ask me.

The story I want to tell you today comes courtesy of a little-examined file from the office of the Prime Minister, released on the national archives website back in 2009. I had to pay for this but naturally I’ll make it freely available and there will be a link on the page for this episode on boiling frogs It’s about 140 pages, and the PDF has the papers themselves – the letters, memos and so on, in reverse chronological order, as they are in the original paper file. So it isn’t the easiest PDF to navigate, sadly.

Just before I tell you the story, let’s make sure we’re all on the same page – MI6 is broadly the British equivalent of the CIA, in theory answerable to the Foreign Office. MI5 is the domestic intelligence agency under the command of the Home Office. They don’t really appear in this story. GCHQ is the electronic eavesdropping agency like the NSA. They are answerable to the JIC, the Joint Intelligence Committee who oversee the work of all the agencies. The existence of MI5 was very much an open secret in the Cold War period, but GCHQ, MI6 and the JIC were better protected.

Our story begins in the early 70s, at which time the post-war existence of the intelligence services was not admitted. The first paper in this file from the Prime Minister’s Office is a copy of guidance to government departments on records mentioning the intelligence services. It says in big bold letters: Papers containing references to GCHQ in an intelligence context, or to the existence after 31 December 1945 of the Joint Intelligence Committee or of MI6, should be withheld.

While records from within the intelligence services themselves were not subject to the 30 year rule, there were lots of records in other departments mentioning MI6 or the JIC or that GCHQ was involved in intelligence gathering, or what we now call mass surveillance. In the mid 1970s, shortly before these post-WW2 records would be coming up for release under the 30 year rule, a working group was established to look at this problem. One exchange with the Prime Minister’s office mentions how:


So you get the idea of just how seriously this was taken within the institutions of power, however absurd it might be in reality. If you’ve got hundreds of thousands of civil servants, many of whom have access to these records mentioning MI6 or whatever, then the fact those files aren’t publicly released makes little difference. Civil servants talk, word gets out.

This came to a head during the government of James Callaghan in the late 70s. For one thing he had to consider whether to make publicly available two volumes of an official history of wartime intelligence operations. This is one of those internal histories commissioned by the government, like the CIA’s report on the Bay of Pigs.

The Cabinet Secretary – the head of the civil service Sir John Hunt, was all in favour of publishing the wartime histories. ‘C’, the head of MI6, was against it. The thinking was that if they revealed the names of wartime sources and assets that current and future assets might be in doubt about whether to perform their roles for MI6. The decision was taken to ‘sanitise’ the official history to remove people’s names.

While this was being resolved, the issue of the 30 year rule and the release of documents referring to the security services came up once again. They had to decide whether to publish files from other departments that simply mentioned the existence of MI6 or the JIC, or that GCHQ was an intelligence agency. In turn this meant they would have to have a response to the obvious question of if ‘if MI6 continued to exist after World War 2, does it still exist today in the 1970s?’

One particularly bonkers suggestion posed by the Cabinet Secretary was to have a rolling 30 year rule, i.e. in 1978 they could admit that MI6 existed in 1948, in 1979 they could admit it existed in 1949, and so on. But each time they would deny that it still existed today, presumably all the way up to 2009 when the rolling 30 year rule caught up with itself.

To my mind this is an absolute gem of an idea, the sort of thing only a lifetime bureaucrat could come up with. Everyone knew that MI6 existed, and for a few years after World War 2 the government denied it. Then the position became non-avowal, i.e. they did not admit it existed. The notion that 30 years after denying it existed in 1948 you could simply admit its existence but only up to 1948, and then the following year do the same for 1949, without people assuming that this meant it still existed in the present day, is ludicrous. It is the sort of insanity only someone who lives inside an office, detached from the real world, could or would come up with.

I’m sure of one question that I’ll be asked to elaborate on – who is the Cabinet Secretary? He is the head the civil service, the permanent staff of the government. He is also the primary administrative authority at the highest level of the government – he has permanent access to the Prime Minister, is the prime Minister’s number 1 advisor and is present at every Cabinet meeting. It is always a man, and he is almost always educated at Oxford university for at least part of his higher education. He is, in short, the permanent representative of the deep state within government.

Going back to the ‘rolling 30 year rule’ idea, in particular the response to this suggestion from the Foreign Secretary, who was of course being advised by MI6, is a perfect piece of doublethink. In a message to the Prime Minister he wrote:


The Ministry of Defence had a different view, saying that:


Again you get the idea – the primary concern here was whether admitting that MI6 existed would open the floodgates to other questions, or would actually help in keeping their operations secret. The Cabinet Secretary responded to this difference of opinion by siding with the Foreign Office and MI6, though he did dispute the Foreign Secretary’s claim that the practice of denying MI6’s very existence was ‘the envy of the Americans’. John Hunt wrote that ‘Admiral Turner told me he thought our position was quite indefensible’. I’m assuming that is Stansfield Turner, at that time head of the CIA.

After more back and forth between the Prime Minister and the Foreign Office, the decision was made to admit that MI6 existed, but to continue to pretend that GCHQ weren’t an intelligence agency and that the Foreign Office’s own IRD, Information Research Department, did not exist. Documents from other departments mentioning MI6’s postwar existence could be released. Draft answers to possible questions in parliament were drawn up and the issue was, at least inside the government, settled.

The Prime Minister then wrote to the leaders of the opposition parties David Steel, head of the Liberal Party, and Margaret Thatcher, leader of the Tories. To Steel he only discussed the issue of the publication of the histories of wartime intelligence, but he consulted Thatcher’s view on whether to begin admitting that MI6 existed. Obviously the Tories were the bigger party, but I also think it is because Thatcher was more keenly connected to the deep state, being the wife of an oil baron among other things, than Steel ever was. Perhaps Callaghan saw the writing on the wall, since it was less than a year after he wrote to Thatcher about all this that she defeated him in the general election of 1979.

Thatcher’s response was typically cutting and to the point:


By contrast, David Steel had no objections to publishing the history of wartime intelligence. So the Cabinet Secretary met with Thatcher, who was still adamantly opposed to publishing the histories, saying they never should have been written in the first place (they were commissioned by a Tory Prime Minister, Ted Heath). She also maintained her opposition to any public acknowledgement of the postwar existence of the intelligence services, saying it was the thin end of a very dangerous wedge. As a result of this meeting the Cabinet Secretary suggested to the Prime Minister that they look again at the issue of avowing the existence of MI6, in part because it would affect future governments.

There was more back and forth between the relevant departments and officials and a new position was reached: the old records mentioning MI6 and the JIC would be released, but if anyone asked if this meant that MI6 still existed then those questions would continue to not be answered. In other words, they actually ended up, after all these shenanigans, going with a version of the ‘rolling 30 year rule’ suggested early on by the Cabinet Secretary. This remained the default position until the 1990s, with the Cold War safely out of the way, they could finally admit that MI6 does exist. And within years they were hosting premieres of Bond movies and helping to produce Darwinian reality TV shows, just like their American counterparts. Makes you proud to be British.

But in all seriousness, it is ironic that Thatcher asked whether she really had any say in the matter, because in some ways she did. The Callaghan government had made up their minds, they were going to abandon the fiction that MI6 was shut down at the end of WW2.   Then they asked Thatcher if this was OK, and it was her objections that the Cabinet Secretary used as an excuse to rethink things. Not so ironically, the new plans – to release a few old files but otherwise maintain the fiction – met with the approval of the intelligence services, whereas remember that ‘C’, the head of MI6, always had objections to the original plans for disclosure.

This is deep state logic we’re dealing with here – the government had made up its mind despite the issues raised by the Foreign Secretary, in effect a mouthpiece for MI6 at that point. Then for no obvious reason the Prime Minister asked permission from the opposition leader to pursue the policy they had decided on. Predictably, Thatcher was opposed to Callaghan’s plan, so the Cabinet Secretary intervened and met with Thatcher, reporting back that she was resolute and unwavering in her opposition, and asking that the Prime Minister reconsider the plan that they’d already agreed on. The Prime Minister duly did so, and everyone conveniently agreed with the new plan which then became policy.

And, going back to the clip I played at the start of the show, if any of you are thinking that this sounds an awful lot like an episode of Yes Minister where Sir Humphrey tricks Jim Hacker into doing what he wants the minister to do, then yes, it does. Because that’s actually how the government operates in this country, or at least how it operated in the period we’re talking about. It is one part conspiracy, one part farce. It is absurd and horrifying at the same time.