In the 1980 case Haig, Secretary of State v. Agee, the U.S. Supreme Court ruled in favor of the government's right to revoke a citizen's passport on grounds of national security and foreign policy considerations. Philip Agee was a former CIA agent who had begun revealing classified information about agency operations after leaving his position. The State Department revoked his passport in 1979 under its statutory authority to deny passports for reasons of national security or foreign policy, arguing that Agee’s actions were causing significant damage to U.S interests abroad by endangering lives and disrupting relations with other countries. In response, Agee claimed this violated his Fifth Amendment due process rights and First Amendment free speech rights. The court held that while travel is indeed protected under the Fifth Amendment as an aspect of liberty, it does not constrain Congress from regulating international travel within constitutional limits set by due process - which they found were met here given sufficient notice and opportunity for hearing provided before revocation occurred. Furthermore, they rejected Agee’s First Amendment claim stating that though freedom of speech is highly valued; it doesn't protect actions having potentially grave consequences like those caused by him disclosing classified information.

In the dissenting opinion for Haig v. Agee, Justice Brennan argued that the majority's decision to uphold the revocation of Agee's passport on national security grounds was a violation of his First Amendment rights. He contended that such an action constituted prior restraint on speech and should be subjected to strict scrutiny, which it would not withstand in this case. Moreover, he criticized the majority for deferring too much to executive power without sufficient evidence or justification for doing so. In his view, there were other less restrictive means available to protect national security interests without infriving upon individual constitutional rights as drastically as revoking a citizen’s passport does.