In the 1957 case Kent et al. v. Dulles, Secretary of State, the U.S Supreme Court ruled that the right to travel is a part of "liberty" as outlined in the Fifth Amendment and thus cannot be impeded without due process of law. The plaintiffs were two American citizens who had been denied passports by then-Secretary of State John Foster Dulles because they refused to sign non-communist affidavits as required by departmental regulations at that time. They argued this violated their constitutional rights under both First and Fifth Amendments. In a 5-4 decision, Justice William O Douglas delivered an opinion stating that freedom to travel is indeed protected by Constitution unless national security or foreign policy interests are at stake; however, these restrictions must be explicitly stated in law passed by Congress rather than being left up to discretion of executive branch officials.

In the dissenting opinion for Kent et al. v. Dulles, Justice Clark argued that the majority's decision was based on an incorrect interpretation of the Passport Act of 1926 and a misunderstanding of executive power in foreign affairs. He contended that Congress had implicitly granted the Secretary of State authority to deny passports for national security reasons when it passed legislation requiring citizens traveling abroad to carry them. Furthermore, he asserted that this case did not involve any constitutional rights because international travel is a privilege rather than a right guaranteed by law or constitution. Therefore, according to him, there was no violation of due process or freedom of speech involved in denying passports to communists during Cold War tensions with Soviet Union as they could pose potential threats to national security.